Aug 13 01:24:42.846328 kernel: Linux version 6.12.40-flatcar (build@pony-truck.infra.kinvolk.io) (x86_64-cros-linux-gnu-gcc (Gentoo Hardened 14.2.1_p20241221 p7) 14.2.1 20241221, GNU ld (Gentoo 2.44 p1) 2.44.0) #1 SMP PREEMPT_DYNAMIC Tue Aug 12 21:42:48 -00 2025 Aug 13 01:24:42.846349 kernel: Command line: BOOT_IMAGE=/flatcar/vmlinuz-a mount.usr=/dev/mapper/usr verity.usr=PARTUUID=7130c94a-213a-4e5a-8e26-6cce9662f132 rootflags=rw mount.usrflags=ro consoleblank=0 root=LABEL=ROOT console=ttyS0,115200n8 console=tty0 flatcar.first_boot=detected flatcar.oem.id=akamai verity.usrhash=215bdedb8de38f6b96ec4f9db80853e25015f60454b867e319fdcb9244320a21 Aug 13 01:24:42.846357 kernel: BIOS-provided physical RAM map: Aug 13 01:24:42.846365 kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009f7ff] usable Aug 13 01:24:42.846371 kernel: BIOS-e820: [mem 0x000000000009f800-0x000000000009ffff] reserved Aug 13 01:24:42.846376 kernel: BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved Aug 13 01:24:42.846382 kernel: BIOS-e820: [mem 0x0000000000100000-0x000000007ffdcfff] usable Aug 13 01:24:42.846388 kernel: BIOS-e820: [mem 0x000000007ffdd000-0x000000007fffffff] reserved Aug 13 01:24:42.846394 kernel: BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved Aug 13 01:24:42.846399 kernel: BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved Aug 13 01:24:42.846405 kernel: BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved Aug 13 01:24:42.846410 kernel: BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved Aug 13 01:24:42.846418 kernel: BIOS-e820: [mem 0x0000000100000000-0x000000017fffffff] usable Aug 13 01:24:42.846423 kernel: BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved Aug 13 01:24:42.846430 kernel: NX (Execute Disable) protection: active Aug 13 01:24:42.846436 kernel: APIC: Static calls initialized Aug 13 01:24:42.846442 kernel: SMBIOS 2.8 present. Aug 13 01:24:42.846450 kernel: DMI: Linode Compute Instance/Standard PC (Q35 + ICH9, 2009), BIOS Not Specified Aug 13 01:24:42.846456 kernel: DMI: Memory slots populated: 1/1 Aug 13 01:24:42.846461 kernel: Hypervisor detected: KVM Aug 13 01:24:42.846467 kernel: kvm-clock: Using msrs 4b564d01 and 4b564d00 Aug 13 01:24:42.846473 kernel: kvm-clock: using sched offset of 5784245764 cycles Aug 13 01:24:42.846480 kernel: clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns Aug 13 01:24:42.846486 kernel: tsc: Detected 1999.999 MHz processor Aug 13 01:24:42.846492 kernel: e820: update [mem 0x00000000-0x00000fff] usable ==> reserved Aug 13 01:24:42.846499 kernel: e820: remove [mem 0x000a0000-0x000fffff] usable Aug 13 01:24:42.846505 kernel: last_pfn = 0x180000 max_arch_pfn = 0x400000000 Aug 13 01:24:42.846513 kernel: MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs Aug 13 01:24:42.846520 kernel: x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT Aug 13 01:24:42.846526 kernel: last_pfn = 0x7ffdd max_arch_pfn = 0x400000000 Aug 13 01:24:42.846532 kernel: Using GB pages for direct mapping Aug 13 01:24:42.846537 kernel: ACPI: Early table checksum verification disabled Aug 13 01:24:42.846543 kernel: ACPI: RSDP 0x00000000000F5160 000014 (v00 BOCHS ) Aug 13 01:24:42.846549 kernel: ACPI: RSDT 0x000000007FFE2307 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) Aug 13 01:24:42.846556 kernel: ACPI: FACP 0x000000007FFE20F7 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001) Aug 13 01:24:42.846562 kernel: ACPI: DSDT 0x000000007FFE0040 0020B7 (v01 BOCHS BXPC 00000001 BXPC 00000001) Aug 13 01:24:42.846570 kernel: ACPI: FACS 0x000000007FFE0000 000040 Aug 13 01:24:42.846576 kernel: ACPI: APIC 0x000000007FFE21EB 000080 (v01 BOCHS BXPC 00000001 BXPC 00000001) Aug 13 01:24:42.846582 kernel: ACPI: HPET 0x000000007FFE226B 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) Aug 13 01:24:42.846588 kernel: ACPI: MCFG 0x000000007FFE22A3 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001) Aug 13 01:24:42.846597 kernel: ACPI: WAET 0x000000007FFE22DF 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) Aug 13 01:24:42.846603 kernel: ACPI: Reserving FACP table memory at [mem 0x7ffe20f7-0x7ffe21ea] Aug 13 01:24:42.846611 kernel: ACPI: Reserving DSDT table memory at [mem 0x7ffe0040-0x7ffe20f6] Aug 13 01:24:42.846618 kernel: ACPI: Reserving FACS table memory at [mem 0x7ffe0000-0x7ffe003f] Aug 13 01:24:42.846624 kernel: ACPI: Reserving APIC table memory at [mem 0x7ffe21eb-0x7ffe226a] Aug 13 01:24:42.846631 kernel: ACPI: Reserving HPET table memory at [mem 0x7ffe226b-0x7ffe22a2] Aug 13 01:24:42.846637 kernel: ACPI: Reserving MCFG table memory at [mem 0x7ffe22a3-0x7ffe22de] Aug 13 01:24:42.846643 kernel: ACPI: Reserving WAET table memory at [mem 0x7ffe22df-0x7ffe2306] Aug 13 01:24:42.846649 kernel: No NUMA configuration found Aug 13 01:24:42.846655 kernel: Faking a node at [mem 0x0000000000000000-0x000000017fffffff] Aug 13 01:24:42.846664 kernel: NODE_DATA(0) allocated [mem 0x17fff8dc0-0x17fffffff] Aug 13 01:24:42.846670 kernel: Zone ranges: Aug 13 01:24:42.846676 kernel: DMA [mem 0x0000000000001000-0x0000000000ffffff] Aug 13 01:24:42.846683 kernel: DMA32 [mem 0x0000000001000000-0x00000000ffffffff] Aug 13 01:24:42.846689 kernel: Normal [mem 0x0000000100000000-0x000000017fffffff] Aug 13 01:24:42.846695 kernel: Device empty Aug 13 01:24:42.846701 kernel: Movable zone start for each node Aug 13 01:24:42.846707 kernel: Early memory node ranges Aug 13 01:24:42.846714 kernel: node 0: [mem 0x0000000000001000-0x000000000009efff] Aug 13 01:24:42.846720 kernel: node 0: [mem 0x0000000000100000-0x000000007ffdcfff] Aug 13 01:24:42.846728 kernel: node 0: [mem 0x0000000100000000-0x000000017fffffff] Aug 13 01:24:42.846734 kernel: Initmem setup node 0 [mem 0x0000000000001000-0x000000017fffffff] Aug 13 01:24:42.846741 kernel: On node 0, zone DMA: 1 pages in unavailable ranges Aug 13 01:24:42.846747 kernel: On node 0, zone DMA: 97 pages in unavailable ranges Aug 13 01:24:42.846753 kernel: On node 0, zone Normal: 35 pages in unavailable ranges Aug 13 01:24:42.846759 kernel: ACPI: PM-Timer IO Port: 0x608 Aug 13 01:24:42.846766 kernel: ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) Aug 13 01:24:42.846772 kernel: IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 Aug 13 01:24:42.846778 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) Aug 13 01:24:42.846786 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) Aug 13 01:24:42.846793 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) Aug 13 01:24:42.846799 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) Aug 13 01:24:42.846805 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) Aug 13 01:24:42.846811 kernel: ACPI: Using ACPI (MADT) for SMP configuration information Aug 13 01:24:42.846818 kernel: ACPI: HPET id: 0x8086a201 base: 0xfed00000 Aug 13 01:24:42.846824 kernel: TSC deadline timer available Aug 13 01:24:42.846830 kernel: CPU topo: Max. logical packages: 1 Aug 13 01:24:42.846836 kernel: CPU topo: Max. logical dies: 1 Aug 13 01:24:42.846844 kernel: CPU topo: Max. dies per package: 1 Aug 13 01:24:42.846850 kernel: CPU topo: Max. threads per core: 1 Aug 13 01:24:42.849380 kernel: CPU topo: Num. cores per package: 2 Aug 13 01:24:42.849392 kernel: CPU topo: Num. threads per package: 2 Aug 13 01:24:42.849399 kernel: CPU topo: Allowing 2 present CPUs plus 0 hotplug CPUs Aug 13 01:24:42.849405 kernel: kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() Aug 13 01:24:42.849412 kernel: kvm-guest: KVM setup pv remote TLB flush Aug 13 01:24:42.849418 kernel: kvm-guest: setup PV sched yield Aug 13 01:24:42.849425 kernel: [mem 0xc0000000-0xfed1bfff] available for PCI devices Aug 13 01:24:42.849434 kernel: Booting paravirtualized kernel on KVM Aug 13 01:24:42.849441 kernel: clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns Aug 13 01:24:42.849447 kernel: setup_percpu: NR_CPUS:512 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1 Aug 13 01:24:42.849453 kernel: percpu: Embedded 60 pages/cpu s207832 r8192 d29736 u1048576 Aug 13 01:24:42.849460 kernel: pcpu-alloc: s207832 r8192 d29736 u1048576 alloc=1*2097152 Aug 13 01:24:42.849466 kernel: pcpu-alloc: [0] 0 1 Aug 13 01:24:42.849472 kernel: kvm-guest: PV spinlocks enabled Aug 13 01:24:42.849479 kernel: PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) Aug 13 01:24:42.849486 kernel: Kernel command line: rootflags=rw mount.usrflags=ro BOOT_IMAGE=/flatcar/vmlinuz-a mount.usr=/dev/mapper/usr verity.usr=PARTUUID=7130c94a-213a-4e5a-8e26-6cce9662f132 rootflags=rw mount.usrflags=ro consoleblank=0 root=LABEL=ROOT console=ttyS0,115200n8 console=tty0 flatcar.first_boot=detected flatcar.oem.id=akamai verity.usrhash=215bdedb8de38f6b96ec4f9db80853e25015f60454b867e319fdcb9244320a21 Aug 13 01:24:42.849495 kernel: Unknown kernel command line parameters "BOOT_IMAGE=/flatcar/vmlinuz-a", will be passed to user space. Aug 13 01:24:42.849501 kernel: random: crng init done Aug 13 01:24:42.849507 kernel: Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) Aug 13 01:24:42.849513 kernel: Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Aug 13 01:24:42.849520 kernel: Fallback order for Node 0: 0 Aug 13 01:24:42.849526 kernel: Built 1 zonelists, mobility grouping on. Total pages: 1048443 Aug 13 01:24:42.849532 kernel: Policy zone: Normal Aug 13 01:24:42.849539 kernel: mem auto-init: stack:off, heap alloc:off, heap free:off Aug 13 01:24:42.849547 kernel: software IO TLB: area num 2. Aug 13 01:24:42.849553 kernel: SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 Aug 13 01:24:42.849559 kernel: ftrace: allocating 40098 entries in 157 pages Aug 13 01:24:42.849565 kernel: ftrace: allocated 157 pages with 5 groups Aug 13 01:24:42.849572 kernel: Dynamic Preempt: voluntary Aug 13 01:24:42.849578 kernel: rcu: Preemptible hierarchical RCU implementation. Aug 13 01:24:42.849585 kernel: rcu: RCU event tracing is enabled. Aug 13 01:24:42.849592 kernel: rcu: RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=2. Aug 13 01:24:42.849598 kernel: Trampoline variant of Tasks RCU enabled. Aug 13 01:24:42.849606 kernel: Rude variant of Tasks RCU enabled. Aug 13 01:24:42.849613 kernel: Tracing variant of Tasks RCU enabled. Aug 13 01:24:42.849619 kernel: rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. Aug 13 01:24:42.849626 kernel: rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 Aug 13 01:24:42.849632 kernel: RCU Tasks: Setting shift to 1 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=2. Aug 13 01:24:42.849645 kernel: RCU Tasks Rude: Setting shift to 1 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=2. Aug 13 01:24:42.849653 kernel: RCU Tasks Trace: Setting shift to 1 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=2. Aug 13 01:24:42.849660 kernel: NR_IRQS: 33024, nr_irqs: 440, preallocated irqs: 16 Aug 13 01:24:42.849666 kernel: rcu: srcu_init: Setting srcu_struct sizes based on contention. Aug 13 01:24:42.849673 kernel: Console: colour VGA+ 80x25 Aug 13 01:24:42.849680 kernel: printk: legacy console [tty0] enabled Aug 13 01:24:42.849686 kernel: printk: legacy console [ttyS0] enabled Aug 13 01:24:42.849695 kernel: ACPI: Core revision 20240827 Aug 13 01:24:42.849701 kernel: clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns Aug 13 01:24:42.849708 kernel: APIC: Switch to symmetric I/O mode setup Aug 13 01:24:42.849715 kernel: x2apic enabled Aug 13 01:24:42.849721 kernel: APIC: Switched APIC routing to: physical x2apic Aug 13 01:24:42.849730 kernel: kvm-guest: APIC: send_IPI_mask() replaced with kvm_send_ipi_mask() Aug 13 01:24:42.849736 kernel: kvm-guest: APIC: send_IPI_mask_allbutself() replaced with kvm_send_ipi_mask_allbutself() Aug 13 01:24:42.849743 kernel: kvm-guest: setup PV IPIs Aug 13 01:24:42.849750 kernel: ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 Aug 13 01:24:42.849756 kernel: clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x39a85afc727, max_idle_ns: 881590685098 ns Aug 13 01:24:42.849763 kernel: Calibrating delay loop (skipped) preset value.. 3999.99 BogoMIPS (lpj=1999999) Aug 13 01:24:42.849770 kernel: x86/cpu: User Mode Instruction Prevention (UMIP) activated Aug 13 01:24:42.849776 kernel: Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127 Aug 13 01:24:42.849783 kernel: Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0 Aug 13 01:24:42.849791 kernel: Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization Aug 13 01:24:42.849798 kernel: Spectre V2 : Mitigation: Retpolines Aug 13 01:24:42.849804 kernel: Spectre V2 : Spectre v2 / SpectreRSB: Filling RSB on context switch and VMEXIT Aug 13 01:24:42.849811 kernel: Spectre V2 : Enabling Restricted Speculation for firmware calls Aug 13 01:24:42.849818 kernel: Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier Aug 13 01:24:42.849824 kernel: Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl Aug 13 01:24:42.849831 kernel: Speculative Return Stack Overflow: IBPB-extending microcode not applied! Aug 13 01:24:42.849838 kernel: Speculative Return Stack Overflow: WARNING: See https://kernel.org/doc/html/latest/admin-guide/hw-vuln/srso.html for mitigation options. Aug 13 01:24:42.849847 kernel: Speculative Return Stack Overflow: Vulnerable: Safe RET, no microcode Aug 13 01:24:42.849853 kernel: Transient Scheduler Attacks: Forcing mitigation on in a VM Aug 13 01:24:42.849876 kernel: Transient Scheduler Attacks: Vulnerable: Clear CPU buffers attempted, no microcode Aug 13 01:24:42.849883 kernel: x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' Aug 13 01:24:42.849889 kernel: x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' Aug 13 01:24:42.849896 kernel: x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' Aug 13 01:24:42.849902 kernel: x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers' Aug 13 01:24:42.849909 kernel: x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 Aug 13 01:24:42.849916 kernel: x86/fpu: xstate_offset[9]: 832, xstate_sizes[9]: 8 Aug 13 01:24:42.849925 kernel: x86/fpu: Enabled xstate features 0x207, context size is 840 bytes, using 'compacted' format. Aug 13 01:24:42.849931 kernel: Freeing SMP alternatives memory: 32K Aug 13 01:24:42.849938 kernel: pid_max: default: 32768 minimum: 301 Aug 13 01:24:42.849944 kernel: LSM: initializing lsm=lockdown,capability,landlock,selinux,ima Aug 13 01:24:42.849951 kernel: landlock: Up and running. Aug 13 01:24:42.849957 kernel: SELinux: Initializing. Aug 13 01:24:42.849964 kernel: Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear) Aug 13 01:24:42.849971 kernel: Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear) Aug 13 01:24:42.849978 kernel: smpboot: CPU0: AMD EPYC 7713 64-Core Processor (family: 0x19, model: 0x1, stepping: 0x1) Aug 13 01:24:42.849986 kernel: Performance Events: Fam17h+ core perfctr, AMD PMU driver. Aug 13 01:24:42.849993 kernel: ... version: 0 Aug 13 01:24:42.849999 kernel: ... bit width: 48 Aug 13 01:24:42.850006 kernel: ... generic registers: 6 Aug 13 01:24:42.850012 kernel: ... value mask: 0000ffffffffffff Aug 13 01:24:42.850019 kernel: ... max period: 00007fffffffffff Aug 13 01:24:42.850025 kernel: ... fixed-purpose events: 0 Aug 13 01:24:42.850032 kernel: ... event mask: 000000000000003f Aug 13 01:24:42.850038 kernel: signal: max sigframe size: 3376 Aug 13 01:24:42.850047 kernel: rcu: Hierarchical SRCU implementation. Aug 13 01:24:42.850053 kernel: rcu: Max phase no-delay instances is 400. Aug 13 01:24:42.850060 kernel: Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level Aug 13 01:24:42.850067 kernel: smp: Bringing up secondary CPUs ... Aug 13 01:24:42.850073 kernel: smpboot: x86: Booting SMP configuration: Aug 13 01:24:42.850080 kernel: .... node #0, CPUs: #1 Aug 13 01:24:42.850086 kernel: smp: Brought up 1 node, 2 CPUs Aug 13 01:24:42.850093 kernel: smpboot: Total of 2 processors activated (7999.99 BogoMIPS) Aug 13 01:24:42.850100 kernel: Memory: 3961808K/4193772K available (14336K kernel code, 2430K rwdata, 9960K rodata, 54444K init, 2524K bss, 227288K reserved, 0K cma-reserved) Aug 13 01:24:42.850108 kernel: devtmpfs: initialized Aug 13 01:24:42.850115 kernel: x86/mm: Memory block size: 128MB Aug 13 01:24:42.850122 kernel: clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns Aug 13 01:24:42.850128 kernel: futex hash table entries: 512 (order: 3, 32768 bytes, linear) Aug 13 01:24:42.850135 kernel: pinctrl core: initialized pinctrl subsystem Aug 13 01:24:42.850141 kernel: NET: Registered PF_NETLINK/PF_ROUTE protocol family Aug 13 01:24:42.850148 kernel: audit: initializing netlink subsys (disabled) Aug 13 01:24:42.850154 kernel: audit: type=2000 audit(1755048280.413:1): state=initialized audit_enabled=0 res=1 Aug 13 01:24:42.850161 kernel: thermal_sys: Registered thermal governor 'step_wise' Aug 13 01:24:42.850170 kernel: thermal_sys: Registered thermal governor 'user_space' Aug 13 01:24:42.850176 kernel: cpuidle: using governor menu Aug 13 01:24:42.850183 kernel: acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 Aug 13 01:24:42.850190 kernel: dca service started, version 1.12.1 Aug 13 01:24:42.850196 kernel: PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff] Aug 13 01:24:42.850203 kernel: PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry Aug 13 01:24:42.850209 kernel: PCI: Using configuration type 1 for base access Aug 13 01:24:42.850216 kernel: kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. Aug 13 01:24:42.850223 kernel: HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages Aug 13 01:24:42.850231 kernel: HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page Aug 13 01:24:42.850238 kernel: HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages Aug 13 01:24:42.850244 kernel: HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page Aug 13 01:24:42.850251 kernel: ACPI: Added _OSI(Module Device) Aug 13 01:24:42.850257 kernel: ACPI: Added _OSI(Processor Device) Aug 13 01:24:42.850264 kernel: ACPI: Added _OSI(Processor Aggregator Device) Aug 13 01:24:42.850270 kernel: ACPI: 1 ACPI AML tables successfully acquired and loaded Aug 13 01:24:42.850277 kernel: ACPI: Interpreter enabled Aug 13 01:24:42.850283 kernel: ACPI: PM: (supports S0 S3 S5) Aug 13 01:24:42.850292 kernel: ACPI: Using IOAPIC for interrupt routing Aug 13 01:24:42.850298 kernel: PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug Aug 13 01:24:42.850305 kernel: PCI: Using E820 reservations for host bridge windows Aug 13 01:24:42.850312 kernel: ACPI: Enabled 2 GPEs in block 00 to 3F Aug 13 01:24:42.850318 kernel: ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) Aug 13 01:24:42.850487 kernel: acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3] Aug 13 01:24:42.850601 kernel: acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR] Aug 13 01:24:42.850722 kernel: acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability] Aug 13 01:24:42.850731 kernel: PCI host bridge to bus 0000:00 Aug 13 01:24:42.850846 kernel: pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] Aug 13 01:24:42.852112 kernel: pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] Aug 13 01:24:42.852216 kernel: pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] Aug 13 01:24:42.852314 kernel: pci_bus 0000:00: root bus resource [mem 0x80000000-0xafffffff window] Aug 13 01:24:42.852409 kernel: pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] Aug 13 01:24:42.852510 kernel: pci_bus 0000:00: root bus resource [mem 0x180000000-0x97fffffff window] Aug 13 01:24:42.852605 kernel: pci_bus 0000:00: root bus resource [bus 00-ff] Aug 13 01:24:42.852736 kernel: pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint Aug 13 01:24:42.852875 kernel: pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint Aug 13 01:24:42.852991 kernel: pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref] Aug 13 01:24:42.853097 kernel: pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff] Aug 13 01:24:42.853202 kernel: pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref] Aug 13 01:24:42.853312 kernel: pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] Aug 13 01:24:42.853430 kernel: pci 0000:00:02.0: [1af4:1004] type 00 class 0x010000 conventional PCI endpoint Aug 13 01:24:42.853538 kernel: pci 0000:00:02.0: BAR 0 [io 0xc000-0xc03f] Aug 13 01:24:42.853643 kernel: pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff] Aug 13 01:24:42.853748 kernel: pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref] Aug 13 01:24:42.856895 kernel: pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint Aug 13 01:24:42.857032 kernel: pci 0000:00:03.0: BAR 0 [io 0xc040-0xc07f] Aug 13 01:24:42.857145 kernel: pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff] Aug 13 01:24:42.857273 kernel: pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref] Aug 13 01:24:42.857382 kernel: pci 0000:00:03.0: ROM [mem 0xfeb80000-0xfebbffff pref] Aug 13 01:24:42.857505 kernel: pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint Aug 13 01:24:42.857613 kernel: pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO Aug 13 01:24:42.857730 kernel: pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint Aug 13 01:24:42.857854 kernel: pci 0000:00:1f.2: BAR 4 [io 0xc0c0-0xc0df] Aug 13 01:24:42.858056 kernel: pci 0000:00:1f.2: BAR 5 [mem 0xfebd3000-0xfebd3fff] Aug 13 01:24:42.858177 kernel: pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint Aug 13 01:24:42.858285 kernel: pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f] Aug 13 01:24:42.858295 kernel: ACPI: PCI: Interrupt link LNKA configured for IRQ 10 Aug 13 01:24:42.858302 kernel: ACPI: PCI: Interrupt link LNKB configured for IRQ 10 Aug 13 01:24:42.858309 kernel: ACPI: PCI: Interrupt link LNKC configured for IRQ 11 Aug 13 01:24:42.858319 kernel: ACPI: PCI: Interrupt link LNKD configured for IRQ 11 Aug 13 01:24:42.858326 kernel: ACPI: PCI: Interrupt link LNKE configured for IRQ 10 Aug 13 01:24:42.858333 kernel: ACPI: PCI: Interrupt link LNKF configured for IRQ 10 Aug 13 01:24:42.858339 kernel: ACPI: PCI: Interrupt link LNKG configured for IRQ 11 Aug 13 01:24:42.858346 kernel: ACPI: PCI: Interrupt link LNKH configured for IRQ 11 Aug 13 01:24:42.858353 kernel: ACPI: PCI: Interrupt link GSIA configured for IRQ 16 Aug 13 01:24:42.858360 kernel: ACPI: PCI: Interrupt link GSIB configured for IRQ 17 Aug 13 01:24:42.858366 kernel: ACPI: PCI: Interrupt link GSIC configured for IRQ 18 Aug 13 01:24:42.858373 kernel: ACPI: PCI: Interrupt link GSID configured for IRQ 19 Aug 13 01:24:42.858381 kernel: ACPI: PCI: Interrupt link GSIE configured for IRQ 20 Aug 13 01:24:42.858388 kernel: ACPI: PCI: Interrupt link GSIF configured for IRQ 21 Aug 13 01:24:42.858395 kernel: ACPI: PCI: Interrupt link GSIG configured for IRQ 22 Aug 13 01:24:42.858401 kernel: ACPI: PCI: Interrupt link GSIH configured for IRQ 23 Aug 13 01:24:42.858408 kernel: iommu: Default domain type: Translated Aug 13 01:24:42.858415 kernel: iommu: DMA domain TLB invalidation policy: lazy mode Aug 13 01:24:42.858421 kernel: PCI: Using ACPI for IRQ routing Aug 13 01:24:42.858428 kernel: PCI: pci_cache_line_size set to 64 bytes Aug 13 01:24:42.858434 kernel: e820: reserve RAM buffer [mem 0x0009f800-0x0009ffff] Aug 13 01:24:42.858443 kernel: e820: reserve RAM buffer [mem 0x7ffdd000-0x7fffffff] Aug 13 01:24:42.858550 kernel: pci 0000:00:01.0: vgaarb: setting as boot VGA device Aug 13 01:24:42.858655 kernel: pci 0000:00:01.0: vgaarb: bridge control possible Aug 13 01:24:42.858761 kernel: pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none Aug 13 01:24:42.858770 kernel: vgaarb: loaded Aug 13 01:24:42.858777 kernel: hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 Aug 13 01:24:42.858784 kernel: hpet0: 3 comparators, 64-bit 100.000000 MHz counter Aug 13 01:24:42.858791 kernel: clocksource: Switched to clocksource kvm-clock Aug 13 01:24:42.858798 kernel: VFS: Disk quotas dquot_6.6.0 Aug 13 01:24:42.858807 kernel: VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) Aug 13 01:24:42.858814 kernel: pnp: PnP ACPI init Aug 13 01:24:42.860971 kernel: system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved Aug 13 01:24:42.860985 kernel: pnp: PnP ACPI: found 5 devices Aug 13 01:24:42.860993 kernel: clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns Aug 13 01:24:42.861000 kernel: NET: Registered PF_INET protocol family Aug 13 01:24:42.861007 kernel: IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear) Aug 13 01:24:42.861013 kernel: tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear) Aug 13 01:24:42.861024 kernel: Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) Aug 13 01:24:42.861031 kernel: TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear) Aug 13 01:24:42.861037 kernel: TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear) Aug 13 01:24:42.861044 kernel: TCP: Hash tables configured (established 32768 bind 32768) Aug 13 01:24:42.861051 kernel: UDP hash table entries: 2048 (order: 4, 65536 bytes, linear) Aug 13 01:24:42.861057 kernel: UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear) Aug 13 01:24:42.861064 kernel: NET: Registered PF_UNIX/PF_LOCAL protocol family Aug 13 01:24:42.861071 kernel: NET: Registered PF_XDP protocol family Aug 13 01:24:42.861174 kernel: pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] Aug 13 01:24:42.861277 kernel: pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] Aug 13 01:24:42.861507 kernel: pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] Aug 13 01:24:42.861603 kernel: pci_bus 0000:00: resource 7 [mem 0x80000000-0xafffffff window] Aug 13 01:24:42.861699 kernel: pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window] Aug 13 01:24:42.861795 kernel: pci_bus 0000:00: resource 9 [mem 0x180000000-0x97fffffff window] Aug 13 01:24:42.861804 kernel: PCI: CLS 0 bytes, default 64 Aug 13 01:24:42.861810 kernel: PCI-DMA: Using software bounce buffering for IO (SWIOTLB) Aug 13 01:24:42.861817 kernel: software IO TLB: mapped [mem 0x000000007bfdd000-0x000000007ffdd000] (64MB) Aug 13 01:24:42.861827 kernel: clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x39a85afc727, max_idle_ns: 881590685098 ns Aug 13 01:24:42.861833 kernel: Initialise system trusted keyrings Aug 13 01:24:42.861840 kernel: workingset: timestamp_bits=39 max_order=20 bucket_order=0 Aug 13 01:24:42.861847 kernel: Key type asymmetric registered Aug 13 01:24:42.861853 kernel: Asymmetric key parser 'x509' registered Aug 13 01:24:42.861880 kernel: Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250) Aug 13 01:24:42.861886 kernel: io scheduler mq-deadline registered Aug 13 01:24:42.861893 kernel: io scheduler kyber registered Aug 13 01:24:42.861900 kernel: io scheduler bfq registered Aug 13 01:24:42.861909 kernel: ioatdma: Intel(R) QuickData Technology Driver 5.00 Aug 13 01:24:42.861916 kernel: ACPI: \_SB_.GSIG: Enabled at IRQ 22 Aug 13 01:24:42.861924 kernel: ACPI: \_SB_.GSIH: Enabled at IRQ 23 Aug 13 01:24:42.861930 kernel: Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled Aug 13 01:24:42.861937 kernel: 00:02: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A Aug 13 01:24:42.861944 kernel: i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 Aug 13 01:24:42.861950 kernel: serio: i8042 KBD port at 0x60,0x64 irq 1 Aug 13 01:24:42.861957 kernel: serio: i8042 AUX port at 0x60,0x64 irq 12 Aug 13 01:24:42.862076 kernel: rtc_cmos 00:03: RTC can wake from S4 Aug 13 01:24:42.862089 kernel: input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 Aug 13 01:24:42.862190 kernel: rtc_cmos 00:03: registered as rtc0 Aug 13 01:24:42.862291 kernel: rtc_cmos 00:03: setting system clock to 2025-08-13T01:24:42 UTC (1755048282) Aug 13 01:24:42.862433 kernel: rtc_cmos 00:03: alarms up to one day, y3k, 242 bytes nvram, hpet irqs Aug 13 01:24:42.862442 kernel: amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled Aug 13 01:24:42.862449 kernel: NET: Registered PF_INET6 protocol family Aug 13 01:24:42.862456 kernel: Segment Routing with IPv6 Aug 13 01:24:42.862462 kernel: In-situ OAM (IOAM) with IPv6 Aug 13 01:24:42.862472 kernel: NET: Registered PF_PACKET protocol family Aug 13 01:24:42.862478 kernel: Key type dns_resolver registered Aug 13 01:24:42.862485 kernel: IPI shorthand broadcast: enabled Aug 13 01:24:42.862491 kernel: sched_clock: Marking stable (2776003798, 229875339)->(3043997211, -38118074) Aug 13 01:24:42.862498 kernel: registered taskstats version 1 Aug 13 01:24:42.862504 kernel: Loading compiled-in X.509 certificates Aug 13 01:24:42.862512 kernel: Loaded X.509 cert 'Kinvolk GmbH: Module signing key for 6.12.40-flatcar: dee0b464d3f7f8d09744a2392f69dde258bc95c0' Aug 13 01:24:42.862519 kernel: Demotion targets for Node 0: null Aug 13 01:24:42.862525 kernel: Key type .fscrypt registered Aug 13 01:24:42.862534 kernel: Key type fscrypt-provisioning registered Aug 13 01:24:42.862540 kernel: ima: No TPM chip found, activating TPM-bypass! Aug 13 01:24:42.862547 kernel: ima: Allocated hash algorithm: sha1 Aug 13 01:24:42.862554 kernel: ima: No architecture policies found Aug 13 01:24:42.862560 kernel: clk: Disabling unused clocks Aug 13 01:24:42.862567 kernel: Warning: unable to open an initial console. Aug 13 01:24:42.862574 kernel: Freeing unused kernel image (initmem) memory: 54444K Aug 13 01:24:42.862581 kernel: Write protecting the kernel read-only data: 24576k Aug 13 01:24:42.862587 kernel: Freeing unused kernel image (rodata/data gap) memory: 280K Aug 13 01:24:42.862596 kernel: Run /init as init process Aug 13 01:24:42.862602 kernel: with arguments: Aug 13 01:24:42.862609 kernel: /init Aug 13 01:24:42.862615 kernel: with environment: Aug 13 01:24:42.862622 kernel: HOME=/ Aug 13 01:24:42.862643 kernel: TERM=linux Aug 13 01:24:42.862652 kernel: BOOT_IMAGE=/flatcar/vmlinuz-a Aug 13 01:24:42.862660 systemd[1]: Successfully made /usr/ read-only. Aug 13 01:24:42.862672 systemd[1]: systemd 256.8 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL -ACL +BLKID +CURL +ELFUTILS -FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 -PWQUALITY -P11KIT -QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP -SYSVINIT +LIBARCHIVE) Aug 13 01:24:42.862680 systemd[1]: Detected virtualization kvm. Aug 13 01:24:42.862688 systemd[1]: Detected architecture x86-64. Aug 13 01:24:42.862695 systemd[1]: Running in initrd. Aug 13 01:24:42.862702 systemd[1]: No hostname configured, using default hostname. Aug 13 01:24:42.862710 systemd[1]: Hostname set to . Aug 13 01:24:42.862717 systemd[1]: Initializing machine ID from random generator. Aug 13 01:24:42.862725 systemd[1]: Queued start job for default target initrd.target. Aug 13 01:24:42.862734 systemd[1]: Started clevis-luks-askpass.path - Forward Password Requests to Clevis Directory Watch. Aug 13 01:24:42.862741 systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch. Aug 13 01:24:42.862750 systemd[1]: Expecting device dev-disk-by\x2dlabel-EFI\x2dSYSTEM.device - /dev/disk/by-label/EFI-SYSTEM... Aug 13 01:24:42.862757 systemd[1]: Expecting device dev-disk-by\x2dlabel-OEM.device - /dev/disk/by-label/OEM... Aug 13 01:24:42.862765 systemd[1]: Expecting device dev-disk-by\x2dlabel-ROOT.device - /dev/disk/by-label/ROOT... Aug 13 01:24:42.862773 systemd[1]: Expecting device dev-disk-by\x2dpartlabel-USR\x2dA.device - /dev/disk/by-partlabel/USR-A... Aug 13 01:24:42.862783 systemd[1]: Expecting device dev-disk-by\x2dpartuuid-7130c94a\x2d213a\x2d4e5a\x2d8e26\x2d6cce9662f132.device - /dev/disk/by-partuuid/7130c94a-213a-4e5a-8e26-6cce9662f132... Aug 13 01:24:42.862791 systemd[1]: Expecting device dev-mapper-usr.device - /dev/mapper/usr... Aug 13 01:24:42.862798 systemd[1]: Reached target cryptsetup-pre.target - Local Encrypted Volumes (Pre). Aug 13 01:24:42.862806 systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes. Aug 13 01:24:42.862813 systemd[1]: Reached target paths.target - Path Units. Aug 13 01:24:42.862820 systemd[1]: Reached target slices.target - Slice Units. Aug 13 01:24:42.862827 systemd[1]: Reached target swap.target - Swaps. Aug 13 01:24:42.862835 systemd[1]: Reached target timers.target - Timer Units. Aug 13 01:24:42.862843 systemd[1]: Listening on iscsid.socket - Open-iSCSI iscsid Socket. Aug 13 01:24:42.862852 systemd[1]: Listening on iscsiuio.socket - Open-iSCSI iscsiuio Socket. Aug 13 01:24:42.864882 systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log). Aug 13 01:24:42.864893 systemd[1]: Listening on systemd-journald.socket - Journal Sockets. Aug 13 01:24:42.864901 systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket. Aug 13 01:24:42.864908 systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket. Aug 13 01:24:42.864916 systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket. Aug 13 01:24:42.864923 systemd[1]: Reached target sockets.target - Socket Units. Aug 13 01:24:42.864934 systemd[1]: Starting ignition-setup-pre.service - Ignition env setup... Aug 13 01:24:42.864941 systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes... Aug 13 01:24:42.864949 systemd[1]: Finished network-cleanup.service - Network Cleanup. Aug 13 01:24:42.864957 systemd[1]: systemd-battery-check.service - Check battery level during early boot was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/class/power_supply). Aug 13 01:24:42.864964 systemd[1]: Starting systemd-fsck-usr.service... Aug 13 01:24:42.864972 systemd[1]: Starting systemd-journald.service - Journal Service... Aug 13 01:24:42.864981 systemd[1]: Starting systemd-modules-load.service - Load Kernel Modules... Aug 13 01:24:42.864988 systemd[1]: Starting systemd-vconsole-setup.service - Virtual Console Setup... Aug 13 01:24:42.864996 systemd[1]: Finished ignition-setup-pre.service - Ignition env setup. Aug 13 01:24:42.865004 systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes. Aug 13 01:24:42.865032 systemd-journald[206]: Collecting audit messages is disabled. Aug 13 01:24:42.865053 systemd[1]: Finished systemd-fsck-usr.service. Aug 13 01:24:42.865062 systemd[1]: Starting systemd-tmpfiles-setup-dev-early.service - Create Static Device Nodes in /dev gracefully... Aug 13 01:24:42.865070 systemd-journald[206]: Journal started Aug 13 01:24:42.865089 systemd-journald[206]: Runtime Journal (/run/log/journal/734e7d26670048ad88c782b35a5a92b6) is 8M, max 78.5M, 70.5M free. Aug 13 01:24:42.853893 systemd-modules-load[207]: Inserted module 'overlay' Aug 13 01:24:42.931379 kernel: bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this. Aug 13 01:24:42.931399 kernel: Bridge firewalling registered Aug 13 01:24:42.881779 systemd-modules-load[207]: Inserted module 'br_netfilter' Aug 13 01:24:42.934957 systemd[1]: Started systemd-journald.service - Journal Service. Aug 13 01:24:42.935546 systemd[1]: Finished systemd-modules-load.service - Load Kernel Modules. Aug 13 01:24:42.936380 systemd[1]: Finished systemd-vconsole-setup.service - Virtual Console Setup. Aug 13 01:24:42.937618 systemd[1]: Finished systemd-tmpfiles-setup-dev-early.service - Create Static Device Nodes in /dev gracefully. Aug 13 01:24:42.941360 systemd[1]: Starting dracut-cmdline-ask.service - dracut ask for additional cmdline parameters... Aug 13 01:24:42.943998 systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables... Aug 13 01:24:42.953659 systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev... Aug 13 01:24:42.956462 systemd[1]: Starting systemd-tmpfiles-setup.service - Create System Files and Directories... Aug 13 01:24:42.961257 systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables. Aug 13 01:24:42.971765 systemd-tmpfiles[228]: /usr/lib/tmpfiles.d/var.conf:14: Duplicate line for path "/var/log", ignoring. Aug 13 01:24:42.973548 systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev. Aug 13 01:24:42.975620 systemd[1]: Finished dracut-cmdline-ask.service - dracut ask for additional cmdline parameters. Aug 13 01:24:42.978325 systemd[1]: Starting dracut-cmdline.service - dracut cmdline hook... Aug 13 01:24:42.979686 systemd[1]: Finished systemd-tmpfiles-setup.service - Create System Files and Directories. Aug 13 01:24:42.987929 systemd[1]: Starting systemd-resolved.service - Network Name Resolution... Aug 13 01:24:42.999331 dracut-cmdline[243]: Using kernel command line parameters: rd.driver.pre=btrfs SYSTEMD_SULOGIN_FORCE=1 rootflags=rw mount.usrflags=ro BOOT_IMAGE=/flatcar/vmlinuz-a mount.usr=/dev/mapper/usr verity.usr=PARTUUID=7130c94a-213a-4e5a-8e26-6cce9662f132 rootflags=rw mount.usrflags=ro consoleblank=0 root=LABEL=ROOT console=ttyS0,115200n8 console=tty0 flatcar.first_boot=detected flatcar.oem.id=akamai verity.usrhash=215bdedb8de38f6b96ec4f9db80853e25015f60454b867e319fdcb9244320a21 Aug 13 01:24:43.029836 systemd-resolved[245]: Positive Trust Anchors: Aug 13 01:24:43.030511 systemd-resolved[245]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d Aug 13 01:24:43.030540 systemd-resolved[245]: Negative trust anchors: home.arpa 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 170.0.0.192.in-addr.arpa 171.0.0.192.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa ipv4only.arpa resolver.arpa corp home internal intranet lan local private test Aug 13 01:24:43.035698 systemd-resolved[245]: Defaulting to hostname 'linux'. Aug 13 01:24:43.036744 systemd[1]: Started systemd-resolved.service - Network Name Resolution. Aug 13 01:24:43.037635 systemd[1]: Reached target nss-lookup.target - Host and Network Name Lookups. Aug 13 01:24:43.084917 kernel: SCSI subsystem initialized Aug 13 01:24:43.093884 kernel: Loading iSCSI transport class v2.0-870. Aug 13 01:24:43.103888 kernel: iscsi: registered transport (tcp) Aug 13 01:24:43.123132 kernel: iscsi: registered transport (qla4xxx) Aug 13 01:24:43.123171 kernel: QLogic iSCSI HBA Driver Aug 13 01:24:43.142620 systemd[1]: Starting systemd-network-generator.service - Generate network units from Kernel command line... Aug 13 01:24:43.165240 systemd[1]: Finished systemd-network-generator.service - Generate network units from Kernel command line. Aug 13 01:24:43.167470 systemd[1]: Reached target network-pre.target - Preparation for Network. Aug 13 01:24:43.206175 systemd[1]: Finished dracut-cmdline.service - dracut cmdline hook. Aug 13 01:24:43.207935 systemd[1]: Starting dracut-pre-udev.service - dracut pre-udev hook... Aug 13 01:24:43.254877 kernel: raid6: avx2x4 gen() 33011 MB/s Aug 13 01:24:43.272884 kernel: raid6: avx2x2 gen() 30810 MB/s Aug 13 01:24:43.291275 kernel: raid6: avx2x1 gen() 22393 MB/s Aug 13 01:24:43.291290 kernel: raid6: using algorithm avx2x4 gen() 33011 MB/s Aug 13 01:24:43.310258 kernel: raid6: .... xor() 4578 MB/s, rmw enabled Aug 13 01:24:43.310285 kernel: raid6: using avx2x2 recovery algorithm Aug 13 01:24:43.330889 kernel: xor: automatically using best checksumming function avx Aug 13 01:24:43.462896 kernel: Btrfs loaded, zoned=no, fsverity=no Aug 13 01:24:43.469264 systemd[1]: Finished dracut-pre-udev.service - dracut pre-udev hook. Aug 13 01:24:43.471189 systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files... Aug 13 01:24:43.498936 systemd-udevd[454]: Using default interface naming scheme 'v255'. Aug 13 01:24:43.503842 systemd[1]: Started systemd-udevd.service - Rule-based Manager for Device Events and Files. Aug 13 01:24:43.506742 systemd[1]: Starting dracut-pre-trigger.service - dracut pre-trigger hook... Aug 13 01:24:43.528185 dracut-pre-trigger[461]: rd.md=0: removing MD RAID activation Aug 13 01:24:43.551120 systemd[1]: Finished dracut-pre-trigger.service - dracut pre-trigger hook. Aug 13 01:24:43.553060 systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices... Aug 13 01:24:43.618284 systemd[1]: Finished systemd-udev-trigger.service - Coldplug All udev Devices. Aug 13 01:24:43.620970 systemd[1]: Starting dracut-initqueue.service - dracut initqueue hook... Aug 13 01:24:43.676902 kernel: cryptd: max_cpu_qlen set to 1000 Aug 13 01:24:43.681658 kernel: input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input2 Aug 13 01:24:43.681699 kernel: virtio_scsi virtio0: 2/0/0 default/read/poll queues Aug 13 01:24:43.688917 kernel: scsi host0: Virtio SCSI HBA Aug 13 01:24:43.697263 kernel: AES CTR mode by8 optimization enabled Aug 13 01:24:43.697285 kernel: scsi 0:0:0:0: Direct-Access QEMU QEMU HARDDISK 2.5+ PQ: 0 ANSI: 5 Aug 13 01:24:43.741256 systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. Aug 13 01:24:43.742176 systemd[1]: Stopped systemd-vconsole-setup.service - Virtual Console Setup. Aug 13 01:24:43.856767 systemd[1]: Stopping systemd-vconsole-setup.service - Virtual Console Setup... Aug 13 01:24:43.861204 systemd[1]: Starting systemd-vconsole-setup.service - Virtual Console Setup... Aug 13 01:24:43.864236 systemd[1]: run-credentials-systemd\x2dvconsole\x2dsetup.service.mount: Deactivated successfully. Aug 13 01:24:43.870678 kernel: sd 0:0:0:0: Power-on or device reset occurred Aug 13 01:24:43.874439 kernel: sd 0:0:0:0: [sda] 9297920 512-byte logical blocks: (4.76 GB/4.43 GiB) Aug 13 01:24:43.874597 kernel: sd 0:0:0:0: [sda] Write Protect is off Aug 13 01:24:43.874734 kernel: sd 0:0:0:0: [sda] Mode Sense: 63 00 00 08 Aug 13 01:24:43.880889 kernel: sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA Aug 13 01:24:43.912660 kernel: libata version 3.00 loaded. Aug 13 01:24:43.914903 kernel: GPT:Primary header thinks Alt. header is not at the end of the disk. Aug 13 01:24:43.914923 kernel: GPT:9289727 != 9297919 Aug 13 01:24:43.914935 kernel: GPT:Alternate GPT header not at the end of the disk. Aug 13 01:24:43.914944 kernel: GPT:9289727 != 9297919 Aug 13 01:24:43.914953 kernel: GPT: Use GNU Parted to correct GPT errors. Aug 13 01:24:43.914961 kernel: sda: sda1 sda2 sda3 sda4 sda6 sda7 sda9 Aug 13 01:24:43.923889 kernel: sd 0:0:0:0: [sda] Attached SCSI disk Aug 13 01:24:43.934981 kernel: ahci 0000:00:1f.2: version 3.0 Aug 13 01:24:43.939902 kernel: ACPI: \_SB_.GSIA: Enabled at IRQ 16 Aug 13 01:24:43.941403 kernel: ahci 0000:00:1f.2: AHCI vers 0001.0000, 32 command slots, 1.5 Gbps, SATA mode Aug 13 01:24:43.941657 kernel: ahci 0000:00:1f.2: 6/6 ports implemented (port mask 0x3f) Aug 13 01:24:43.941791 kernel: ahci 0000:00:1f.2: flags: 64bit ncq only Aug 13 01:24:43.943893 kernel: scsi host1: ahci Aug 13 01:24:43.944885 kernel: scsi host2: ahci Aug 13 01:24:43.945048 kernel: scsi host3: ahci Aug 13 01:24:43.945187 kernel: scsi host4: ahci Aug 13 01:24:43.947268 kernel: scsi host5: ahci Aug 13 01:24:43.947430 kernel: scsi host6: ahci Aug 13 01:24:43.947565 kernel: ata1: SATA max UDMA/133 abar m4096@0xfebd3000 port 0xfebd3100 irq 46 lpm-pol 0 Aug 13 01:24:43.947576 kernel: ata2: SATA max UDMA/133 abar m4096@0xfebd3000 port 0xfebd3180 irq 46 lpm-pol 0 Aug 13 01:24:43.947585 kernel: ata3: SATA max UDMA/133 abar m4096@0xfebd3000 port 0xfebd3200 irq 46 lpm-pol 0 Aug 13 01:24:43.947594 kernel: ata4: SATA max UDMA/133 abar m4096@0xfebd3000 port 0xfebd3280 irq 46 lpm-pol 0 Aug 13 01:24:43.947608 kernel: ata5: SATA max UDMA/133 abar m4096@0xfebd3000 port 0xfebd3300 irq 46 lpm-pol 0 Aug 13 01:24:43.947617 kernel: ata6: SATA max UDMA/133 abar m4096@0xfebd3000 port 0xfebd3380 irq 46 lpm-pol 0 Aug 13 01:24:44.012414 systemd[1]: Found device dev-disk-by\x2dlabel-EFI\x2dSYSTEM.device - QEMU_HARDDISK EFI-SYSTEM. Aug 13 01:24:44.064165 systemd[1]: Found device dev-disk-by\x2dlabel-ROOT.device - QEMU_HARDDISK ROOT. Aug 13 01:24:44.065089 systemd[1]: Finished systemd-vconsole-setup.service - Virtual Console Setup. Aug 13 01:24:44.075634 systemd[1]: Found device dev-disk-by\x2dlabel-OEM.device - QEMU_HARDDISK OEM. Aug 13 01:24:44.083186 systemd[1]: Found device dev-disk-by\x2dpartuuid-7130c94a\x2d213a\x2d4e5a\x2d8e26\x2d6cce9662f132.device - QEMU_HARDDISK USR-A. Aug 13 01:24:44.083815 systemd[1]: Found device dev-disk-by\x2dpartlabel-USR\x2dA.device - QEMU_HARDDISK USR-A. Aug 13 01:24:44.086899 systemd[1]: Starting disk-uuid.service - Generate new UUID for disk GPT if necessary... Aug 13 01:24:44.101493 disk-uuid[625]: Primary Header is updated. Aug 13 01:24:44.101493 disk-uuid[625]: Secondary Entries is updated. Aug 13 01:24:44.101493 disk-uuid[625]: Secondary Header is updated. Aug 13 01:24:44.114173 kernel: sda: sda1 sda2 sda3 sda4 sda6 sda7 sda9 Aug 13 01:24:44.127914 kernel: sda: sda1 sda2 sda3 sda4 sda6 sda7 sda9 Aug 13 01:24:44.263737 kernel: ata6: SATA link down (SStatus 0 SControl 300) Aug 13 01:24:44.263826 kernel: ata5: SATA link down (SStatus 0 SControl 300) Aug 13 01:24:44.263838 kernel: ata3: SATA link down (SStatus 0 SControl 300) Aug 13 01:24:44.263848 kernel: ata2: SATA link down (SStatus 0 SControl 300) Aug 13 01:24:44.263871 kernel: ata4: SATA link down (SStatus 0 SControl 300) Aug 13 01:24:44.263884 kernel: ata1: SATA link down (SStatus 0 SControl 300) Aug 13 01:24:44.291754 systemd[1]: Finished dracut-initqueue.service - dracut initqueue hook. Aug 13 01:24:44.302229 systemd[1]: Reached target remote-fs-pre.target - Preparation for Remote File Systems. Aug 13 01:24:44.303659 systemd[1]: Reached target remote-cryptsetup.target - Remote Encrypted Volumes. Aug 13 01:24:44.304401 systemd[1]: Reached target remote-fs.target - Remote File Systems. Aug 13 01:24:44.307101 systemd[1]: Starting dracut-pre-mount.service - dracut pre-mount hook... Aug 13 01:24:44.331482 systemd[1]: Finished dracut-pre-mount.service - dracut pre-mount hook. Aug 13 01:24:45.126912 kernel: sda: sda1 sda2 sda3 sda4 sda6 sda7 sda9 Aug 13 01:24:45.129149 disk-uuid[626]: The operation has completed successfully. Aug 13 01:24:45.182810 systemd[1]: disk-uuid.service: Deactivated successfully. Aug 13 01:24:45.182969 systemd[1]: Finished disk-uuid.service - Generate new UUID for disk GPT if necessary. Aug 13 01:24:45.206414 systemd[1]: Starting verity-setup.service - Verity Setup for /dev/mapper/usr... Aug 13 01:24:45.218112 sh[653]: Success Aug 13 01:24:45.236604 kernel: device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. Aug 13 01:24:45.236646 kernel: device-mapper: uevent: version 1.0.3 Aug 13 01:24:45.239897 kernel: device-mapper: ioctl: 4.48.0-ioctl (2023-03-01) initialised: dm-devel@lists.linux.dev Aug 13 01:24:45.249950 kernel: device-mapper: verity: sha256 using shash "sha256-ni" Aug 13 01:24:45.292034 systemd[1]: Found device dev-mapper-usr.device - /dev/mapper/usr. Aug 13 01:24:45.295932 systemd[1]: Mounting sysusr-usr.mount - /sysusr/usr... Aug 13 01:24:45.305519 systemd[1]: Finished verity-setup.service - Verity Setup for /dev/mapper/usr. Aug 13 01:24:45.318005 kernel: BTRFS info: 'norecovery' is for compatibility only, recommended to use 'rescue=nologreplay' Aug 13 01:24:45.318042 kernel: BTRFS: device fsid 0c0338fb-9434-41c1-99a2-737cbe2351c4 devid 1 transid 44 /dev/mapper/usr (254:0) scanned by mount (665) Aug 13 01:24:45.323976 kernel: BTRFS info (device dm-0): first mount of filesystem 0c0338fb-9434-41c1-99a2-737cbe2351c4 Aug 13 01:24:45.323999 kernel: BTRFS info (device dm-0): using crc32c (crc32c-intel) checksum algorithm Aug 13 01:24:45.325891 kernel: BTRFS info (device dm-0): using free-space-tree Aug 13 01:24:45.335538 systemd[1]: Mounted sysusr-usr.mount - /sysusr/usr. Aug 13 01:24:45.336678 systemd[1]: Reached target initrd-usr-fs.target - Initrd /usr File System. Aug 13 01:24:45.337757 systemd[1]: afterburn-network-kargs.service - Afterburn Initrd Setup Network Kernel Arguments was skipped because no trigger condition checks were met. Aug 13 01:24:45.338515 systemd[1]: Starting ignition-setup.service - Ignition (setup)... Aug 13 01:24:45.341955 systemd[1]: Starting parse-ip-for-networkd.service - Write systemd-networkd units from cmdline... Aug 13 01:24:45.371906 kernel: BTRFS: device label OEM devid 1 transid 11 /dev/sda6 (8:6) scanned by mount (700) Aug 13 01:24:45.371940 kernel: BTRFS info (device sda6): first mount of filesystem 900bf3f4-cc50-4925-b275-d85854bb916f Aug 13 01:24:45.376085 kernel: BTRFS info (device sda6): using crc32c (crc32c-intel) checksum algorithm Aug 13 01:24:45.376116 kernel: BTRFS info (device sda6): using free-space-tree Aug 13 01:24:45.386941 kernel: BTRFS info (device sda6): last unmount of filesystem 900bf3f4-cc50-4925-b275-d85854bb916f Aug 13 01:24:45.387539 systemd[1]: Finished ignition-setup.service - Ignition (setup). Aug 13 01:24:45.390148 systemd[1]: Starting ignition-fetch-offline.service - Ignition (fetch-offline)... Aug 13 01:24:45.471906 systemd[1]: Finished parse-ip-for-networkd.service - Write systemd-networkd units from cmdline. Aug 13 01:24:45.477353 systemd[1]: Starting systemd-networkd.service - Network Configuration... Aug 13 01:24:45.498522 ignition[757]: Ignition 2.21.0 Aug 13 01:24:45.498539 ignition[757]: Stage: fetch-offline Aug 13 01:24:45.498576 ignition[757]: no configs at "/usr/lib/ignition/base.d" Aug 13 01:24:45.501543 systemd[1]: Finished ignition-fetch-offline.service - Ignition (fetch-offline). Aug 13 01:24:45.498587 ignition[757]: no config dir at "/usr/lib/ignition/base.platform.d/akamai" Aug 13 01:24:45.498667 ignition[757]: parsed url from cmdline: "" Aug 13 01:24:45.498671 ignition[757]: no config URL provided Aug 13 01:24:45.498676 ignition[757]: reading system config file "/usr/lib/ignition/user.ign" Aug 13 01:24:45.498685 ignition[757]: no config at "/usr/lib/ignition/user.ign" Aug 13 01:24:45.498690 ignition[757]: failed to fetch config: resource requires networking Aug 13 01:24:45.498839 ignition[757]: Ignition finished successfully Aug 13 01:24:45.523473 systemd-networkd[840]: lo: Link UP Aug 13 01:24:45.524156 systemd-networkd[840]: lo: Gained carrier Aug 13 01:24:45.525952 systemd-networkd[840]: Enumeration completed Aug 13 01:24:45.526065 systemd[1]: Started systemd-networkd.service - Network Configuration. Aug 13 01:24:45.526772 systemd-networkd[840]: eth0: found matching network '/usr/lib/systemd/network/zz-default.network', based on potentially unpredictable interface name. Aug 13 01:24:45.526776 systemd-networkd[840]: eth0: Configuring with /usr/lib/systemd/network/zz-default.network. Aug 13 01:24:45.528214 systemd[1]: Reached target network.target - Network. Aug 13 01:24:45.529039 systemd-networkd[840]: eth0: Link UP Aug 13 01:24:45.529241 systemd-networkd[840]: eth0: Gained carrier Aug 13 01:24:45.529251 systemd-networkd[840]: eth0: found matching network '/usr/lib/systemd/network/zz-default.network', based on potentially unpredictable interface name. Aug 13 01:24:45.531432 systemd[1]: Starting ignition-fetch.service - Ignition (fetch)... Aug 13 01:24:45.556816 ignition[844]: Ignition 2.21.0 Aug 13 01:24:45.556831 ignition[844]: Stage: fetch Aug 13 01:24:45.557669 ignition[844]: no configs at "/usr/lib/ignition/base.d" Aug 13 01:24:45.557680 ignition[844]: no config dir at "/usr/lib/ignition/base.platform.d/akamai" Aug 13 01:24:45.557790 ignition[844]: parsed url from cmdline: "" Aug 13 01:24:45.557794 ignition[844]: no config URL provided Aug 13 01:24:45.557799 ignition[844]: reading system config file "/usr/lib/ignition/user.ign" Aug 13 01:24:45.557808 ignition[844]: no config at "/usr/lib/ignition/user.ign" Aug 13 01:24:45.557848 ignition[844]: PUT http://169.254.169.254/v1/token: attempt #1 Aug 13 01:24:45.558070 ignition[844]: PUT error: Put "http://169.254.169.254/v1/token": dial tcp 169.254.169.254:80: connect: network is unreachable Aug 13 01:24:45.759181 ignition[844]: PUT http://169.254.169.254/v1/token: attempt #2 Aug 13 01:24:45.759340 ignition[844]: PUT error: Put "http://169.254.169.254/v1/token": dial tcp 169.254.169.254:80: connect: network is unreachable Aug 13 01:24:46.060965 systemd-networkd[840]: eth0: DHCPv4 address 172.234.25.215/24, gateway 172.234.25.1 acquired from 23.194.118.58 Aug 13 01:24:46.159542 ignition[844]: PUT http://169.254.169.254/v1/token: attempt #3 Aug 13 01:24:46.264221 ignition[844]: PUT result: OK Aug 13 01:24:46.265280 ignition[844]: GET http://169.254.169.254/v1/user-data: attempt #1 Aug 13 01:24:46.395095 ignition[844]: GET result: OK Aug 13 01:24:46.395230 ignition[844]: parsing config with SHA512: fa4f7c9d84c77f0db68ac4268571e15ef866c44b10959fb126dbeb137f6eb8bc6284e4a91c1bac31cb4cb2a517b64e0201187bf8bea06f92fd4f6c3c5bb650c1 Aug 13 01:24:46.399086 unknown[844]: fetched base config from "system" Aug 13 01:24:46.399097 unknown[844]: fetched base config from "system" Aug 13 01:24:46.399336 ignition[844]: fetch: fetch complete Aug 13 01:24:46.399103 unknown[844]: fetched user config from "akamai" Aug 13 01:24:46.399343 ignition[844]: fetch: fetch passed Aug 13 01:24:46.403634 systemd[1]: Finished ignition-fetch.service - Ignition (fetch). Aug 13 01:24:46.399384 ignition[844]: Ignition finished successfully Aug 13 01:24:46.407988 systemd[1]: Starting ignition-kargs.service - Ignition (kargs)... Aug 13 01:24:46.447138 ignition[852]: Ignition 2.21.0 Aug 13 01:24:46.447151 ignition[852]: Stage: kargs Aug 13 01:24:46.447289 ignition[852]: no configs at "/usr/lib/ignition/base.d" Aug 13 01:24:46.447300 ignition[852]: no config dir at "/usr/lib/ignition/base.platform.d/akamai" Aug 13 01:24:46.448217 ignition[852]: kargs: kargs passed Aug 13 01:24:46.449953 systemd[1]: Finished ignition-kargs.service - Ignition (kargs). Aug 13 01:24:46.448264 ignition[852]: Ignition finished successfully Aug 13 01:24:46.453009 systemd[1]: Starting ignition-disks.service - Ignition (disks)... Aug 13 01:24:46.475898 ignition[858]: Ignition 2.21.0 Aug 13 01:24:46.476504 ignition[858]: Stage: disks Aug 13 01:24:46.476657 ignition[858]: no configs at "/usr/lib/ignition/base.d" Aug 13 01:24:46.476669 ignition[858]: no config dir at "/usr/lib/ignition/base.platform.d/akamai" Aug 13 01:24:46.477269 ignition[858]: disks: disks passed Aug 13 01:24:46.477305 ignition[858]: Ignition finished successfully Aug 13 01:24:46.479032 systemd[1]: Finished ignition-disks.service - Ignition (disks). Aug 13 01:24:46.480292 systemd[1]: Reached target initrd-root-device.target - Initrd Root Device. Aug 13 01:24:46.481418 systemd[1]: Reached target local-fs-pre.target - Preparation for Local File Systems. Aug 13 01:24:46.482065 systemd[1]: Reached target local-fs.target - Local File Systems. Aug 13 01:24:46.483334 systemd[1]: Reached target sysinit.target - System Initialization. Aug 13 01:24:46.484613 systemd[1]: Reached target basic.target - Basic System. Aug 13 01:24:46.486530 systemd[1]: Starting systemd-fsck-root.service - File System Check on /dev/disk/by-label/ROOT... Aug 13 01:24:46.525935 systemd-fsck[866]: ROOT: clean, 15/553520 files, 52789/553472 blocks Aug 13 01:24:46.528291 systemd[1]: Finished systemd-fsck-root.service - File System Check on /dev/disk/by-label/ROOT. Aug 13 01:24:46.531150 systemd[1]: Mounting sysroot.mount - /sysroot... Aug 13 01:24:46.642027 kernel: EXT4-fs (sda9): mounted filesystem 069caac6-7833-4acd-8940-01a7ff7d1281 r/w with ordered data mode. Quota mode: none. Aug 13 01:24:46.645842 systemd[1]: Mounted sysroot.mount - /sysroot. Aug 13 01:24:46.646829 systemd[1]: Reached target initrd-root-fs.target - Initrd Root File System. Aug 13 01:24:46.649954 systemd[1]: Mounting sysroot-oem.mount - /sysroot/oem... Aug 13 01:24:46.652932 systemd[1]: Mounting sysroot-usr.mount - /sysroot/usr... Aug 13 01:24:46.654407 systemd[1]: flatcar-metadata-hostname.service - Flatcar Metadata Hostname Agent was skipped because no trigger condition checks were met. Aug 13 01:24:46.654454 systemd[1]: ignition-remount-sysroot.service - Remount /sysroot read-write for Ignition was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/sysroot). Aug 13 01:24:46.654477 systemd[1]: Reached target ignition-diskful.target - Ignition Boot Disk Setup. Aug 13 01:24:46.665757 systemd[1]: Mounted sysroot-usr.mount - /sysroot/usr. Aug 13 01:24:46.668498 systemd[1]: Starting initrd-setup-root.service - Root filesystem setup... Aug 13 01:24:46.675889 kernel: BTRFS: device label OEM devid 1 transid 11 /dev/sda6 (8:6) scanned by mount (874) Aug 13 01:24:46.675923 kernel: BTRFS info (device sda6): first mount of filesystem 900bf3f4-cc50-4925-b275-d85854bb916f Aug 13 01:24:46.680762 kernel: BTRFS info (device sda6): using crc32c (crc32c-intel) checksum algorithm Aug 13 01:24:46.680787 kernel: BTRFS info (device sda6): using free-space-tree Aug 13 01:24:46.686686 systemd[1]: Mounted sysroot-oem.mount - /sysroot/oem. Aug 13 01:24:46.721823 initrd-setup-root[898]: cut: /sysroot/etc/passwd: No such file or directory Aug 13 01:24:46.727792 initrd-setup-root[905]: cut: /sysroot/etc/group: No such file or directory Aug 13 01:24:46.732714 initrd-setup-root[912]: cut: /sysroot/etc/shadow: No such file or directory Aug 13 01:24:46.737939 initrd-setup-root[919]: cut: /sysroot/etc/gshadow: No such file or directory Aug 13 01:24:46.830575 systemd[1]: Finished initrd-setup-root.service - Root filesystem setup. Aug 13 01:24:46.832887 systemd[1]: Starting ignition-mount.service - Ignition (mount)... Aug 13 01:24:46.834676 systemd[1]: Starting sysroot-boot.service - /sysroot/boot... Aug 13 01:24:46.849061 systemd[1]: sysroot-oem.mount: Deactivated successfully. Aug 13 01:24:46.851959 kernel: BTRFS info (device sda6): last unmount of filesystem 900bf3f4-cc50-4925-b275-d85854bb916f Aug 13 01:24:46.869616 systemd[1]: Finished sysroot-boot.service - /sysroot/boot. Aug 13 01:24:46.876767 ignition[987]: INFO : Ignition 2.21.0 Aug 13 01:24:46.876767 ignition[987]: INFO : Stage: mount Aug 13 01:24:46.879974 ignition[987]: INFO : no configs at "/usr/lib/ignition/base.d" Aug 13 01:24:46.879974 ignition[987]: INFO : no config dir at "/usr/lib/ignition/base.platform.d/akamai" Aug 13 01:24:46.879974 ignition[987]: INFO : mount: mount passed Aug 13 01:24:46.879974 ignition[987]: INFO : Ignition finished successfully Aug 13 01:24:46.880185 systemd[1]: Finished ignition-mount.service - Ignition (mount). Aug 13 01:24:46.882204 systemd[1]: Starting ignition-files.service - Ignition (files)... Aug 13 01:24:47.369084 systemd-networkd[840]: eth0: Gained IPv6LL Aug 13 01:24:47.644885 systemd[1]: Mounting sysroot-oem.mount - /sysroot/oem... Aug 13 01:24:47.670896 kernel: BTRFS: device label OEM devid 1 transid 11 /dev/sda6 (8:6) scanned by mount (999) Aug 13 01:24:47.674419 kernel: BTRFS info (device sda6): first mount of filesystem 900bf3f4-cc50-4925-b275-d85854bb916f Aug 13 01:24:47.674496 kernel: BTRFS info (device sda6): using crc32c (crc32c-intel) checksum algorithm Aug 13 01:24:47.677627 kernel: BTRFS info (device sda6): using free-space-tree Aug 13 01:24:47.682735 systemd[1]: Mounted sysroot-oem.mount - /sysroot/oem. Aug 13 01:24:47.710824 ignition[1015]: INFO : Ignition 2.21.0 Aug 13 01:24:47.710824 ignition[1015]: INFO : Stage: files Aug 13 01:24:47.712512 ignition[1015]: INFO : no configs at "/usr/lib/ignition/base.d" Aug 13 01:24:47.712512 ignition[1015]: INFO : no config dir at "/usr/lib/ignition/base.platform.d/akamai" Aug 13 01:24:47.714338 ignition[1015]: DEBUG : files: compiled without relabeling support, skipping Aug 13 01:24:47.715682 ignition[1015]: INFO : files: ensureUsers: op(1): [started] creating or modifying user "core" Aug 13 01:24:47.715682 ignition[1015]: DEBUG : files: ensureUsers: op(1): executing: "usermod" "--root" "/sysroot" "core" Aug 13 01:24:47.717659 ignition[1015]: INFO : files: ensureUsers: op(1): [finished] creating or modifying user "core" Aug 13 01:24:47.718698 ignition[1015]: INFO : files: ensureUsers: op(2): [started] adding ssh keys to user "core" Aug 13 01:24:47.718698 ignition[1015]: INFO : files: ensureUsers: op(2): [finished] adding ssh keys to user "core" Aug 13 01:24:47.718192 unknown[1015]: wrote ssh authorized keys file for user: core Aug 13 01:24:47.721088 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(3): [started] writing file "/sysroot/opt/helm-v3.17.3-linux-amd64.tar.gz" Aug 13 01:24:47.721088 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(3): GET https://get.helm.sh/helm-v3.17.3-linux-amd64.tar.gz: attempt #1 Aug 13 01:24:47.917791 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(3): GET result: OK Aug 13 01:24:48.129994 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(3): [finished] writing file "/sysroot/opt/helm-v3.17.3-linux-amd64.tar.gz" Aug 13 01:24:48.131186 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(4): [started] writing file "/sysroot/home/core/install.sh" Aug 13 01:24:48.131186 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(4): [finished] writing file "/sysroot/home/core/install.sh" Aug 13 01:24:48.131186 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(5): [started] writing file "/sysroot/home/core/nginx.yaml" Aug 13 01:24:48.131186 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(5): [finished] writing file "/sysroot/home/core/nginx.yaml" Aug 13 01:24:48.131186 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(6): [started] writing file "/sysroot/home/core/nfs-pod.yaml" Aug 13 01:24:48.131186 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(6): [finished] writing file "/sysroot/home/core/nfs-pod.yaml" Aug 13 01:24:48.131186 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(7): [started] writing file "/sysroot/home/core/nfs-pvc.yaml" Aug 13 01:24:48.131186 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(7): [finished] writing file "/sysroot/home/core/nfs-pvc.yaml" Aug 13 01:24:48.138059 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(8): [started] writing file "/sysroot/etc/flatcar/update.conf" Aug 13 01:24:48.138059 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(8): [finished] writing file "/sysroot/etc/flatcar/update.conf" Aug 13 01:24:48.138059 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(9): [started] writing link "/sysroot/etc/extensions/kubernetes.raw" -> "/opt/extensions/kubernetes/kubernetes-v1.33.0-x86-64.raw" Aug 13 01:24:48.138059 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(9): [finished] writing link "/sysroot/etc/extensions/kubernetes.raw" -> "/opt/extensions/kubernetes/kubernetes-v1.33.0-x86-64.raw" Aug 13 01:24:48.138059 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(a): [started] writing file "/sysroot/opt/extensions/kubernetes/kubernetes-v1.33.0-x86-64.raw" Aug 13 01:24:48.138059 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(a): GET https://extensions.flatcar.org/extensions/kubernetes-v1.33.0-x86-64.raw: attempt #1 Aug 13 01:24:48.646604 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(a): GET result: OK Aug 13 01:24:48.995254 ignition[1015]: INFO : files: createFilesystemsFiles: createFiles: op(a): [finished] writing file "/sysroot/opt/extensions/kubernetes/kubernetes-v1.33.0-x86-64.raw" Aug 13 01:24:48.995254 ignition[1015]: INFO : files: op(b): [started] processing unit "prepare-helm.service" Aug 13 01:24:48.997687 ignition[1015]: INFO : files: op(b): op(c): [started] writing unit "prepare-helm.service" at "/sysroot/etc/systemd/system/prepare-helm.service" Aug 13 01:24:48.998899 ignition[1015]: INFO : files: op(b): op(c): [finished] writing unit "prepare-helm.service" at "/sysroot/etc/systemd/system/prepare-helm.service" Aug 13 01:24:48.998899 ignition[1015]: INFO : files: op(b): [finished] processing unit "prepare-helm.service" Aug 13 01:24:48.998899 ignition[1015]: INFO : files: op(d): [started] processing unit "coreos-metadata.service" Aug 13 01:24:48.998899 ignition[1015]: INFO : files: op(d): op(e): [started] writing systemd drop-in "00-custom-metadata.conf" at "/sysroot/etc/systemd/system/coreos-metadata.service.d/00-custom-metadata.conf" Aug 13 01:24:49.002876 ignition[1015]: INFO : files: op(d): op(e): [finished] writing systemd drop-in "00-custom-metadata.conf" at "/sysroot/etc/systemd/system/coreos-metadata.service.d/00-custom-metadata.conf" Aug 13 01:24:49.002876 ignition[1015]: INFO : files: op(d): [finished] processing unit "coreos-metadata.service" Aug 13 01:24:49.002876 ignition[1015]: INFO : files: op(f): [started] setting preset to enabled for "prepare-helm.service" Aug 13 01:24:49.002876 ignition[1015]: INFO : files: op(f): [finished] setting preset to enabled for "prepare-helm.service" Aug 13 01:24:49.002876 ignition[1015]: INFO : files: createResultFile: createFiles: op(10): [started] writing file "/sysroot/etc/.ignition-result.json" Aug 13 01:24:49.002876 ignition[1015]: INFO : files: createResultFile: createFiles: op(10): [finished] writing file "/sysroot/etc/.ignition-result.json" Aug 13 01:24:49.002876 ignition[1015]: INFO : files: files passed Aug 13 01:24:49.002876 ignition[1015]: INFO : Ignition finished successfully Aug 13 01:24:49.002091 systemd[1]: Finished ignition-files.service - Ignition (files). Aug 13 01:24:49.005986 systemd[1]: Starting ignition-quench.service - Ignition (record completion)... Aug 13 01:24:49.009019 systemd[1]: Starting initrd-setup-root-after-ignition.service - Root filesystem completion... Aug 13 01:24:49.020687 systemd[1]: ignition-quench.service: Deactivated successfully. Aug 13 01:24:49.021541 systemd[1]: Finished ignition-quench.service - Ignition (record completion). Aug 13 01:24:49.029283 initrd-setup-root-after-ignition[1046]: grep: /sysroot/etc/flatcar/enabled-sysext.conf: No such file or directory Aug 13 01:24:49.029283 initrd-setup-root-after-ignition[1046]: grep: /sysroot/usr/share/flatcar/enabled-sysext.conf: No such file or directory Aug 13 01:24:49.031475 initrd-setup-root-after-ignition[1050]: grep: /sysroot/etc/flatcar/enabled-sysext.conf: No such file or directory Aug 13 01:24:49.033274 systemd[1]: Finished initrd-setup-root-after-ignition.service - Root filesystem completion. Aug 13 01:24:49.035162 systemd[1]: Reached target ignition-complete.target - Ignition Complete. Aug 13 01:24:49.036834 systemd[1]: Starting initrd-parse-etc.service - Mountpoints Configured in the Real Root... Aug 13 01:24:49.086645 systemd[1]: initrd-parse-etc.service: Deactivated successfully. Aug 13 01:24:49.086771 systemd[1]: Finished initrd-parse-etc.service - Mountpoints Configured in the Real Root. Aug 13 01:24:49.088389 systemd[1]: Reached target initrd-fs.target - Initrd File Systems. Aug 13 01:24:49.089284 systemd[1]: Reached target initrd.target - Initrd Default Target. Aug 13 01:24:49.090535 systemd[1]: dracut-mount.service - dracut mount hook was skipped because no trigger condition checks were met. Aug 13 01:24:49.091286 systemd[1]: Starting dracut-pre-pivot.service - dracut pre-pivot and cleanup hook... Aug 13 01:24:49.113712 systemd[1]: Finished dracut-pre-pivot.service - dracut pre-pivot and cleanup hook. Aug 13 01:24:49.116788 systemd[1]: Starting initrd-cleanup.service - Cleaning Up and Shutting Down Daemons... Aug 13 01:24:49.132001 systemd[1]: Stopped target nss-lookup.target - Host and Network Name Lookups. Aug 13 01:24:49.133725 systemd[1]: Stopped target remote-cryptsetup.target - Remote Encrypted Volumes. Aug 13 01:24:49.135397 systemd[1]: Stopped target timers.target - Timer Units. Aug 13 01:24:49.136501 systemd[1]: dracut-pre-pivot.service: Deactivated successfully. Aug 13 01:24:49.136613 systemd[1]: Stopped dracut-pre-pivot.service - dracut pre-pivot and cleanup hook. Aug 13 01:24:49.137706 systemd[1]: Stopped target initrd.target - Initrd Default Target. Aug 13 01:24:49.138457 systemd[1]: Stopped target basic.target - Basic System. Aug 13 01:24:49.139708 systemd[1]: Stopped target ignition-complete.target - Ignition Complete. Aug 13 01:24:49.140904 systemd[1]: Stopped target ignition-diskful.target - Ignition Boot Disk Setup. Aug 13 01:24:49.142811 systemd[1]: Stopped target initrd-root-device.target - Initrd Root Device. Aug 13 01:24:49.144132 systemd[1]: Stopped target initrd-usr-fs.target - Initrd /usr File System. Aug 13 01:24:49.145460 systemd[1]: Stopped target remote-fs.target - Remote File Systems. Aug 13 01:24:49.146692 systemd[1]: Stopped target remote-fs-pre.target - Preparation for Remote File Systems. Aug 13 01:24:49.148059 systemd[1]: Stopped target sysinit.target - System Initialization. Aug 13 01:24:49.149356 systemd[1]: Stopped target local-fs.target - Local File Systems. Aug 13 01:24:49.150742 systemd[1]: Stopped target swap.target - Swaps. Aug 13 01:24:49.151790 systemd[1]: dracut-pre-mount.service: Deactivated successfully. Aug 13 01:24:49.151959 systemd[1]: Stopped dracut-pre-mount.service - dracut pre-mount hook. Aug 13 01:24:49.153425 systemd[1]: Stopped target cryptsetup.target - Local Encrypted Volumes. Aug 13 01:24:49.154264 systemd[1]: Stopped target cryptsetup-pre.target - Local Encrypted Volumes (Pre). Aug 13 01:24:49.155437 systemd[1]: clevis-luks-askpass.path: Deactivated successfully. Aug 13 01:24:49.157064 systemd[1]: Stopped clevis-luks-askpass.path - Forward Password Requests to Clevis Directory Watch. Aug 13 01:24:49.158082 systemd[1]: dracut-initqueue.service: Deactivated successfully. Aug 13 01:24:49.158213 systemd[1]: Stopped dracut-initqueue.service - dracut initqueue hook. Aug 13 01:24:49.159740 systemd[1]: initrd-setup-root-after-ignition.service: Deactivated successfully. Aug 13 01:24:49.159904 systemd[1]: Stopped initrd-setup-root-after-ignition.service - Root filesystem completion. Aug 13 01:24:49.160590 systemd[1]: ignition-files.service: Deactivated successfully. Aug 13 01:24:49.160716 systemd[1]: Stopped ignition-files.service - Ignition (files). Aug 13 01:24:49.163946 systemd[1]: Stopping ignition-mount.service - Ignition (mount)... Aug 13 01:24:49.165537 systemd[1]: Stopping sysroot-boot.service - /sysroot/boot... Aug 13 01:24:49.167615 systemd[1]: systemd-udev-trigger.service: Deactivated successfully. Aug 13 01:24:49.167748 systemd[1]: Stopped systemd-udev-trigger.service - Coldplug All udev Devices. Aug 13 01:24:49.171368 systemd[1]: dracut-pre-trigger.service: Deactivated successfully. Aug 13 01:24:49.171553 systemd[1]: Stopped dracut-pre-trigger.service - dracut pre-trigger hook. Aug 13 01:24:49.179138 systemd[1]: initrd-cleanup.service: Deactivated successfully. Aug 13 01:24:49.179251 systemd[1]: Finished initrd-cleanup.service - Cleaning Up and Shutting Down Daemons. Aug 13 01:24:49.189853 ignition[1070]: INFO : Ignition 2.21.0 Aug 13 01:24:49.190839 ignition[1070]: INFO : Stage: umount Aug 13 01:24:49.194517 ignition[1070]: INFO : no configs at "/usr/lib/ignition/base.d" Aug 13 01:24:49.194517 ignition[1070]: INFO : no config dir at "/usr/lib/ignition/base.platform.d/akamai" Aug 13 01:24:49.196999 ignition[1070]: INFO : umount: umount passed Aug 13 01:24:49.196999 ignition[1070]: INFO : Ignition finished successfully Aug 13 01:24:49.198450 systemd[1]: sysroot-boot.mount: Deactivated successfully. Aug 13 01:24:49.199884 systemd[1]: ignition-mount.service: Deactivated successfully. Aug 13 01:24:49.200035 systemd[1]: Stopped ignition-mount.service - Ignition (mount). Aug 13 01:24:49.225960 systemd[1]: ignition-disks.service: Deactivated successfully. Aug 13 01:24:49.226047 systemd[1]: Stopped ignition-disks.service - Ignition (disks). Aug 13 01:24:49.229106 systemd[1]: ignition-kargs.service: Deactivated successfully. Aug 13 01:24:49.229156 systemd[1]: Stopped ignition-kargs.service - Ignition (kargs). Aug 13 01:24:49.229750 systemd[1]: ignition-fetch.service: Deactivated successfully. Aug 13 01:24:49.229796 systemd[1]: Stopped ignition-fetch.service - Ignition (fetch). Aug 13 01:24:49.230604 systemd[1]: Stopped target network.target - Network. Aug 13 01:24:49.231851 systemd[1]: ignition-fetch-offline.service: Deactivated successfully. Aug 13 01:24:49.231923 systemd[1]: Stopped ignition-fetch-offline.service - Ignition (fetch-offline). Aug 13 01:24:49.233036 systemd[1]: Stopped target paths.target - Path Units. Aug 13 01:24:49.234031 systemd[1]: systemd-ask-password-console.path: Deactivated successfully. Aug 13 01:24:49.236262 systemd[1]: Stopped systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch. Aug 13 01:24:49.237122 systemd[1]: Stopped target slices.target - Slice Units. Aug 13 01:24:49.238179 systemd[1]: Stopped target sockets.target - Socket Units. Aug 13 01:24:49.239410 systemd[1]: iscsid.socket: Deactivated successfully. Aug 13 01:24:49.239456 systemd[1]: Closed iscsid.socket - Open-iSCSI iscsid Socket. Aug 13 01:24:49.240717 systemd[1]: iscsiuio.socket: Deactivated successfully. Aug 13 01:24:49.240757 systemd[1]: Closed iscsiuio.socket - Open-iSCSI iscsiuio Socket. Aug 13 01:24:49.241755 systemd[1]: ignition-setup.service: Deactivated successfully. Aug 13 01:24:49.241809 systemd[1]: Stopped ignition-setup.service - Ignition (setup). Aug 13 01:24:49.242824 systemd[1]: ignition-setup-pre.service: Deactivated successfully. Aug 13 01:24:49.242887 systemd[1]: Stopped ignition-setup-pre.service - Ignition env setup. Aug 13 01:24:49.244027 systemd[1]: Stopping systemd-networkd.service - Network Configuration... Aug 13 01:24:49.245095 systemd[1]: Stopping systemd-resolved.service - Network Name Resolution... Aug 13 01:24:49.247006 systemd[1]: sysroot-boot.service: Deactivated successfully. Aug 13 01:24:49.247109 systemd[1]: Stopped sysroot-boot.service - /sysroot/boot. Aug 13 01:24:49.248504 systemd[1]: initrd-setup-root.service: Deactivated successfully. Aug 13 01:24:49.248587 systemd[1]: Stopped initrd-setup-root.service - Root filesystem setup. Aug 13 01:24:49.250142 systemd[1]: systemd-resolved.service: Deactivated successfully. Aug 13 01:24:49.250277 systemd[1]: Stopped systemd-resolved.service - Network Name Resolution. Aug 13 01:24:49.254687 systemd[1]: run-credentials-systemd\x2dresolved.service.mount: Deactivated successfully. Aug 13 01:24:49.254978 systemd[1]: systemd-networkd.service: Deactivated successfully. Aug 13 01:24:49.255103 systemd[1]: Stopped systemd-networkd.service - Network Configuration. Aug 13 01:24:49.257509 systemd[1]: run-credentials-systemd\x2dnetworkd.service.mount: Deactivated successfully. Aug 13 01:24:49.258401 systemd[1]: Stopped target network-pre.target - Preparation for Network. Aug 13 01:24:49.259533 systemd[1]: systemd-networkd.socket: Deactivated successfully. Aug 13 01:24:49.259576 systemd[1]: Closed systemd-networkd.socket - Network Service Netlink Socket. Aug 13 01:24:49.261534 systemd[1]: Stopping network-cleanup.service - Network Cleanup... Aug 13 01:24:49.263219 systemd[1]: parse-ip-for-networkd.service: Deactivated successfully. Aug 13 01:24:49.263289 systemd[1]: Stopped parse-ip-for-networkd.service - Write systemd-networkd units from cmdline. Aug 13 01:24:49.263907 systemd[1]: systemd-sysctl.service: Deactivated successfully. Aug 13 01:24:49.263953 systemd[1]: Stopped systemd-sysctl.service - Apply Kernel Variables. Aug 13 01:24:49.266680 systemd[1]: systemd-modules-load.service: Deactivated successfully. Aug 13 01:24:49.266727 systemd[1]: Stopped systemd-modules-load.service - Load Kernel Modules. Aug 13 01:24:49.267908 systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. Aug 13 01:24:49.267953 systemd[1]: Stopped systemd-tmpfiles-setup.service - Create System Files and Directories. Aug 13 01:24:49.269384 systemd[1]: Stopping systemd-udevd.service - Rule-based Manager for Device Events and Files... Aug 13 01:24:49.272745 systemd[1]: run-credentials-systemd\x2dsysctl.service.mount: Deactivated successfully. Aug 13 01:24:49.272809 systemd[1]: run-credentials-systemd\x2dtmpfiles\x2dsetup.service.mount: Deactivated successfully. Aug 13 01:24:49.285401 systemd[1]: network-cleanup.service: Deactivated successfully. Aug 13 01:24:49.286128 systemd[1]: Stopped network-cleanup.service - Network Cleanup. Aug 13 01:24:49.289635 systemd[1]: systemd-udevd.service: Deactivated successfully. Aug 13 01:24:49.289822 systemd[1]: Stopped systemd-udevd.service - Rule-based Manager for Device Events and Files. Aug 13 01:24:49.291128 systemd[1]: systemd-udevd-control.socket: Deactivated successfully. Aug 13 01:24:49.291173 systemd[1]: Closed systemd-udevd-control.socket - udev Control Socket. Aug 13 01:24:49.292263 systemd[1]: systemd-udevd-kernel.socket: Deactivated successfully. Aug 13 01:24:49.292300 systemd[1]: Closed systemd-udevd-kernel.socket - udev Kernel Socket. Aug 13 01:24:49.293525 systemd[1]: dracut-pre-udev.service: Deactivated successfully. Aug 13 01:24:49.293574 systemd[1]: Stopped dracut-pre-udev.service - dracut pre-udev hook. Aug 13 01:24:49.295317 systemd[1]: dracut-cmdline.service: Deactivated successfully. Aug 13 01:24:49.295363 systemd[1]: Stopped dracut-cmdline.service - dracut cmdline hook. Aug 13 01:24:49.296912 systemd[1]: dracut-cmdline-ask.service: Deactivated successfully. Aug 13 01:24:49.296965 systemd[1]: Stopped dracut-cmdline-ask.service - dracut ask for additional cmdline parameters. Aug 13 01:24:49.299004 systemd[1]: Starting initrd-udevadm-cleanup-db.service - Cleanup udev Database... Aug 13 01:24:49.299592 systemd[1]: systemd-network-generator.service: Deactivated successfully. Aug 13 01:24:49.299641 systemd[1]: Stopped systemd-network-generator.service - Generate network units from Kernel command line. Aug 13 01:24:49.301972 systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. Aug 13 01:24:49.302022 systemd[1]: Stopped systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev. Aug 13 01:24:49.303889 systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. Aug 13 01:24:49.303945 systemd[1]: Stopped systemd-tmpfiles-setup-dev-early.service - Create Static Device Nodes in /dev gracefully. Aug 13 01:24:49.306010 systemd[1]: kmod-static-nodes.service: Deactivated successfully. Aug 13 01:24:49.306060 systemd[1]: Stopped kmod-static-nodes.service - Create List of Static Device Nodes. Aug 13 01:24:49.307416 systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. Aug 13 01:24:49.307463 systemd[1]: Stopped systemd-vconsole-setup.service - Virtual Console Setup. Aug 13 01:24:49.312180 systemd[1]: run-credentials-systemd\x2dnetwork\x2dgenerator.service.mount: Deactivated successfully. Aug 13 01:24:49.312241 systemd[1]: run-credentials-systemd\x2dtmpfiles\x2dsetup\x2ddev\x2dearly.service.mount: Deactivated successfully. Aug 13 01:24:49.312284 systemd[1]: run-credentials-systemd\x2dtmpfiles\x2dsetup\x2ddev.service.mount: Deactivated successfully. Aug 13 01:24:49.312328 systemd[1]: run-credentials-systemd\x2dvconsole\x2dsetup.service.mount: Deactivated successfully. Aug 13 01:24:49.316920 systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. Aug 13 01:24:49.317028 systemd[1]: Finished initrd-udevadm-cleanup-db.service - Cleanup udev Database. Aug 13 01:24:49.318216 systemd[1]: Reached target initrd-switch-root.target - Switch Root. Aug 13 01:24:49.319771 systemd[1]: Starting initrd-switch-root.service - Switch Root... Aug 13 01:24:49.339995 systemd[1]: Switching root. Aug 13 01:24:49.384394 systemd-journald[206]: Journal stopped Aug 13 01:24:50.409698 systemd-journald[206]: Received SIGTERM from PID 1 (systemd). Aug 13 01:24:50.409725 kernel: SELinux: policy capability network_peer_controls=1 Aug 13 01:24:50.409737 kernel: SELinux: policy capability open_perms=1 Aug 13 01:24:50.409749 kernel: SELinux: policy capability extended_socket_class=1 Aug 13 01:24:50.409758 kernel: SELinux: policy capability always_check_network=0 Aug 13 01:24:50.409766 kernel: SELinux: policy capability cgroup_seclabel=1 Aug 13 01:24:50.409776 kernel: SELinux: policy capability nnp_nosuid_transition=1 Aug 13 01:24:50.409785 kernel: SELinux: policy capability genfs_seclabel_symlinks=0 Aug 13 01:24:50.409794 kernel: SELinux: policy capability ioctl_skip_cloexec=0 Aug 13 01:24:50.409803 kernel: SELinux: policy capability userspace_initial_context=0 Aug 13 01:24:50.409997 kernel: audit: type=1403 audit(1755048289.530:2): auid=4294967295 ses=4294967295 lsm=selinux res=1 Aug 13 01:24:50.410036 systemd[1]: Successfully loaded SELinux policy in 75.698ms. Aug 13 01:24:50.410049 systemd[1]: Relabeled /dev/, /dev/shm/, /run/ in 11.410ms. Aug 13 01:24:50.410059 systemd[1]: systemd 256.8 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP -GCRYPT -GNUTLS +OPENSSL -ACL +BLKID +CURL +ELFUTILS -FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 -PWQUALITY -P11KIT -QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP -SYSVINIT +LIBARCHIVE) Aug 13 01:24:50.410070 systemd[1]: Detected virtualization kvm. Aug 13 01:24:50.410083 systemd[1]: Detected architecture x86-64. Aug 13 01:24:50.410092 systemd[1]: Detected first boot. Aug 13 01:24:50.410102 systemd[1]: Initializing machine ID from random generator. Aug 13 01:24:50.410112 zram_generator::config[1115]: No configuration found. Aug 13 01:24:50.410123 kernel: Guest personality initialized and is inactive Aug 13 01:24:50.410132 kernel: VMCI host device registered (name=vmci, major=10, minor=125) Aug 13 01:24:50.410141 kernel: Initialized host personality Aug 13 01:24:50.410152 kernel: NET: Registered PF_VSOCK protocol family Aug 13 01:24:50.410162 systemd[1]: Populated /etc with preset unit settings. Aug 13 01:24:50.410172 systemd[1]: run-credentials-systemd\x2djournald.service.mount: Deactivated successfully. Aug 13 01:24:50.410182 systemd[1]: initrd-switch-root.service: Deactivated successfully. Aug 13 01:24:50.410192 systemd[1]: Stopped initrd-switch-root.service - Switch Root. Aug 13 01:24:50.410201 systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. Aug 13 01:24:50.410211 systemd[1]: Created slice system-addon\x2dconfig.slice - Slice /system/addon-config. Aug 13 01:24:50.410222 systemd[1]: Created slice system-addon\x2drun.slice - Slice /system/addon-run. Aug 13 01:24:50.410232 systemd[1]: Created slice system-getty.slice - Slice /system/getty. Aug 13 01:24:50.410242 systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe. Aug 13 01:24:50.410252 systemd[1]: Created slice system-serial\x2dgetty.slice - Slice /system/serial-getty. Aug 13 01:24:50.410262 systemd[1]: Created slice system-system\x2dcloudinit.slice - Slice /system/system-cloudinit. Aug 13 01:24:50.410271 systemd[1]: Created slice system-systemd\x2dfsck.slice - Slice /system/systemd-fsck. Aug 13 01:24:50.410281 systemd[1]: Created slice user.slice - User and Session Slice. Aug 13 01:24:50.410293 systemd[1]: Started clevis-luks-askpass.path - Forward Password Requests to Clevis Directory Watch. Aug 13 01:24:50.410303 systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch. Aug 13 01:24:50.410314 systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch. Aug 13 01:24:50.410324 systemd[1]: Set up automount boot.automount - Boot partition Automount Point. Aug 13 01:24:50.410337 systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point. Aug 13 01:24:50.410347 systemd[1]: Expecting device dev-disk-by\x2dlabel-OEM.device - /dev/disk/by-label/OEM... Aug 13 01:24:50.410357 systemd[1]: Expecting device dev-ttyS0.device - /dev/ttyS0... Aug 13 01:24:50.410367 systemd[1]: Reached target cryptsetup-pre.target - Local Encrypted Volumes (Pre). Aug 13 01:24:50.410378 systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes. Aug 13 01:24:50.410388 systemd[1]: Stopped target initrd-switch-root.target - Switch Root. Aug 13 01:24:50.410398 systemd[1]: Stopped target initrd-fs.target - Initrd File Systems. Aug 13 01:24:50.410408 systemd[1]: Stopped target initrd-root-fs.target - Initrd Root File System. Aug 13 01:24:50.410418 systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes. Aug 13 01:24:50.410428 systemd[1]: Reached target remote-cryptsetup.target - Remote Encrypted Volumes. Aug 13 01:24:50.410438 systemd[1]: Reached target remote-fs.target - Remote File Systems. Aug 13 01:24:50.410447 systemd[1]: Reached target slices.target - Slice Units. Aug 13 01:24:50.410459 systemd[1]: Reached target swap.target - Swaps. Aug 13 01:24:50.410469 systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes. Aug 13 01:24:50.410479 systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket. Aug 13 01:24:50.410489 systemd[1]: Listening on systemd-creds.socket - Credential Encryption/Decryption. Aug 13 01:24:50.410556 systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket. Aug 13 01:24:50.410570 systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket. Aug 13 01:24:50.410580 systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket. Aug 13 01:24:50.410592 systemd[1]: Listening on systemd-userdbd.socket - User Database Manager Socket. Aug 13 01:24:50.410602 systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System... Aug 13 01:24:50.410612 systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System... Aug 13 01:24:50.410622 systemd[1]: Mounting media.mount - External Media Directory... Aug 13 01:24:50.410632 systemd[1]: proc-xen.mount - /proc/xen was skipped because of an unmet condition check (ConditionVirtualization=xen). Aug 13 01:24:50.410642 systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System... Aug 13 01:24:50.410654 systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System... Aug 13 01:24:50.410664 systemd[1]: Mounting tmp.mount - Temporary Directory /tmp... Aug 13 01:24:50.410674 systemd[1]: var-lib-machines.mount - Virtual Machine and Container Storage (Compatibility) was skipped because of an unmet condition check (ConditionPathExists=/var/lib/machines.raw). Aug 13 01:24:50.410684 systemd[1]: Reached target machines.target - Containers. Aug 13 01:24:50.410694 systemd[1]: Starting flatcar-tmpfiles.service - Create missing system files... Aug 13 01:24:50.410704 systemd[1]: ignition-delete-config.service - Ignition (delete config) was skipped because no trigger condition checks were met. Aug 13 01:24:50.410714 systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes... Aug 13 01:24:50.410724 systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs... Aug 13 01:24:50.410742 systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod... Aug 13 01:24:50.410756 systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm... Aug 13 01:24:50.410767 systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore... Aug 13 01:24:50.410777 systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse... Aug 13 01:24:50.410788 systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop... Aug 13 01:24:50.410798 systemd[1]: setup-nsswitch.service - Create /etc/nsswitch.conf was skipped because of an unmet condition check (ConditionPathExists=!/etc/nsswitch.conf). Aug 13 01:24:50.410808 systemd[1]: systemd-fsck-root.service: Deactivated successfully. Aug 13 01:24:50.410820 systemd[1]: Stopped systemd-fsck-root.service - File System Check on Root Device. Aug 13 01:24:50.410830 systemd[1]: systemd-fsck-usr.service: Deactivated successfully. Aug 13 01:24:50.410842 systemd[1]: Stopped systemd-fsck-usr.service. Aug 13 01:24:50.410853 systemd[1]: systemd-hibernate-clear.service - Clear Stale Hibernate Storage Info was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67). Aug 13 01:24:50.410886 systemd[1]: Starting systemd-journald.service - Journal Service... Aug 13 01:24:50.410897 kernel: fuse: init (API version 7.41) Aug 13 01:24:50.410907 systemd[1]: Starting systemd-modules-load.service - Load Kernel Modules... Aug 13 01:24:50.410917 systemd[1]: Starting systemd-network-generator.service - Generate network units from Kernel command line... Aug 13 01:24:50.410927 kernel: loop: module loaded Aug 13 01:24:50.410936 systemd[1]: Starting systemd-remount-fs.service - Remount Root and Kernel File Systems... Aug 13 01:24:50.410970 systemd-journald[1206]: Collecting audit messages is disabled. Aug 13 01:24:50.410994 systemd[1]: Starting systemd-udev-load-credentials.service - Load udev Rules from Credentials... Aug 13 01:24:50.411005 kernel: ACPI: bus type drm_connector registered Aug 13 01:24:50.411015 systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices... Aug 13 01:24:50.411032 systemd[1]: verity-setup.service: Deactivated successfully. Aug 13 01:24:50.411048 systemd-journald[1206]: Journal started Aug 13 01:24:50.411068 systemd-journald[1206]: Runtime Journal (/run/log/journal/0b00b93386f04c079e515a3339cbd7ac) is 8M, max 78.5M, 70.5M free. Aug 13 01:24:50.096205 systemd[1]: Queued start job for default target multi-user.target. Aug 13 01:24:50.116659 systemd[1]: Unnecessary job was removed for dev-sda6.device - /dev/sda6. Aug 13 01:24:50.117265 systemd[1]: systemd-journald.service: Deactivated successfully. Aug 13 01:24:50.418663 systemd[1]: Stopped verity-setup.service. Aug 13 01:24:50.418733 systemd[1]: xenserver-pv-version.service - Set fake PV driver version for XenServer was skipped because of an unmet condition check (ConditionVirtualization=xen). Aug 13 01:24:50.425524 systemd[1]: Started systemd-journald.service - Journal Service. Aug 13 01:24:50.426459 systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System. Aug 13 01:24:50.427171 systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System. Aug 13 01:24:50.427978 systemd[1]: Mounted media.mount - External Media Directory. Aug 13 01:24:50.428656 systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System. Aug 13 01:24:50.429388 systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System. Aug 13 01:24:50.430144 systemd[1]: Mounted tmp.mount - Temporary Directory /tmp. Aug 13 01:24:50.431023 systemd[1]: Finished flatcar-tmpfiles.service - Create missing system files. Aug 13 01:24:50.432261 systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes. Aug 13 01:24:50.435213 systemd[1]: modprobe@configfs.service: Deactivated successfully. Aug 13 01:24:50.435657 systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs. Aug 13 01:24:50.441242 systemd[1]: modprobe@dm_mod.service: Deactivated successfully. Aug 13 01:24:50.441663 systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod. Aug 13 01:24:50.442522 systemd[1]: modprobe@drm.service: Deactivated successfully. Aug 13 01:24:50.442709 systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm. Aug 13 01:24:50.443733 systemd[1]: modprobe@efi_pstore.service: Deactivated successfully. Aug 13 01:24:50.444399 systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore. Aug 13 01:24:50.445316 systemd[1]: modprobe@fuse.service: Deactivated successfully. Aug 13 01:24:50.445628 systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse. Aug 13 01:24:50.446597 systemd[1]: modprobe@loop.service: Deactivated successfully. Aug 13 01:24:50.446913 systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop. Aug 13 01:24:50.448055 systemd[1]: Finished systemd-modules-load.service - Load Kernel Modules. Aug 13 01:24:50.449036 systemd[1]: Finished systemd-network-generator.service - Generate network units from Kernel command line. Aug 13 01:24:50.450169 systemd[1]: Finished systemd-remount-fs.service - Remount Root and Kernel File Systems. Aug 13 01:24:50.451170 systemd[1]: Finished systemd-udev-load-credentials.service - Load udev Rules from Credentials. Aug 13 01:24:50.465993 systemd[1]: Reached target network-pre.target - Preparation for Network. Aug 13 01:24:50.469944 systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System... Aug 13 01:24:50.472024 systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System... Aug 13 01:24:50.472684 systemd[1]: remount-root.service - Remount Root File System was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/). Aug 13 01:24:50.472766 systemd[1]: Reached target local-fs.target - Local File Systems. Aug 13 01:24:50.475798 systemd[1]: Listening on systemd-sysext.socket - System Extension Image Management. Aug 13 01:24:50.478085 systemd[1]: Starting ldconfig.service - Rebuild Dynamic Linker Cache... Aug 13 01:24:50.481028 systemd[1]: systemd-binfmt.service - Set Up Additional Binary Formats was skipped because no trigger condition checks were met. Aug 13 01:24:50.483823 systemd[1]: Starting systemd-hwdb-update.service - Rebuild Hardware Database... Aug 13 01:24:50.488410 systemd[1]: Starting systemd-journal-flush.service - Flush Journal to Persistent Storage... Aug 13 01:24:50.489800 systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore). Aug 13 01:24:50.491340 systemd[1]: Starting systemd-random-seed.service - Load/Save OS Random Seed... Aug 13 01:24:50.491965 systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met. Aug 13 01:24:50.494944 systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables... Aug 13 01:24:50.499372 systemd[1]: Starting systemd-sysext.service - Merge System Extension Images into /usr/ and /opt/... Aug 13 01:24:50.504097 systemd[1]: Starting systemd-tmpfiles-setup-dev-early.service - Create Static Device Nodes in /dev gracefully... Aug 13 01:24:50.512038 systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System. Aug 13 01:24:50.514016 systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System. Aug 13 01:24:50.545801 systemd-journald[1206]: Time spent on flushing to /var/log/journal/0b00b93386f04c079e515a3339cbd7ac is 31.295ms for 1001 entries. Aug 13 01:24:50.545801 systemd-journald[1206]: System Journal (/var/log/journal/0b00b93386f04c079e515a3339cbd7ac) is 8M, max 195.6M, 187.6M free. Aug 13 01:24:50.593291 systemd-journald[1206]: Received client request to flush runtime journal. Aug 13 01:24:50.593776 kernel: loop0: detected capacity change from 0 to 8 Aug 13 01:24:50.593801 kernel: squashfs: version 4.0 (2009/01/31) Phillip Lougher Aug 13 01:24:50.550952 systemd[1]: Finished systemd-random-seed.service - Load/Save OS Random Seed. Aug 13 01:24:50.553362 systemd[1]: Finished systemd-udev-trigger.service - Coldplug All udev Devices. Aug 13 01:24:50.557389 systemd[1]: Reached target first-boot-complete.target - First Boot Complete. Aug 13 01:24:50.562154 systemd[1]: Starting systemd-machine-id-commit.service - Save Transient machine-id to Disk... Aug 13 01:24:50.595758 systemd[1]: Finished systemd-journal-flush.service - Flush Journal to Persistent Storage. Aug 13 01:24:50.604955 kernel: loop1: detected capacity change from 0 to 229808 Aug 13 01:24:50.599245 systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables. Aug 13 01:24:50.604488 systemd-tmpfiles[1241]: ACLs are not supported, ignoring. Aug 13 01:24:50.604503 systemd-tmpfiles[1241]: ACLs are not supported, ignoring. Aug 13 01:24:50.617454 systemd[1]: Finished systemd-machine-id-commit.service - Save Transient machine-id to Disk. Aug 13 01:24:50.623916 systemd[1]: Finished systemd-tmpfiles-setup-dev-early.service - Create Static Device Nodes in /dev gracefully. Aug 13 01:24:50.628853 systemd[1]: Starting systemd-sysusers.service - Create System Users... Aug 13 01:24:50.649918 kernel: loop2: detected capacity change from 0 to 113872 Aug 13 01:24:50.681965 kernel: loop3: detected capacity change from 0 to 146240 Aug 13 01:24:50.680468 systemd[1]: Finished systemd-sysusers.service - Create System Users. Aug 13 01:24:50.689371 systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev... Aug 13 01:24:50.729921 kernel: loop4: detected capacity change from 0 to 8 Aug 13 01:24:50.737224 systemd-tmpfiles[1262]: ACLs are not supported, ignoring. Aug 13 01:24:50.737514 systemd-tmpfiles[1262]: ACLs are not supported, ignoring. Aug 13 01:24:50.739924 kernel: loop5: detected capacity change from 0 to 229808 Aug 13 01:24:50.742948 systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev. Aug 13 01:24:50.768921 kernel: loop6: detected capacity change from 0 to 113872 Aug 13 01:24:50.785890 kernel: loop7: detected capacity change from 0 to 146240 Aug 13 01:24:50.804637 (sd-merge)[1265]: Using extensions 'containerd-flatcar', 'docker-flatcar', 'kubernetes', 'oem-akamai'. Aug 13 01:24:50.805239 (sd-merge)[1265]: Merged extensions into '/usr'. Aug 13 01:24:50.810640 systemd[1]: Reload requested from client PID 1240 ('systemd-sysext') (unit systemd-sysext.service)... Aug 13 01:24:50.810659 systemd[1]: Reloading... Aug 13 01:24:50.901926 zram_generator::config[1291]: No configuration found. Aug 13 01:24:51.037139 systemd[1]: /usr/lib/systemd/system/docker.socket:6: ListenStream= references a path below legacy directory /var/run/, updating /var/run/docker.sock → /run/docker.sock; please update the unit file accordingly. Aug 13 01:24:51.075919 ldconfig[1235]: /sbin/ldconfig: /usr/lib/ld.so.conf is not an ELF file - it has the wrong magic bytes at the start. Aug 13 01:24:51.109108 systemd[1]: etc-machine\x2did.mount: Deactivated successfully. Aug 13 01:24:51.109643 systemd[1]: Reloading finished in 298 ms. Aug 13 01:24:51.140975 systemd[1]: Finished ldconfig.service - Rebuild Dynamic Linker Cache. Aug 13 01:24:51.142017 systemd[1]: Finished systemd-sysext.service - Merge System Extension Images into /usr/ and /opt/. Aug 13 01:24:51.157010 systemd[1]: Starting ensure-sysext.service... Aug 13 01:24:51.159987 systemd[1]: Starting systemd-tmpfiles-setup.service - Create System Files and Directories... Aug 13 01:24:51.182290 systemd-tmpfiles[1336]: /usr/lib/tmpfiles.d/nfs-utils.conf:6: Duplicate line for path "/var/lib/nfs/sm", ignoring. Aug 13 01:24:51.182567 systemd-tmpfiles[1336]: /usr/lib/tmpfiles.d/nfs-utils.conf:7: Duplicate line for path "/var/lib/nfs/sm.bak", ignoring. Aug 13 01:24:51.182987 systemd-tmpfiles[1336]: /usr/lib/tmpfiles.d/provision.conf:20: Duplicate line for path "/root", ignoring. Aug 13 01:24:51.183290 systemd-tmpfiles[1336]: /usr/lib/tmpfiles.d/systemd-flatcar.conf:6: Duplicate line for path "/var/log/journal", ignoring. Aug 13 01:24:51.183843 systemd[1]: Reload requested from client PID 1335 ('systemctl') (unit ensure-sysext.service)... Aug 13 01:24:51.184023 systemd[1]: Reloading... Aug 13 01:24:51.185696 systemd-tmpfiles[1336]: /usr/lib/tmpfiles.d/systemd.conf:29: Duplicate line for path "/var/lib/systemd", ignoring. Aug 13 01:24:51.186041 systemd-tmpfiles[1336]: ACLs are not supported, ignoring. Aug 13 01:24:51.186165 systemd-tmpfiles[1336]: ACLs are not supported, ignoring. Aug 13 01:24:51.191170 systemd-tmpfiles[1336]: Detected autofs mount point /boot during canonicalization of boot. Aug 13 01:24:51.192224 systemd-tmpfiles[1336]: Skipping /boot Aug 13 01:24:51.205340 systemd-tmpfiles[1336]: Detected autofs mount point /boot during canonicalization of boot. Aug 13 01:24:51.205354 systemd-tmpfiles[1336]: Skipping /boot Aug 13 01:24:51.262935 zram_generator::config[1366]: No configuration found. Aug 13 01:24:51.351774 systemd[1]: /usr/lib/systemd/system/docker.socket:6: ListenStream= references a path below legacy directory /var/run/, updating /var/run/docker.sock → /run/docker.sock; please update the unit file accordingly. Aug 13 01:24:51.431707 systemd[1]: Reloading finished in 247 ms. Aug 13 01:24:51.445905 systemd[1]: Finished systemd-hwdb-update.service - Rebuild Hardware Database. Aug 13 01:24:51.460097 systemd[1]: Finished systemd-tmpfiles-setup.service - Create System Files and Directories. Aug 13 01:24:51.470140 systemd[1]: Starting audit-rules.service - Load Audit Rules... Aug 13 01:24:51.474786 systemd[1]: Starting clean-ca-certificates.service - Clean up broken links in /etc/ssl/certs... Aug 13 01:24:51.486132 systemd[1]: Starting systemd-journal-catalog-update.service - Rebuild Journal Catalog... Aug 13 01:24:51.491940 systemd[1]: Starting systemd-resolved.service - Network Name Resolution... Aug 13 01:24:51.496002 systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files... Aug 13 01:24:51.499305 systemd[1]: Starting systemd-update-utmp.service - Record System Boot/Shutdown in UTMP... Aug 13 01:24:51.504842 systemd[1]: proc-xen.mount - /proc/xen was skipped because of an unmet condition check (ConditionVirtualization=xen). Aug 13 01:24:51.505076 systemd[1]: ignition-delete-config.service - Ignition (delete config) was skipped because no trigger condition checks were met. Aug 13 01:24:51.509087 systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod... Aug 13 01:24:51.513692 systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore... Aug 13 01:24:51.516926 systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop... Aug 13 01:24:51.519013 systemd[1]: systemd-binfmt.service - Set Up Additional Binary Formats was skipped because no trigger condition checks were met. Aug 13 01:24:51.519109 systemd[1]: systemd-hibernate-clear.service - Clear Stale Hibernate Storage Info was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67). Aug 13 01:24:51.519187 systemd[1]: xenserver-pv-version.service - Set fake PV driver version for XenServer was skipped because of an unmet condition check (ConditionVirtualization=xen). Aug 13 01:24:51.523809 systemd[1]: proc-xen.mount - /proc/xen was skipped because of an unmet condition check (ConditionVirtualization=xen). Aug 13 01:24:51.524001 systemd[1]: ignition-delete-config.service - Ignition (delete config) was skipped because no trigger condition checks were met. Aug 13 01:24:51.524149 systemd[1]: systemd-binfmt.service - Set Up Additional Binary Formats was skipped because no trigger condition checks were met. Aug 13 01:24:51.524221 systemd[1]: systemd-hibernate-clear.service - Clear Stale Hibernate Storage Info was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67). Aug 13 01:24:51.524291 systemd[1]: xenserver-pv-version.service - Set fake PV driver version for XenServer was skipped because of an unmet condition check (ConditionVirtualization=xen). Aug 13 01:24:51.535170 systemd[1]: proc-xen.mount - /proc/xen was skipped because of an unmet condition check (ConditionVirtualization=xen). Aug 13 01:24:51.536089 systemd[1]: ignition-delete-config.service - Ignition (delete config) was skipped because no trigger condition checks were met. Aug 13 01:24:51.544982 systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm... Aug 13 01:24:51.546362 systemd[1]: systemd-binfmt.service - Set Up Additional Binary Formats was skipped because no trigger condition checks were met. Aug 13 01:24:51.547000 systemd[1]: systemd-hibernate-clear.service - Clear Stale Hibernate Storage Info was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67). Aug 13 01:24:51.547123 systemd[1]: xenserver-pv-version.service - Set fake PV driver version for XenServer was skipped because of an unmet condition check (ConditionVirtualization=xen). Aug 13 01:24:51.559788 systemd[1]: Starting systemd-userdbd.service - User Database Manager... Aug 13 01:24:51.560991 systemd[1]: Finished ensure-sysext.service. Aug 13 01:24:51.562732 systemd-udevd[1415]: Using default interface naming scheme 'v255'. Aug 13 01:24:51.573842 systemd[1]: Finished systemd-update-utmp.service - Record System Boot/Shutdown in UTMP. Aug 13 01:24:51.581255 systemd[1]: Starting systemd-timesyncd.service - Network Time Synchronization... Aug 13 01:24:51.583047 systemd[1]: Finished clean-ca-certificates.service - Clean up broken links in /etc/ssl/certs. Aug 13 01:24:51.585529 systemd[1]: update-ca-certificates.service - Update CA bundle at /etc/ssl/certs/ca-certificates.crt was skipped because of an unmet condition check (ConditionPathIsSymbolicLink=!/etc/ssl/certs/ca-certificates.crt). Aug 13 01:24:51.602030 systemd[1]: modprobe@dm_mod.service: Deactivated successfully. Aug 13 01:24:51.602311 systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod. Aug 13 01:24:51.605071 systemd[1]: modprobe@loop.service: Deactivated successfully. Aug 13 01:24:51.610290 systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop. Aug 13 01:24:51.612687 systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met. Aug 13 01:24:51.615583 systemd[1]: modprobe@efi_pstore.service: Deactivated successfully. Aug 13 01:24:51.615803 systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore. Aug 13 01:24:51.618321 systemd[1]: modprobe@drm.service: Deactivated successfully. Aug 13 01:24:51.618655 systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm. Aug 13 01:24:51.620601 systemd[1]: Finished systemd-journal-catalog-update.service - Rebuild Journal Catalog. Aug 13 01:24:51.622266 systemd[1]: Started systemd-udevd.service - Rule-based Manager for Device Events and Files. Aug 13 01:24:51.632036 systemd[1]: Starting systemd-networkd.service - Network Configuration... Aug 13 01:24:51.632706 systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore). Aug 13 01:24:51.637015 systemd[1]: Starting systemd-update-done.service - Update is Completed... Aug 13 01:24:51.667236 augenrules[1473]: No rules Aug 13 01:24:51.670766 systemd[1]: audit-rules.service: Deactivated successfully. Aug 13 01:24:51.671050 systemd[1]: Finished audit-rules.service - Load Audit Rules. Aug 13 01:24:51.683463 systemd[1]: Finished systemd-update-done.service - Update is Completed. Aug 13 01:24:51.695095 systemd[1]: Started systemd-userdbd.service - User Database Manager. Aug 13 01:24:51.838595 systemd[1]: Condition check resulted in dev-ttyS0.device - /dev/ttyS0 being skipped. Aug 13 01:24:51.912347 kernel: mousedev: PS/2 mouse device common for all mice Aug 13 01:24:51.912417 kernel: input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3 Aug 13 01:24:51.945892 kernel: ACPI: button: Power Button [PWRF] Aug 13 01:24:51.978374 systemd[1]: Found device dev-disk-by\x2dlabel-OEM.device - QEMU_HARDDISK OEM. Aug 13 01:24:51.983103 systemd[1]: Starting systemd-fsck@dev-disk-by\x2dlabel-OEM.service - File System Check on /dev/disk/by-label/OEM... Aug 13 01:24:51.986752 systemd-networkd[1449]: lo: Link UP Aug 13 01:24:51.986764 systemd-networkd[1449]: lo: Gained carrier Aug 13 01:24:51.989304 systemd-networkd[1449]: Enumeration completed Aug 13 01:24:51.989418 systemd[1]: Started systemd-networkd.service - Network Configuration. Aug 13 01:24:51.989743 systemd-networkd[1449]: eth0: found matching network '/usr/lib/systemd/network/zz-default.network', based on potentially unpredictable interface name. Aug 13 01:24:51.989748 systemd-networkd[1449]: eth0: Configuring with /usr/lib/systemd/network/zz-default.network. Aug 13 01:24:51.992448 systemd-networkd[1449]: eth0: Link UP Aug 13 01:24:51.992609 systemd-networkd[1449]: eth0: Gained carrier Aug 13 01:24:51.992623 systemd-networkd[1449]: eth0: found matching network '/usr/lib/systemd/network/zz-default.network', based on potentially unpredictable interface name. Aug 13 01:24:51.994087 systemd[1]: Starting systemd-networkd-persistent-storage.service - Enable Persistent Storage in systemd-networkd... Aug 13 01:24:52.002099 systemd[1]: Starting systemd-networkd-wait-online.service - Wait for Network to be Configured... Aug 13 01:24:52.037530 kernel: i801_smbus 0000:00:1f.3: SMBus using PCI interrupt Aug 13 01:24:52.037808 kernel: i2c i2c-0: Memory type 0x07 not supported yet, not instantiating SPD Aug 13 01:24:52.044350 systemd[1]: Finished systemd-networkd-persistent-storage.service - Enable Persistent Storage in systemd-networkd. Aug 13 01:24:52.056611 systemd[1]: Finished systemd-fsck@dev-disk-by\x2dlabel-OEM.service - File System Check on /dev/disk/by-label/OEM. Aug 13 01:24:52.083332 systemd[1]: Started systemd-timesyncd.service - Network Time Synchronization. Aug 13 01:24:52.084664 systemd[1]: Reached target time-set.target - System Time Set. Aug 13 01:24:52.115882 systemd-resolved[1412]: Positive Trust Anchors: Aug 13 01:24:52.115895 systemd-resolved[1412]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d Aug 13 01:24:52.115923 systemd-resolved[1412]: Negative trust anchors: home.arpa 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 170.0.0.192.in-addr.arpa 171.0.0.192.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa ipv4only.arpa resolver.arpa corp home internal intranet lan local private test Aug 13 01:24:52.119659 systemd-resolved[1412]: Defaulting to hostname 'linux'. Aug 13 01:24:52.123318 systemd[1]: Started systemd-resolved.service - Network Name Resolution. Aug 13 01:24:52.125099 systemd[1]: Reached target network.target - Network. Aug 13 01:24:52.125806 systemd[1]: Reached target nss-lookup.target - Host and Network Name Lookups. Aug 13 01:24:52.126582 systemd[1]: Reached target sysinit.target - System Initialization. Aug 13 01:24:52.128006 systemd[1]: Started motdgen.path - Watch for update engine configuration changes. Aug 13 01:24:52.128606 systemd[1]: Started user-cloudinit@var-lib-flatcar\x2dinstall-user_data.path - Watch for a cloud-config at /var/lib/flatcar-install/user_data. Aug 13 01:24:52.129523 systemd[1]: Started google-oslogin-cache.timer - NSS cache refresh timer. Aug 13 01:24:52.130771 systemd[1]: Started logrotate.timer - Daily rotation of log files. Aug 13 01:24:52.132539 systemd[1]: Started mdadm.timer - Weekly check for MD array's redundancy information.. Aug 13 01:24:52.133126 systemd[1]: Started systemd-tmpfiles-clean.timer - Daily Cleanup of Temporary Directories. Aug 13 01:24:52.134049 systemd[1]: update-engine-stub.timer - Update Engine Stub Timer was skipped because of an unmet condition check (ConditionPathExists=/usr/.noupdate). Aug 13 01:24:52.134079 systemd[1]: Reached target paths.target - Path Units. Aug 13 01:24:52.134916 systemd[1]: Reached target timers.target - Timer Units. Aug 13 01:24:52.138197 systemd[1]: Listening on dbus.socket - D-Bus System Message Bus Socket. Aug 13 01:24:52.143063 systemd[1]: Starting docker.socket - Docker Socket for the API... Aug 13 01:24:52.148669 systemd[1]: Listening on sshd-unix-local.socket - OpenSSH Server Socket (systemd-ssh-generator, AF_UNIX Local). Aug 13 01:24:52.151045 systemd[1]: Listening on sshd-vsock.socket - OpenSSH Server Socket (systemd-ssh-generator, AF_VSOCK). Aug 13 01:24:52.153976 systemd[1]: Reached target ssh-access.target - SSH Access Available. Aug 13 01:24:52.162899 kernel: EDAC MC: Ver: 3.0.0 Aug 13 01:24:52.162910 systemd[1]: Listening on sshd.socket - OpenSSH Server Socket. Aug 13 01:24:52.165376 systemd[1]: Listening on systemd-hostnamed.socket - Hostname Service Socket. Aug 13 01:24:52.166935 systemd[1]: Listening on docker.socket - Docker Socket for the API. Aug 13 01:24:52.173842 systemd[1]: Reached target sockets.target - Socket Units. Aug 13 01:24:52.177755 systemd[1]: Reached target basic.target - Basic System. Aug 13 01:24:52.178534 systemd[1]: addon-config@oem.service - Configure Addon /oem was skipped because no trigger condition checks were met. Aug 13 01:24:52.178570 systemd[1]: addon-run@oem.service - Run Addon /oem was skipped because no trigger condition checks were met. Aug 13 01:24:52.179761 systemd[1]: Starting containerd.service - containerd container runtime... Aug 13 01:24:52.187083 systemd[1]: Starting coreos-metadata.service - Flatcar Metadata Agent... Aug 13 01:24:52.190457 systemd[1]: Starting dbus.service - D-Bus System Message Bus... Aug 13 01:24:52.193457 systemd[1]: Starting dracut-shutdown.service - Restore /run/initramfs on shutdown... Aug 13 01:24:52.206028 systemd[1]: Starting enable-oem-cloudinit.service - Enable cloudinit... Aug 13 01:24:52.216525 systemd[1]: Starting extend-filesystems.service - Extend Filesystems... Aug 13 01:24:52.217104 systemd[1]: flatcar-setup-environment.service - Modifies /etc/environment for CoreOS was skipped because of an unmet condition check (ConditionPathExists=/oem/bin/flatcar-setup-environment). Aug 13 01:24:52.223069 systemd[1]: Starting google-oslogin-cache.service - NSS cache refresh... Aug 13 01:24:52.238198 systemd[1]: Starting motdgen.service - Generate /run/flatcar/motd... Aug 13 01:24:52.248019 systemd[1]: Starting prepare-helm.service - Unpack helm to /opt/bin... Aug 13 01:24:52.252073 systemd[1]: Starting ssh-key-proc-cmdline.service - Install an ssh key from /proc/cmdline... Aug 13 01:24:52.259227 jq[1528]: false Aug 13 01:24:52.261017 systemd[1]: Starting sshd-keygen.service - Generate sshd host keys... Aug 13 01:24:52.296201 systemd[1]: Starting systemd-logind.service - User Login Management... Aug 13 01:24:52.298625 systemd[1]: tcsd.service - TCG Core Services Daemon was skipped because of an unmet condition check (ConditionPathExists=/dev/tpm0). Aug 13 01:24:52.299087 systemd[1]: cgroup compatibility translation between legacy and unified hierarchy settings activated. See cgroup-compat debug messages for details. Aug 13 01:24:52.301074 systemd[1]: Starting update-engine.service - Update Engine... Aug 13 01:24:52.308790 systemd[1]: Starting update-ssh-keys-after-ignition.service - Run update-ssh-keys once after Ignition... Aug 13 01:24:52.315664 systemd[1]: Finished dracut-shutdown.service - Restore /run/initramfs on shutdown. Aug 13 01:24:52.318132 systemd[1]: enable-oem-cloudinit.service: Skipped due to 'exec-condition'. Aug 13 01:24:52.318854 systemd[1]: Condition check resulted in enable-oem-cloudinit.service - Enable cloudinit being skipped. Aug 13 01:24:52.324949 jq[1545]: true Aug 13 01:24:52.327999 extend-filesystems[1529]: Found /dev/sda6 Aug 13 01:24:52.344896 extend-filesystems[1529]: Found /dev/sda9 Aug 13 01:24:52.345624 google_oslogin_nss_cache[1530]: oslogin_cache_refresh[1530]: Refreshing passwd entry cache Aug 13 01:24:52.342896 oslogin_cache_refresh[1530]: Refreshing passwd entry cache Aug 13 01:24:52.346306 systemd[1]: ssh-key-proc-cmdline.service: Deactivated successfully. Aug 13 01:24:52.349057 systemd[1]: Finished ssh-key-proc-cmdline.service - Install an ssh key from /proc/cmdline. Aug 13 01:24:52.355792 google_oslogin_nss_cache[1530]: oslogin_cache_refresh[1530]: Failure getting users, quitting Aug 13 01:24:52.355792 google_oslogin_nss_cache[1530]: oslogin_cache_refresh[1530]: Produced empty passwd cache file, removing /etc/oslogin_passwd.cache.bak. Aug 13 01:24:52.355792 google_oslogin_nss_cache[1530]: oslogin_cache_refresh[1530]: Refreshing group entry cache Aug 13 01:24:52.354731 oslogin_cache_refresh[1530]: Failure getting users, quitting Aug 13 01:24:52.354747 oslogin_cache_refresh[1530]: Produced empty passwd cache file, removing /etc/oslogin_passwd.cache.bak. Aug 13 01:24:52.354788 oslogin_cache_refresh[1530]: Refreshing group entry cache Aug 13 01:24:52.357466 extend-filesystems[1529]: Checking size of /dev/sda9 Aug 13 01:24:52.358132 systemd[1]: Starting systemd-vconsole-setup.service - Virtual Console Setup... Aug 13 01:24:52.359251 google_oslogin_nss_cache[1530]: oslogin_cache_refresh[1530]: Failure getting groups, quitting Aug 13 01:24:52.359251 google_oslogin_nss_cache[1530]: oslogin_cache_refresh[1530]: Produced empty group cache file, removing /etc/oslogin_group.cache.bak. Aug 13 01:24:52.358650 oslogin_cache_refresh[1530]: Failure getting groups, quitting Aug 13 01:24:52.358659 oslogin_cache_refresh[1530]: Produced empty group cache file, removing /etc/oslogin_group.cache.bak. Aug 13 01:24:52.362585 systemd[1]: google-oslogin-cache.service: Deactivated successfully. Aug 13 01:24:52.371160 systemd[1]: Finished google-oslogin-cache.service - NSS cache refresh. Aug 13 01:24:52.376596 update_engine[1544]: I20250813 01:24:52.376533 1544 main.cc:92] Flatcar Update Engine starting Aug 13 01:24:52.391222 jq[1553]: true Aug 13 01:24:52.392089 systemd[1]: motdgen.service: Deactivated successfully. Aug 13 01:24:52.392362 systemd[1]: Finished motdgen.service - Generate /run/flatcar/motd. Aug 13 01:24:52.401542 (ntainerd)[1567]: containerd.service: Referenced but unset environment variable evaluates to an empty string: TORCX_IMAGEDIR, TORCX_UNPACKDIR Aug 13 01:24:52.409925 extend-filesystems[1529]: Resized partition /dev/sda9 Aug 13 01:24:52.417895 extend-filesystems[1578]: resize2fs 1.47.2 (1-Jan-2025) Aug 13 01:24:52.427306 kernel: EXT4-fs (sda9): resizing filesystem from 553472 to 555003 blocks Aug 13 01:24:52.427346 tar[1549]: linux-amd64/LICENSE Aug 13 01:24:52.431889 tar[1549]: linux-amd64/helm Aug 13 01:24:52.442880 kernel: EXT4-fs (sda9): resized filesystem to 555003 Aug 13 01:24:52.444523 dbus-daemon[1526]: [system] SELinux support is enabled Aug 13 01:24:52.449185 systemd[1]: Started dbus.service - D-Bus System Message Bus. Aug 13 01:24:52.454985 systemd[1]: system-cloudinit@usr-share-oem-cloud\x2dconfig.yml.service - Load cloud-config from /usr/share/oem/cloud-config.yml was skipped because of an unmet condition check (ConditionFileNotEmpty=/usr/share/oem/cloud-config.yml). Aug 13 01:24:52.455013 systemd[1]: Reached target system-config.target - Load system-provided cloud configs. Aug 13 01:24:52.456480 systemd[1]: user-cloudinit-proc-cmdline.service - Load cloud-config from url defined in /proc/cmdline was skipped because of an unmet condition check (ConditionKernelCommandLine=cloud-config-url). Aug 13 01:24:52.456497 systemd[1]: Reached target user-config.target - Load user-provided cloud configs. Aug 13 01:24:52.471087 extend-filesystems[1578]: Filesystem at /dev/sda9 is mounted on /; on-line resizing required Aug 13 01:24:52.471087 extend-filesystems[1578]: old_desc_blocks = 1, new_desc_blocks = 1 Aug 13 01:24:52.471087 extend-filesystems[1578]: The filesystem on /dev/sda9 is now 555003 (4k) blocks long. Aug 13 01:24:52.498819 extend-filesystems[1529]: Resized filesystem in /dev/sda9 Aug 13 01:24:52.503531 update_engine[1544]: I20250813 01:24:52.491020 1544 update_check_scheduler.cc:74] Next update check in 8m6s Aug 13 01:24:52.474277 systemd[1]: extend-filesystems.service: Deactivated successfully. Aug 13 01:24:52.474727 systemd[1]: Finished extend-filesystems.service - Extend Filesystems. Aug 13 01:24:52.489200 systemd[1]: Started update-engine.service - Update Engine. Aug 13 01:24:52.501125 systemd[1]: Started locksmithd.service - Cluster reboot manager. Aug 13 01:24:52.543942 systemd-networkd[1449]: eth0: DHCPv4 address 172.234.25.215/24, gateway 172.234.25.1 acquired from 23.194.118.58 Aug 13 01:24:52.546972 systemd-timesyncd[1427]: Network configuration changed, trying to establish connection. Aug 13 01:24:52.548286 dbus-daemon[1526]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.3' (uid=244 pid=1449 comm="/usr/lib/systemd/systemd-networkd" label="system_u:system_r:kernel_t:s0") Aug 13 01:24:52.555113 systemd[1]: Starting systemd-hostnamed.service - Hostname Service... Aug 13 01:24:52.560204 coreos-metadata[1524]: Aug 13 01:24:52.560 INFO Putting http://169.254.169.254/v1/token: Attempt #1 Aug 13 01:24:52.608540 systemd-logind[1543]: Watching system buttons on /dev/input/event2 (Power Button) Aug 13 01:24:52.608572 systemd-logind[1543]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) Aug 13 01:24:52.615065 bash[1604]: Updated "/home/core/.ssh/authorized_keys" Aug 13 01:24:52.617837 systemd[1]: Finished update-ssh-keys-after-ignition.service - Run update-ssh-keys once after Ignition. Aug 13 01:24:52.621068 systemd-logind[1543]: New seat seat0. Aug 13 01:24:52.623513 systemd[1]: Starting sshkeys.service... Aug 13 01:24:52.633259 systemd[1]: Started systemd-logind.service - User Login Management. Aug 13 01:24:52.689954 systemd[1]: Created slice system-coreos\x2dmetadata\x2dsshkeys.slice - Slice /system/coreos-metadata-sshkeys. Aug 13 01:24:52.692788 systemd[1]: Starting coreos-metadata-sshkeys@core.service - Flatcar Metadata Agent (SSH Keys)... Aug 13 01:24:52.707160 containerd[1567]: time="2025-08-13T01:24:52Z" level=warning msg="Ignoring unknown key in TOML" column=1 error="strict mode: fields in the document are missing in the target struct" file=/usr/share/containerd/config.toml key=subreaper row=8 Aug 13 01:24:52.707160 containerd[1567]: time="2025-08-13T01:24:52.706488320Z" level=info msg="starting containerd" revision=06b99ca80cdbfbc6cc8bd567021738c9af2b36ce version=v2.0.4 Aug 13 01:24:52.711136 systemd[1]: Started systemd-hostnamed.service - Hostname Service. Aug 13 01:24:52.711680 dbus-daemon[1526]: [system] Successfully activated service 'org.freedesktop.hostname1' Aug 13 01:24:52.712143 dbus-daemon[1526]: [system] Activating via systemd: service name='org.freedesktop.PolicyKit1' unit='polkit.service' requested by ':1.6' (uid=0 pid=1598 comm="/usr/lib/systemd/systemd-hostnamed" label="system_u:system_r:kernel_t:s0") Aug 13 01:24:53.497779 systemd-resolved[1412]: Clock change detected. Flushing caches. Aug 13 01:24:53.497968 systemd-timesyncd[1427]: Contacted time server 23.155.40.38:123 (0.flatcar.pool.ntp.org). Aug 13 01:24:53.498012 systemd-timesyncd[1427]: Initial clock synchronization to Wed 2025-08-13 01:24:53.497234 UTC. Aug 13 01:24:53.503780 containerd[1567]: time="2025-08-13T01:24:53.503224050Z" level=warning msg="Configuration migrated from version 2, use `containerd config migrate` to avoid migration" t="7.37µs" Aug 13 01:24:53.503780 containerd[1567]: time="2025-08-13T01:24:53.503250640Z" level=info msg="loading plugin" id=io.containerd.image-verifier.v1.bindir type=io.containerd.image-verifier.v1 Aug 13 01:24:53.503780 containerd[1567]: time="2025-08-13T01:24:53.503266010Z" level=info msg="loading plugin" id=io.containerd.internal.v1.opt type=io.containerd.internal.v1 Aug 13 01:24:53.503780 containerd[1567]: time="2025-08-13T01:24:53.503407090Z" level=info msg="loading plugin" id=io.containerd.warning.v1.deprecations type=io.containerd.warning.v1 Aug 13 01:24:53.503780 containerd[1567]: time="2025-08-13T01:24:53.503420970Z" level=info msg="loading plugin" id=io.containerd.content.v1.content type=io.containerd.content.v1 Aug 13 01:24:53.503780 containerd[1567]: time="2025-08-13T01:24:53.503439990Z" level=info msg="loading plugin" id=io.containerd.snapshotter.v1.blockfile type=io.containerd.snapshotter.v1 Aug 13 01:24:53.503780 containerd[1567]: time="2025-08-13T01:24:53.503504490Z" level=info msg="skip loading plugin" error="no scratch file generator: skip plugin" id=io.containerd.snapshotter.v1.blockfile type=io.containerd.snapshotter.v1 Aug 13 01:24:53.503780 containerd[1567]: time="2025-08-13T01:24:53.503514950Z" level=info msg="loading plugin" id=io.containerd.snapshotter.v1.btrfs type=io.containerd.snapshotter.v1 Aug 13 01:24:53.503780 containerd[1567]: time="2025-08-13T01:24:53.503735750Z" level=info msg="skip loading plugin" error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs (ext4) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" id=io.containerd.snapshotter.v1.btrfs type=io.containerd.snapshotter.v1 Aug 13 01:24:53.503780 containerd[1567]: time="2025-08-13T01:24:53.503748790Z" level=info msg="loading plugin" id=io.containerd.snapshotter.v1.devmapper type=io.containerd.snapshotter.v1 Aug 13 01:24:53.503780 containerd[1567]: time="2025-08-13T01:24:53.503758610Z" level=info msg="skip loading plugin" error="devmapper not configured: skip plugin" id=io.containerd.snapshotter.v1.devmapper type=io.containerd.snapshotter.v1 Aug 13 01:24:53.503780 containerd[1567]: time="2025-08-13T01:24:53.503765480Z" level=info msg="loading plugin" id=io.containerd.snapshotter.v1.native type=io.containerd.snapshotter.v1 Aug 13 01:24:53.503993 containerd[1567]: time="2025-08-13T01:24:53.503850920Z" level=info msg="loading plugin" id=io.containerd.snapshotter.v1.overlayfs type=io.containerd.snapshotter.v1 Aug 13 01:24:53.504129 containerd[1567]: time="2025-08-13T01:24:53.504055670Z" level=info msg="loading plugin" id=io.containerd.snapshotter.v1.zfs type=io.containerd.snapshotter.v1 Aug 13 01:24:53.504129 containerd[1567]: time="2025-08-13T01:24:53.504089570Z" level=info msg="skip loading plugin" error="lstat /var/lib/containerd/io.containerd.snapshotter.v1.zfs: no such file or directory: skip plugin" id=io.containerd.snapshotter.v1.zfs type=io.containerd.snapshotter.v1 Aug 13 01:24:53.504129 containerd[1567]: time="2025-08-13T01:24:53.504098510Z" level=info msg="loading plugin" id=io.containerd.event.v1.exchange type=io.containerd.event.v1 Aug 13 01:24:53.505450 coreos-metadata[1524]: Aug 13 01:24:53.504 INFO Fetching http://169.254.169.254/v1/instance: Attempt #1 Aug 13 01:24:53.505495 containerd[1567]: time="2025-08-13T01:24:53.504641601Z" level=info msg="loading plugin" id=io.containerd.monitor.task.v1.cgroups type=io.containerd.monitor.task.v1 Aug 13 01:24:53.505495 containerd[1567]: time="2025-08-13T01:24:53.504969281Z" level=info msg="loading plugin" id=io.containerd.metadata.v1.bolt type=io.containerd.metadata.v1 Aug 13 01:24:53.505495 containerd[1567]: time="2025-08-13T01:24:53.505035281Z" level=info msg="metadata content store policy set" policy=shared Aug 13 01:24:53.511154 containerd[1567]: time="2025-08-13T01:24:53.511120244Z" level=info msg="loading plugin" id=io.containerd.gc.v1.scheduler type=io.containerd.gc.v1 Aug 13 01:24:53.511194 containerd[1567]: time="2025-08-13T01:24:53.511172724Z" level=info msg="loading plugin" id=io.containerd.differ.v1.walking type=io.containerd.differ.v1 Aug 13 01:24:53.511194 containerd[1567]: time="2025-08-13T01:24:53.511191094Z" level=info msg="loading plugin" id=io.containerd.lease.v1.manager type=io.containerd.lease.v1 Aug 13 01:24:53.511229 containerd[1567]: time="2025-08-13T01:24:53.511202034Z" level=info msg="loading plugin" id=io.containerd.service.v1.containers-service type=io.containerd.service.v1 Aug 13 01:24:53.511229 containerd[1567]: time="2025-08-13T01:24:53.511211804Z" level=info msg="loading plugin" id=io.containerd.service.v1.content-service type=io.containerd.service.v1 Aug 13 01:24:53.511229 containerd[1567]: time="2025-08-13T01:24:53.511227394Z" level=info msg="loading plugin" id=io.containerd.service.v1.diff-service type=io.containerd.service.v1 Aug 13 01:24:53.511289 containerd[1567]: time="2025-08-13T01:24:53.511239174Z" level=info msg="loading plugin" id=io.containerd.service.v1.images-service type=io.containerd.service.v1 Aug 13 01:24:53.511289 containerd[1567]: time="2025-08-13T01:24:53.511249404Z" level=info msg="loading plugin" id=io.containerd.service.v1.introspection-service type=io.containerd.service.v1 Aug 13 01:24:53.511289 containerd[1567]: time="2025-08-13T01:24:53.511258264Z" level=info msg="loading plugin" id=io.containerd.service.v1.namespaces-service type=io.containerd.service.v1 Aug 13 01:24:53.511289 containerd[1567]: time="2025-08-13T01:24:53.511266354Z" level=info msg="loading plugin" id=io.containerd.service.v1.snapshots-service type=io.containerd.service.v1 Aug 13 01:24:53.511289 containerd[1567]: time="2025-08-13T01:24:53.511273524Z" level=info msg="loading plugin" id=io.containerd.shim.v1.manager type=io.containerd.shim.v1 Aug 13 01:24:53.511289 containerd[1567]: time="2025-08-13T01:24:53.511283354Z" level=info msg="loading plugin" id=io.containerd.runtime.v2.task type=io.containerd.runtime.v2 Aug 13 01:24:53.511686 containerd[1567]: time="2025-08-13T01:24:53.511380454Z" level=info msg="loading plugin" id=io.containerd.service.v1.tasks-service type=io.containerd.service.v1 Aug 13 01:24:53.511686 containerd[1567]: time="2025-08-13T01:24:53.511405514Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.containers type=io.containerd.grpc.v1 Aug 13 01:24:53.511686 containerd[1567]: time="2025-08-13T01:24:53.511419794Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.content type=io.containerd.grpc.v1 Aug 13 01:24:53.511686 containerd[1567]: time="2025-08-13T01:24:53.511429454Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.diff type=io.containerd.grpc.v1 Aug 13 01:24:53.511686 containerd[1567]: time="2025-08-13T01:24:53.511438414Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.events type=io.containerd.grpc.v1 Aug 13 01:24:53.511686 containerd[1567]: time="2025-08-13T01:24:53.511449344Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.images type=io.containerd.grpc.v1 Aug 13 01:24:53.511686 containerd[1567]: time="2025-08-13T01:24:53.511469784Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.introspection type=io.containerd.grpc.v1 Aug 13 01:24:53.511686 containerd[1567]: time="2025-08-13T01:24:53.511483964Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.leases type=io.containerd.grpc.v1 Aug 13 01:24:53.511686 containerd[1567]: time="2025-08-13T01:24:53.511498534Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.namespaces type=io.containerd.grpc.v1 Aug 13 01:24:53.511686 containerd[1567]: time="2025-08-13T01:24:53.511511574Z" level=info msg="loading plugin" id=io.containerd.sandbox.store.v1.local type=io.containerd.sandbox.store.v1 Aug 13 01:24:53.511686 containerd[1567]: time="2025-08-13T01:24:53.511525584Z" level=info msg="loading plugin" id=io.containerd.cri.v1.images type=io.containerd.cri.v1 Aug 13 01:24:53.511686 containerd[1567]: time="2025-08-13T01:24:53.511608904Z" level=info msg="Get image filesystem path \"/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs\" for snapshotter \"overlayfs\"" Aug 13 01:24:53.511686 containerd[1567]: time="2025-08-13T01:24:53.511633614Z" level=info msg="Start snapshots syncer" Aug 13 01:24:53.511686 containerd[1567]: time="2025-08-13T01:24:53.511674584Z" level=info msg="loading plugin" id=io.containerd.cri.v1.runtime type=io.containerd.cri.v1 Aug 13 01:24:53.512086 containerd[1567]: time="2025-08-13T01:24:53.511979334Z" level=info msg="starting cri plugin" config="{\"containerd\":{\"defaultRuntimeName\":\"runc\",\"runtimes\":{\"runc\":{\"runtimeType\":\"io.containerd.runc.v2\",\"runtimePath\":\"\",\"PodAnnotations\":null,\"ContainerAnnotations\":null,\"options\":{\"BinaryName\":\"\",\"CriuImagePath\":\"\",\"CriuWorkPath\":\"\",\"IoGid\":0,\"IoUid\":0,\"NoNewKeyring\":false,\"Root\":\"\",\"ShimCgroup\":\"\",\"SystemdCgroup\":true},\"privileged_without_host_devices\":false,\"privileged_without_host_devices_all_devices_allowed\":false,\"baseRuntimeSpec\":\"\",\"cniConfDir\":\"\",\"cniMaxConfNum\":0,\"snapshotter\":\"\",\"sandboxer\":\"podsandbox\",\"io_type\":\"\"}},\"ignoreBlockIONotEnabledErrors\":false,\"ignoreRdtNotEnabledErrors\":false},\"cni\":{\"binDir\":\"/opt/cni/bin\",\"confDir\":\"/etc/cni/net.d\",\"maxConfNum\":1,\"setupSerially\":false,\"confTemplate\":\"\",\"ipPref\":\"\",\"useInternalLoopback\":false},\"enableSelinux\":true,\"selinuxCategoryRange\":1024,\"maxContainerLogSize\":16384,\"disableApparmor\":false,\"restrictOOMScoreAdj\":false,\"disableProcMount\":false,\"unsetSeccompProfile\":\"\",\"tolerateMissingHugetlbController\":true,\"disableHugetlbController\":true,\"device_ownership_from_security_context\":false,\"ignoreImageDefinedVolumes\":false,\"netnsMountsUnderStateDir\":false,\"enableUnprivilegedPorts\":true,\"enableUnprivilegedICMP\":true,\"enableCDI\":true,\"cdiSpecDirs\":[\"/etc/cdi\",\"/var/run/cdi\"],\"drainExecSyncIOTimeout\":\"0s\",\"ignoreDeprecationWarnings\":null,\"containerdRootDir\":\"/var/lib/containerd\",\"containerdEndpoint\":\"/run/containerd/containerd.sock\",\"rootDir\":\"/var/lib/containerd/io.containerd.grpc.v1.cri\",\"stateDir\":\"/run/containerd/io.containerd.grpc.v1.cri\"}" Aug 13 01:24:53.512086 containerd[1567]: time="2025-08-13T01:24:53.512070664Z" level=info msg="loading plugin" id=io.containerd.podsandbox.controller.v1.podsandbox type=io.containerd.podsandbox.controller.v1 Aug 13 01:24:53.516912 containerd[1567]: time="2025-08-13T01:24:53.514148885Z" level=info msg="loading plugin" id=io.containerd.sandbox.controller.v1.shim type=io.containerd.sandbox.controller.v1 Aug 13 01:24:53.516912 containerd[1567]: time="2025-08-13T01:24:53.514469606Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.sandbox-controllers type=io.containerd.grpc.v1 Aug 13 01:24:53.516912 containerd[1567]: time="2025-08-13T01:24:53.514487966Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.sandboxes type=io.containerd.grpc.v1 Aug 13 01:24:53.516912 containerd[1567]: time="2025-08-13T01:24:53.514501016Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.snapshots type=io.containerd.grpc.v1 Aug 13 01:24:53.516912 containerd[1567]: time="2025-08-13T01:24:53.514511826Z" level=info msg="loading plugin" id=io.containerd.streaming.v1.manager type=io.containerd.streaming.v1 Aug 13 01:24:53.516912 containerd[1567]: time="2025-08-13T01:24:53.514528276Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.streaming type=io.containerd.grpc.v1 Aug 13 01:24:53.516912 containerd[1567]: time="2025-08-13T01:24:53.514542306Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.tasks type=io.containerd.grpc.v1 Aug 13 01:24:53.516912 containerd[1567]: time="2025-08-13T01:24:53.514557926Z" level=info msg="loading plugin" id=io.containerd.transfer.v1.local type=io.containerd.transfer.v1 Aug 13 01:24:53.516912 containerd[1567]: time="2025-08-13T01:24:53.514581806Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.transfer type=io.containerd.grpc.v1 Aug 13 01:24:53.516912 containerd[1567]: time="2025-08-13T01:24:53.514591416Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.version type=io.containerd.grpc.v1 Aug 13 01:24:53.516912 containerd[1567]: time="2025-08-13T01:24:53.514600176Z" level=info msg="loading plugin" id=io.containerd.monitor.container.v1.restart type=io.containerd.monitor.container.v1 Aug 13 01:24:53.518365 containerd[1567]: time="2025-08-13T01:24:53.518340438Z" level=info msg="loading plugin" id=io.containerd.tracing.processor.v1.otlp type=io.containerd.tracing.processor.v1 Aug 13 01:24:53.518401 containerd[1567]: time="2025-08-13T01:24:53.518381358Z" level=info msg="skip loading plugin" error="skip plugin: tracing endpoint not configured" id=io.containerd.tracing.processor.v1.otlp type=io.containerd.tracing.processor.v1 Aug 13 01:24:53.518401 containerd[1567]: time="2025-08-13T01:24:53.518392428Z" level=info msg="loading plugin" id=io.containerd.internal.v1.tracing type=io.containerd.internal.v1 Aug 13 01:24:53.518438 containerd[1567]: time="2025-08-13T01:24:53.518402068Z" level=info msg="skip loading plugin" error="skip plugin: tracing endpoint not configured" id=io.containerd.internal.v1.tracing type=io.containerd.internal.v1 Aug 13 01:24:53.518438 containerd[1567]: time="2025-08-13T01:24:53.518409908Z" level=info msg="loading plugin" id=io.containerd.ttrpc.v1.otelttrpc type=io.containerd.ttrpc.v1 Aug 13 01:24:53.518438 containerd[1567]: time="2025-08-13T01:24:53.518418328Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.healthcheck type=io.containerd.grpc.v1 Aug 13 01:24:53.518438 containerd[1567]: time="2025-08-13T01:24:53.518427558Z" level=info msg="loading plugin" id=io.containerd.nri.v1.nri type=io.containerd.nri.v1 Aug 13 01:24:53.518511 containerd[1567]: time="2025-08-13T01:24:53.518442608Z" level=info msg="runtime interface created" Aug 13 01:24:53.518511 containerd[1567]: time="2025-08-13T01:24:53.518448658Z" level=info msg="created NRI interface" Aug 13 01:24:53.518511 containerd[1567]: time="2025-08-13T01:24:53.518455638Z" level=info msg="loading plugin" id=io.containerd.grpc.v1.cri type=io.containerd.grpc.v1 Aug 13 01:24:53.518511 containerd[1567]: time="2025-08-13T01:24:53.518464878Z" level=info msg="Connect containerd service" Aug 13 01:24:53.518511 containerd[1567]: time="2025-08-13T01:24:53.518487318Z" level=info msg="using experimental NRI integration - disable nri plugin to prevent this" Aug 13 01:24:53.519203 containerd[1567]: time="2025-08-13T01:24:53.519171098Z" level=error msg="failed to load cni during init, please check CRI plugin status before setting up network for pods" error="cni config load failed: no network config found in /etc/cni/net.d: cni plugin not initialized: failed to load cni config" Aug 13 01:24:53.573295 locksmithd[1584]: locksmithd starting currentOperation="UPDATE_STATUS_IDLE" strategy="reboot" Aug 13 01:24:53.701798 systemd[1]: Starting polkit.service - Authorization Manager... Aug 13 01:24:53.765147 containerd[1567]: time="2025-08-13T01:24:53.764947671Z" level=info msg="Start subscribing containerd event" Aug 13 01:24:53.765147 containerd[1567]: time="2025-08-13T01:24:53.765017421Z" level=info msg="Start recovering state" Aug 13 01:24:53.765918 containerd[1567]: time="2025-08-13T01:24:53.765882201Z" level=info msg="Start event monitor" Aug 13 01:24:53.765918 containerd[1567]: time="2025-08-13T01:24:53.765906041Z" level=info msg="Start cni network conf syncer for default" Aug 13 01:24:53.765918 containerd[1567]: time="2025-08-13T01:24:53.765916201Z" level=info msg="Start streaming server" Aug 13 01:24:53.765983 containerd[1567]: time="2025-08-13T01:24:53.765924621Z" level=info msg="Registered namespace \"k8s.io\" with NRI" Aug 13 01:24:53.766380 containerd[1567]: time="2025-08-13T01:24:53.766331011Z" level=info msg="runtime interface starting up..." Aug 13 01:24:53.766380 containerd[1567]: time="2025-08-13T01:24:53.766347911Z" level=info msg="starting plugins..." Aug 13 01:24:53.766380 containerd[1567]: time="2025-08-13T01:24:53.766363661Z" level=info msg="Synchronizing NRI (plugin) with current runtime state" Aug 13 01:24:53.770184 containerd[1567]: time="2025-08-13T01:24:53.770124323Z" level=info msg=serving... address=/run/containerd/containerd.sock.ttrpc Aug 13 01:24:53.770329 containerd[1567]: time="2025-08-13T01:24:53.770281523Z" level=info msg=serving... address=/run/containerd/containerd.sock Aug 13 01:24:53.770958 systemd[1]: Started containerd.service - containerd container runtime. Aug 13 01:24:53.771241 containerd[1567]: time="2025-08-13T01:24:53.771217654Z" level=info msg="containerd successfully booted in 0.281808s" Aug 13 01:24:53.780597 coreos-metadata[1524]: Aug 13 01:24:53.780 INFO Fetch successful Aug 13 01:24:53.780737 coreos-metadata[1524]: Aug 13 01:24:53.780 INFO Fetching http://169.254.169.254/v1/network: Attempt #1 Aug 13 01:24:53.812735 coreos-metadata[1610]: Aug 13 01:24:53.811 INFO Putting http://169.254.169.254/v1/token: Attempt #1 Aug 13 01:24:53.881083 polkitd[1633]: Started polkitd version 126 Aug 13 01:24:53.891613 polkitd[1633]: Loading rules from directory /etc/polkit-1/rules.d Aug 13 01:24:53.894030 polkitd[1633]: Loading rules from directory /run/polkit-1/rules.d Aug 13 01:24:53.894073 polkitd[1633]: Error opening rules directory: Error opening directory “/run/polkit-1/rules.d”: No such file or directory (g-file-error-quark, 4) Aug 13 01:24:53.894269 polkitd[1633]: Loading rules from directory /usr/local/share/polkit-1/rules.d Aug 13 01:24:53.894287 polkitd[1633]: Error opening rules directory: Error opening directory “/usr/local/share/polkit-1/rules.d”: No such file or directory (g-file-error-quark, 4) Aug 13 01:24:53.894321 polkitd[1633]: Loading rules from directory /usr/share/polkit-1/rules.d Aug 13 01:24:53.896161 polkitd[1633]: Finished loading, compiling and executing 2 rules Aug 13 01:24:53.896641 systemd[1]: Finished systemd-vconsole-setup.service - Virtual Console Setup. Aug 13 01:24:53.898018 systemd[1]: Started polkit.service - Authorization Manager. Aug 13 01:24:53.899409 dbus-daemon[1526]: [system] Successfully activated service 'org.freedesktop.PolicyKit1' Aug 13 01:24:53.899761 polkitd[1633]: Acquired the name org.freedesktop.PolicyKit1 on the system bus Aug 13 01:24:53.920385 systemd-hostnamed[1598]: Hostname set to <172-234-25-215> (transient) Aug 13 01:24:53.920408 systemd-resolved[1412]: System hostname changed to '172-234-25-215'. Aug 13 01:24:53.928958 coreos-metadata[1610]: Aug 13 01:24:53.928 INFO Fetching http://169.254.169.254/v1/ssh-keys: Attempt #1 Aug 13 01:24:54.082311 coreos-metadata[1524]: Aug 13 01:24:54.082 INFO Fetch successful Aug 13 01:24:54.084537 coreos-metadata[1610]: Aug 13 01:24:54.084 INFO Fetch successful Aug 13 01:24:54.124456 update-ssh-keys[1647]: Updated "/home/core/.ssh/authorized_keys" Aug 13 01:24:54.126089 systemd[1]: Finished coreos-metadata-sshkeys@core.service - Flatcar Metadata Agent (SSH Keys). Aug 13 01:24:54.131793 systemd[1]: Finished sshkeys.service. Aug 13 01:24:54.215180 tar[1549]: linux-amd64/README.md Aug 13 01:24:54.237747 sshd_keygen[1571]: ssh-keygen: generating new host keys: RSA ECDSA ED25519 Aug 13 01:24:54.245263 systemd[1]: Finished prepare-helm.service - Unpack helm to /opt/bin. Aug 13 01:24:54.254275 systemd[1]: Finished coreos-metadata.service - Flatcar Metadata Agent. Aug 13 01:24:54.255581 systemd[1]: packet-phone-home.service - Report Success to Packet was skipped because no trigger condition checks were met. Aug 13 01:24:54.259700 systemd[1]: Finished sshd-keygen.service - Generate sshd host keys. Aug 13 01:24:54.262266 systemd[1]: Starting issuegen.service - Generate /run/issue... Aug 13 01:24:54.283611 systemd[1]: issuegen.service: Deactivated successfully. Aug 13 01:24:54.283890 systemd[1]: Finished issuegen.service - Generate /run/issue. Aug 13 01:24:54.286785 systemd[1]: Starting systemd-user-sessions.service - Permit User Sessions... Aug 13 01:24:54.304559 systemd[1]: Finished systemd-user-sessions.service - Permit User Sessions. Aug 13 01:24:54.306841 systemd[1]: Started getty@tty1.service - Getty on tty1. Aug 13 01:24:54.309994 systemd[1]: Started serial-getty@ttyS0.service - Serial Getty on ttyS0. Aug 13 01:24:54.310660 systemd[1]: Reached target getty.target - Login Prompts. Aug 13 01:24:54.744985 systemd-networkd[1449]: eth0: Gained IPv6LL Aug 13 01:24:54.748111 systemd[1]: Finished systemd-networkd-wait-online.service - Wait for Network to be Configured. Aug 13 01:24:54.749164 systemd[1]: Reached target network-online.target - Network is Online. Aug 13 01:24:54.751667 systemd[1]: Starting kubelet.service - kubelet: The Kubernetes Node Agent... Aug 13 01:24:54.754875 systemd[1]: Starting nvidia.service - NVIDIA Configure Service... Aug 13 01:24:54.779261 systemd[1]: Finished nvidia.service - NVIDIA Configure Service. Aug 13 01:24:55.645785 systemd[1]: Started kubelet.service - kubelet: The Kubernetes Node Agent. Aug 13 01:24:55.647079 systemd[1]: Reached target multi-user.target - Multi-User System. Aug 13 01:24:55.648569 systemd[1]: Startup finished in 2.858s (kernel) + 6.858s (initrd) + 5.408s (userspace) = 15.125s. Aug 13 01:24:55.701688 (kubelet)[1704]: kubelet.service: Referenced but unset environment variable evaluates to an empty string: KUBELET_EXTRA_ARGS, KUBELET_KUBEADM_ARGS Aug 13 01:24:56.210392 kubelet[1704]: E0813 01:24:56.210309 1704 run.go:72] "command failed" err="failed to load kubelet config file, path: /var/lib/kubelet/config.yaml, error: failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to read kubelet config file \"/var/lib/kubelet/config.yaml\", error: open /var/lib/kubelet/config.yaml: no such file or directory" Aug 13 01:24:56.213814 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE Aug 13 01:24:56.213998 systemd[1]: kubelet.service: Failed with result 'exit-code'. Aug 13 01:24:56.214397 systemd[1]: kubelet.service: Consumed 879ms CPU time, 265.1M memory peak. Aug 13 01:24:57.899531 systemd[1]: Created slice system-sshd.slice - Slice /system/sshd. Aug 13 01:24:57.900995 systemd[1]: Started sshd@0-172.234.25.215:22-147.75.109.163:53564.service - OpenSSH per-connection server daemon (147.75.109.163:53564). Aug 13 01:24:58.247533 sshd[1716]: Accepted publickey for core from 147.75.109.163 port 53564 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:24:58.249211 sshd-session[1716]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:24:58.255681 systemd[1]: Created slice user-500.slice - User Slice of UID 500. Aug 13 01:24:58.256804 systemd[1]: Starting user-runtime-dir@500.service - User Runtime Directory /run/user/500... Aug 13 01:24:58.264603 systemd-logind[1543]: New session 1 of user core. Aug 13 01:24:58.276883 systemd[1]: Finished user-runtime-dir@500.service - User Runtime Directory /run/user/500. Aug 13 01:24:58.280436 systemd[1]: Starting user@500.service - User Manager for UID 500... Aug 13 01:24:58.294203 (systemd)[1720]: pam_unix(systemd-user:session): session opened for user core(uid=500) by (uid=0) Aug 13 01:24:58.296935 systemd-logind[1543]: New session c1 of user core. Aug 13 01:24:58.431704 systemd[1720]: Queued start job for default target default.target. Aug 13 01:24:58.444018 systemd[1720]: Created slice app.slice - User Application Slice. Aug 13 01:24:58.444047 systemd[1720]: Reached target paths.target - Paths. Aug 13 01:24:58.444093 systemd[1720]: Reached target timers.target - Timers. Aug 13 01:24:58.445554 systemd[1720]: Starting dbus.socket - D-Bus User Message Bus Socket... Aug 13 01:24:58.457178 systemd[1720]: Listening on dbus.socket - D-Bus User Message Bus Socket. Aug 13 01:24:58.457306 systemd[1720]: Reached target sockets.target - Sockets. Aug 13 01:24:58.457345 systemd[1720]: Reached target basic.target - Basic System. Aug 13 01:24:58.457386 systemd[1720]: Reached target default.target - Main User Target. Aug 13 01:24:58.457421 systemd[1720]: Startup finished in 154ms. Aug 13 01:24:58.457518 systemd[1]: Started user@500.service - User Manager for UID 500. Aug 13 01:24:58.465917 systemd[1]: Started session-1.scope - Session 1 of User core. Aug 13 01:24:58.729412 systemd[1]: Started sshd@1-172.234.25.215:22-147.75.109.163:51626.service - OpenSSH per-connection server daemon (147.75.109.163:51626). Aug 13 01:24:59.070638 sshd[1731]: Accepted publickey for core from 147.75.109.163 port 51626 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:24:59.072264 sshd-session[1731]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:24:59.077307 systemd-logind[1543]: New session 2 of user core. Aug 13 01:24:59.079847 systemd[1]: Started session-2.scope - Session 2 of User core. Aug 13 01:24:59.320360 sshd[1733]: Connection closed by 147.75.109.163 port 51626 Aug 13 01:24:59.321256 sshd-session[1731]: pam_unix(sshd:session): session closed for user core Aug 13 01:24:59.324195 systemd[1]: sshd@1-172.234.25.215:22-147.75.109.163:51626.service: Deactivated successfully. Aug 13 01:24:59.325857 systemd[1]: session-2.scope: Deactivated successfully. Aug 13 01:24:59.327613 systemd-logind[1543]: Session 2 logged out. Waiting for processes to exit. Aug 13 01:24:59.328636 systemd-logind[1543]: Removed session 2. Aug 13 01:24:59.379316 systemd[1]: Started sshd@2-172.234.25.215:22-147.75.109.163:51638.service - OpenSSH per-connection server daemon (147.75.109.163:51638). Aug 13 01:24:59.717488 sshd[1739]: Accepted publickey for core from 147.75.109.163 port 51638 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:24:59.718976 sshd-session[1739]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:24:59.723729 systemd-logind[1543]: New session 3 of user core. Aug 13 01:24:59.738834 systemd[1]: Started session-3.scope - Session 3 of User core. Aug 13 01:24:59.958568 sshd[1741]: Connection closed by 147.75.109.163 port 51638 Aug 13 01:24:59.959103 sshd-session[1739]: pam_unix(sshd:session): session closed for user core Aug 13 01:24:59.962618 systemd-logind[1543]: Session 3 logged out. Waiting for processes to exit. Aug 13 01:24:59.963164 systemd[1]: sshd@2-172.234.25.215:22-147.75.109.163:51638.service: Deactivated successfully. Aug 13 01:24:59.965031 systemd[1]: session-3.scope: Deactivated successfully. Aug 13 01:24:59.966442 systemd-logind[1543]: Removed session 3. Aug 13 01:25:00.025107 systemd[1]: Started sshd@3-172.234.25.215:22-147.75.109.163:51648.service - OpenSSH per-connection server daemon (147.75.109.163:51648). Aug 13 01:25:00.359877 sshd[1747]: Accepted publickey for core from 147.75.109.163 port 51648 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:25:00.361274 sshd-session[1747]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:25:00.365503 systemd-logind[1543]: New session 4 of user core. Aug 13 01:25:00.371865 systemd[1]: Started session-4.scope - Session 4 of User core. Aug 13 01:25:00.607575 sshd[1749]: Connection closed by 147.75.109.163 port 51648 Aug 13 01:25:00.608054 sshd-session[1747]: pam_unix(sshd:session): session closed for user core Aug 13 01:25:00.610840 systemd[1]: sshd@3-172.234.25.215:22-147.75.109.163:51648.service: Deactivated successfully. Aug 13 01:25:00.612562 systemd[1]: session-4.scope: Deactivated successfully. Aug 13 01:25:00.613932 systemd-logind[1543]: Session 4 logged out. Waiting for processes to exit. Aug 13 01:25:00.615651 systemd-logind[1543]: Removed session 4. Aug 13 01:25:00.672539 systemd[1]: Started sshd@4-172.234.25.215:22-147.75.109.163:51652.service - OpenSSH per-connection server daemon (147.75.109.163:51652). Aug 13 01:25:01.010834 sshd[1755]: Accepted publickey for core from 147.75.109.163 port 51652 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:25:01.012218 sshd-session[1755]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:25:01.017087 systemd-logind[1543]: New session 5 of user core. Aug 13 01:25:01.023842 systemd[1]: Started session-5.scope - Session 5 of User core. Aug 13 01:25:01.214921 sudo[1758]: core : PWD=/home/core ; USER=root ; COMMAND=/usr/sbin/setenforce 1 Aug 13 01:25:01.215232 sudo[1758]: pam_unix(sudo:session): session opened for user root(uid=0) by core(uid=500) Aug 13 01:25:01.231615 sudo[1758]: pam_unix(sudo:session): session closed for user root Aug 13 01:25:01.282502 sshd[1757]: Connection closed by 147.75.109.163 port 51652 Aug 13 01:25:01.283208 sshd-session[1755]: pam_unix(sshd:session): session closed for user core Aug 13 01:25:01.287904 systemd-logind[1543]: Session 5 logged out. Waiting for processes to exit. Aug 13 01:25:01.288161 systemd[1]: sshd@4-172.234.25.215:22-147.75.109.163:51652.service: Deactivated successfully. Aug 13 01:25:01.289900 systemd[1]: session-5.scope: Deactivated successfully. Aug 13 01:25:01.292044 systemd-logind[1543]: Removed session 5. Aug 13 01:25:01.349169 systemd[1]: Started sshd@5-172.234.25.215:22-147.75.109.163:51658.service - OpenSSH per-connection server daemon (147.75.109.163:51658). Aug 13 01:25:01.710046 sshd[1764]: Accepted publickey for core from 147.75.109.163 port 51658 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:25:01.712487 sshd-session[1764]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:25:01.718891 systemd-logind[1543]: New session 6 of user core. Aug 13 01:25:01.725878 systemd[1]: Started session-6.scope - Session 6 of User core. Aug 13 01:25:01.913081 sudo[1768]: core : PWD=/home/core ; USER=root ; COMMAND=/usr/sbin/rm -rf /etc/audit/rules.d/80-selinux.rules /etc/audit/rules.d/99-default.rules Aug 13 01:25:01.913381 sudo[1768]: pam_unix(sudo:session): session opened for user root(uid=0) by core(uid=500) Aug 13 01:25:01.919377 sudo[1768]: pam_unix(sudo:session): session closed for user root Aug 13 01:25:01.925693 sudo[1767]: core : PWD=/home/core ; USER=root ; COMMAND=/usr/sbin/systemctl restart audit-rules Aug 13 01:25:01.926059 sudo[1767]: pam_unix(sudo:session): session opened for user root(uid=0) by core(uid=500) Aug 13 01:25:01.936852 systemd[1]: Starting audit-rules.service - Load Audit Rules... Aug 13 01:25:01.977937 augenrules[1790]: No rules Aug 13 01:25:01.979438 systemd[1]: audit-rules.service: Deactivated successfully. Aug 13 01:25:01.979776 systemd[1]: Finished audit-rules.service - Load Audit Rules. Aug 13 01:25:01.980893 sudo[1767]: pam_unix(sudo:session): session closed for user root Aug 13 01:25:02.033225 sshd[1766]: Connection closed by 147.75.109.163 port 51658 Aug 13 01:25:02.033679 sshd-session[1764]: pam_unix(sshd:session): session closed for user core Aug 13 01:25:02.038310 systemd[1]: sshd@5-172.234.25.215:22-147.75.109.163:51658.service: Deactivated successfully. Aug 13 01:25:02.038477 systemd-logind[1543]: Session 6 logged out. Waiting for processes to exit. Aug 13 01:25:02.040623 systemd[1]: session-6.scope: Deactivated successfully. Aug 13 01:25:02.042498 systemd-logind[1543]: Removed session 6. Aug 13 01:25:02.095925 systemd[1]: Started sshd@6-172.234.25.215:22-147.75.109.163:51660.service - OpenSSH per-connection server daemon (147.75.109.163:51660). Aug 13 01:25:02.430703 sshd[1799]: Accepted publickey for core from 147.75.109.163 port 51660 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:25:02.432494 sshd-session[1799]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:25:02.438556 systemd-logind[1543]: New session 7 of user core. Aug 13 01:25:02.443827 systemd[1]: Started session-7.scope - Session 7 of User core. Aug 13 01:25:02.624532 sudo[1802]: core : PWD=/home/core ; USER=root ; COMMAND=/home/core/install.sh Aug 13 01:25:02.624853 sudo[1802]: pam_unix(sudo:session): session opened for user root(uid=0) by core(uid=500) Aug 13 01:25:02.929495 systemd[1]: Starting docker.service - Docker Application Container Engine... Aug 13 01:25:02.940024 (dockerd)[1820]: docker.service: Referenced but unset environment variable evaluates to an empty string: DOCKER_CGROUPS, DOCKER_OPTS, DOCKER_OPT_BIP, DOCKER_OPT_IPMASQ, DOCKER_OPT_MTU Aug 13 01:25:03.149284 dockerd[1820]: time="2025-08-13T01:25:03.149220870Z" level=info msg="Starting up" Aug 13 01:25:03.150142 dockerd[1820]: time="2025-08-13T01:25:03.150111340Z" level=info msg="OTEL tracing is not configured, using no-op tracer provider" Aug 13 01:25:03.205579 dockerd[1820]: time="2025-08-13T01:25:03.205318198Z" level=info msg="Loading containers: start." Aug 13 01:25:03.215739 kernel: Initializing XFRM netlink socket Aug 13 01:25:03.475274 systemd-networkd[1449]: docker0: Link UP Aug 13 01:25:03.478015 dockerd[1820]: time="2025-08-13T01:25:03.477966714Z" level=info msg="Loading containers: done." Aug 13 01:25:03.495582 dockerd[1820]: time="2025-08-13T01:25:03.495481393Z" level=warning msg="Not using native diff for overlay2, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled" storage-driver=overlay2 Aug 13 01:25:03.495749 dockerd[1820]: time="2025-08-13T01:25:03.495599143Z" level=info msg="Docker daemon" commit=bbd0a17ccc67e48d4a69393287b7fcc4f0578683 containerd-snapshotter=false storage-driver=overlay2 version=28.0.1 Aug 13 01:25:03.495749 dockerd[1820]: time="2025-08-13T01:25:03.495705593Z" level=info msg="Initializing buildkit" Aug 13 01:25:03.496869 systemd[1]: var-lib-docker-overlay2-opaque\x2dbug\x2dcheck999191847-merged.mount: Deactivated successfully. Aug 13 01:25:03.516045 dockerd[1820]: time="2025-08-13T01:25:03.516014503Z" level=info msg="Completed buildkit initialization" Aug 13 01:25:03.523576 dockerd[1820]: time="2025-08-13T01:25:03.523536007Z" level=info msg="Daemon has completed initialization" Aug 13 01:25:03.523781 systemd[1]: Started docker.service - Docker Application Container Engine. Aug 13 01:25:03.525074 dockerd[1820]: time="2025-08-13T01:25:03.524710097Z" level=info msg="API listen on /run/docker.sock" Aug 13 01:25:04.042368 containerd[1567]: time="2025-08-13T01:25:04.042327376Z" level=info msg="PullImage \"registry.k8s.io/kube-apiserver:v1.33.3\"" Aug 13 01:25:04.838990 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount2733162471.mount: Deactivated successfully. Aug 13 01:25:05.774663 containerd[1567]: time="2025-08-13T01:25:05.773778421Z" level=info msg="ImageCreate event name:\"registry.k8s.io/kube-apiserver:v1.33.3\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:05.774663 containerd[1567]: time="2025-08-13T01:25:05.774459801Z" level=info msg="stop pulling image registry.k8s.io/kube-apiserver:v1.33.3: active requests=0, bytes read=30078237" Aug 13 01:25:05.775142 containerd[1567]: time="2025-08-13T01:25:05.775120202Z" level=info msg="ImageCreate event name:\"sha256:a92b4b92a991677d355596cc4aa9b0b12cbc38e8cbdc1e476548518ae045bc4a\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:05.776943 containerd[1567]: time="2025-08-13T01:25:05.776904243Z" level=info msg="ImageCreate event name:\"registry.k8s.io/kube-apiserver@sha256:125a8b488def5ea24e2de5682ab1abf063163aae4d89ce21811a45f3ecf23816\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:05.777929 containerd[1567]: time="2025-08-13T01:25:05.777908123Z" level=info msg="Pulled image \"registry.k8s.io/kube-apiserver:v1.33.3\" with image id \"sha256:a92b4b92a991677d355596cc4aa9b0b12cbc38e8cbdc1e476548518ae045bc4a\", repo tag \"registry.k8s.io/kube-apiserver:v1.33.3\", repo digest \"registry.k8s.io/kube-apiserver@sha256:125a8b488def5ea24e2de5682ab1abf063163aae4d89ce21811a45f3ecf23816\", size \"30075037\" in 1.735542187s" Aug 13 01:25:05.777995 containerd[1567]: time="2025-08-13T01:25:05.777981763Z" level=info msg="PullImage \"registry.k8s.io/kube-apiserver:v1.33.3\" returns image reference \"sha256:a92b4b92a991677d355596cc4aa9b0b12cbc38e8cbdc1e476548518ae045bc4a\"" Aug 13 01:25:05.778759 containerd[1567]: time="2025-08-13T01:25:05.778735674Z" level=info msg="PullImage \"registry.k8s.io/kube-controller-manager:v1.33.3\"" Aug 13 01:25:06.462381 systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 1. Aug 13 01:25:06.464186 systemd[1]: Starting kubelet.service - kubelet: The Kubernetes Node Agent... Aug 13 01:25:06.644232 systemd[1]: Started kubelet.service - kubelet: The Kubernetes Node Agent. Aug 13 01:25:06.653091 (kubelet)[2085]: kubelet.service: Referenced but unset environment variable evaluates to an empty string: KUBELET_EXTRA_ARGS, KUBELET_KUBEADM_ARGS Aug 13 01:25:06.698422 kubelet[2085]: E0813 01:25:06.698159 2085 run.go:72] "command failed" err="failed to load kubelet config file, path: /var/lib/kubelet/config.yaml, error: failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to read kubelet config file \"/var/lib/kubelet/config.yaml\", error: open /var/lib/kubelet/config.yaml: no such file or directory" Aug 13 01:25:06.704336 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE Aug 13 01:25:06.704569 systemd[1]: kubelet.service: Failed with result 'exit-code'. Aug 13 01:25:06.705814 systemd[1]: kubelet.service: Consumed 194ms CPU time, 109.2M memory peak. Aug 13 01:25:07.134877 containerd[1567]: time="2025-08-13T01:25:07.134814431Z" level=info msg="ImageCreate event name:\"registry.k8s.io/kube-controller-manager:v1.33.3\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:07.136157 containerd[1567]: time="2025-08-13T01:25:07.136123362Z" level=info msg="stop pulling image registry.k8s.io/kube-controller-manager:v1.33.3: active requests=0, bytes read=26019361" Aug 13 01:25:07.136671 containerd[1567]: time="2025-08-13T01:25:07.136228912Z" level=info msg="ImageCreate event name:\"sha256:bf97fadcef43049604abcf0caf4f35229fbee25bd0cdb6fdc1d2bbb4f03d9660\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:07.139241 containerd[1567]: time="2025-08-13T01:25:07.139201453Z" level=info msg="ImageCreate event name:\"registry.k8s.io/kube-controller-manager@sha256:96091626e37c5d5920ee6c3203b783cc01a08f287ec0713aeb7809bb62ccea90\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:07.141246 containerd[1567]: time="2025-08-13T01:25:07.141213244Z" level=info msg="Pulled image \"registry.k8s.io/kube-controller-manager:v1.33.3\" with image id \"sha256:bf97fadcef43049604abcf0caf4f35229fbee25bd0cdb6fdc1d2bbb4f03d9660\", repo tag \"registry.k8s.io/kube-controller-manager:v1.33.3\", repo digest \"registry.k8s.io/kube-controller-manager@sha256:96091626e37c5d5920ee6c3203b783cc01a08f287ec0713aeb7809bb62ccea90\", size \"27646922\" in 1.36245325s" Aug 13 01:25:07.141246 containerd[1567]: time="2025-08-13T01:25:07.141240474Z" level=info msg="PullImage \"registry.k8s.io/kube-controller-manager:v1.33.3\" returns image reference \"sha256:bf97fadcef43049604abcf0caf4f35229fbee25bd0cdb6fdc1d2bbb4f03d9660\"" Aug 13 01:25:07.142132 containerd[1567]: time="2025-08-13T01:25:07.142078755Z" level=info msg="PullImage \"registry.k8s.io/kube-scheduler:v1.33.3\"" Aug 13 01:25:08.310997 containerd[1567]: time="2025-08-13T01:25:08.310943319Z" level=info msg="ImageCreate event name:\"registry.k8s.io/kube-scheduler:v1.33.3\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:08.311893 containerd[1567]: time="2025-08-13T01:25:08.311740189Z" level=info msg="stop pulling image registry.k8s.io/kube-scheduler:v1.33.3: active requests=0, bytes read=20155013" Aug 13 01:25:08.312345 containerd[1567]: time="2025-08-13T01:25:08.312320549Z" level=info msg="ImageCreate event name:\"sha256:41376797d5122e388663ab6d0ad583e58cff63e1a0f1eebfb31d615d8f1c1c87\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:08.314521 containerd[1567]: time="2025-08-13T01:25:08.314499801Z" level=info msg="ImageCreate event name:\"registry.k8s.io/kube-scheduler@sha256:f3a2ffdd7483168205236f7762e9a1933f17dd733bc0188b52bddab9c0762868\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:08.315321 containerd[1567]: time="2025-08-13T01:25:08.315300471Z" level=info msg="Pulled image \"registry.k8s.io/kube-scheduler:v1.33.3\" with image id \"sha256:41376797d5122e388663ab6d0ad583e58cff63e1a0f1eebfb31d615d8f1c1c87\", repo tag \"registry.k8s.io/kube-scheduler:v1.33.3\", repo digest \"registry.k8s.io/kube-scheduler@sha256:f3a2ffdd7483168205236f7762e9a1933f17dd733bc0188b52bddab9c0762868\", size \"21782592\" in 1.173194396s" Aug 13 01:25:08.315393 containerd[1567]: time="2025-08-13T01:25:08.315379811Z" level=info msg="PullImage \"registry.k8s.io/kube-scheduler:v1.33.3\" returns image reference \"sha256:41376797d5122e388663ab6d0ad583e58cff63e1a0f1eebfb31d615d8f1c1c87\"" Aug 13 01:25:08.316358 containerd[1567]: time="2025-08-13T01:25:08.316153951Z" level=info msg="PullImage \"registry.k8s.io/kube-proxy:v1.33.3\"" Aug 13 01:25:09.478109 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount1865793609.mount: Deactivated successfully. Aug 13 01:25:09.834584 containerd[1567]: time="2025-08-13T01:25:09.833841250Z" level=info msg="ImageCreate event name:\"registry.k8s.io/kube-proxy:v1.33.3\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:09.834584 containerd[1567]: time="2025-08-13T01:25:09.834462770Z" level=info msg="stop pulling image registry.k8s.io/kube-proxy:v1.33.3: active requests=0, bytes read=31892666" Aug 13 01:25:09.835287 containerd[1567]: time="2025-08-13T01:25:09.835239940Z" level=info msg="ImageCreate event name:\"sha256:af855adae796077ff822e22c0102f686b2ca7b7c51948889b1825388eaac9234\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:09.836421 containerd[1567]: time="2025-08-13T01:25:09.836390021Z" level=info msg="ImageCreate event name:\"registry.k8s.io/kube-proxy@sha256:c69929cfba9e38305eb1e20ca859aeb90e0d2a7326eab9bb1e8298882fe626cd\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:09.836973 containerd[1567]: time="2025-08-13T01:25:09.836949421Z" level=info msg="Pulled image \"registry.k8s.io/kube-proxy:v1.33.3\" with image id \"sha256:af855adae796077ff822e22c0102f686b2ca7b7c51948889b1825388eaac9234\", repo tag \"registry.k8s.io/kube-proxy:v1.33.3\", repo digest \"registry.k8s.io/kube-proxy@sha256:c69929cfba9e38305eb1e20ca859aeb90e0d2a7326eab9bb1e8298882fe626cd\", size \"31891685\" in 1.52074511s" Aug 13 01:25:09.837051 containerd[1567]: time="2025-08-13T01:25:09.837035291Z" level=info msg="PullImage \"registry.k8s.io/kube-proxy:v1.33.3\" returns image reference \"sha256:af855adae796077ff822e22c0102f686b2ca7b7c51948889b1825388eaac9234\"" Aug 13 01:25:09.837597 containerd[1567]: time="2025-08-13T01:25:09.837574342Z" level=info msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\"" Aug 13 01:25:10.555378 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount3921082615.mount: Deactivated successfully. Aug 13 01:25:11.316623 containerd[1567]: time="2025-08-13T01:25:11.315813340Z" level=info msg="ImageCreate event name:\"registry.k8s.io/coredns/coredns:v1.12.0\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:11.316623 containerd[1567]: time="2025-08-13T01:25:11.316488020Z" level=info msg="stop pulling image registry.k8s.io/coredns/coredns:v1.12.0: active requests=0, bytes read=20942238" Aug 13 01:25:11.317334 containerd[1567]: time="2025-08-13T01:25:11.317299571Z" level=info msg="ImageCreate event name:\"sha256:1cf5f116067c67da67f97bff78c4bbc76913f59057c18627b96facaced73ea0b\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:11.319257 containerd[1567]: time="2025-08-13T01:25:11.319231572Z" level=info msg="ImageCreate event name:\"registry.k8s.io/coredns/coredns@sha256:40384aa1f5ea6bfdc77997d243aec73da05f27aed0c5e9d65bfa98933c519d97\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:11.320155 containerd[1567]: time="2025-08-13T01:25:11.320130882Z" level=info msg="Pulled image \"registry.k8s.io/coredns/coredns:v1.12.0\" with image id \"sha256:1cf5f116067c67da67f97bff78c4bbc76913f59057c18627b96facaced73ea0b\", repo tag \"registry.k8s.io/coredns/coredns:v1.12.0\", repo digest \"registry.k8s.io/coredns/coredns@sha256:40384aa1f5ea6bfdc77997d243aec73da05f27aed0c5e9d65bfa98933c519d97\", size \"20939036\" in 1.4825287s" Aug 13 01:25:11.320233 containerd[1567]: time="2025-08-13T01:25:11.320218352Z" level=info msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\" returns image reference \"sha256:1cf5f116067c67da67f97bff78c4bbc76913f59057c18627b96facaced73ea0b\"" Aug 13 01:25:11.321121 containerd[1567]: time="2025-08-13T01:25:11.321086253Z" level=info msg="PullImage \"registry.k8s.io/pause:3.10\"" Aug 13 01:25:11.985628 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount2134249244.mount: Deactivated successfully. Aug 13 01:25:11.989516 containerd[1567]: time="2025-08-13T01:25:11.989470017Z" level=info msg="ImageCreate event name:\"registry.k8s.io/pause:3.10\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"} labels:{key:\"io.cri-containerd.pinned\" value:\"pinned\"}" Aug 13 01:25:11.990195 containerd[1567]: time="2025-08-13T01:25:11.990156457Z" level=info msg="stop pulling image registry.k8s.io/pause:3.10: active requests=0, bytes read=321138" Aug 13 01:25:11.990733 containerd[1567]: time="2025-08-13T01:25:11.990683127Z" level=info msg="ImageCreate event name:\"sha256:873ed75102791e5b0b8a7fcd41606c92fcec98d56d05ead4ac5131650004c136\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"} labels:{key:\"io.cri-containerd.pinned\" value:\"pinned\"}" Aug 13 01:25:11.992535 containerd[1567]: time="2025-08-13T01:25:11.992487738Z" level=info msg="ImageCreate event name:\"registry.k8s.io/pause@sha256:ee6521f290b2168b6e0935a181d4cff9be1ac3f505666ef0e3c98fae8199917a\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"} labels:{key:\"io.cri-containerd.pinned\" value:\"pinned\"}" Aug 13 01:25:11.992911 containerd[1567]: time="2025-08-13T01:25:11.992880778Z" level=info msg="Pulled image \"registry.k8s.io/pause:3.10\" with image id \"sha256:873ed75102791e5b0b8a7fcd41606c92fcec98d56d05ead4ac5131650004c136\", repo tag \"registry.k8s.io/pause:3.10\", repo digest \"registry.k8s.io/pause@sha256:ee6521f290b2168b6e0935a181d4cff9be1ac3f505666ef0e3c98fae8199917a\", size \"320368\" in 671.743955ms" Aug 13 01:25:11.992954 containerd[1567]: time="2025-08-13T01:25:11.992913278Z" level=info msg="PullImage \"registry.k8s.io/pause:3.10\" returns image reference \"sha256:873ed75102791e5b0b8a7fcd41606c92fcec98d56d05ead4ac5131650004c136\"" Aug 13 01:25:11.993927 containerd[1567]: time="2025-08-13T01:25:11.993897879Z" level=info msg="PullImage \"registry.k8s.io/etcd:3.5.21-0\"" Aug 13 01:25:12.675113 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount53945860.mount: Deactivated successfully. Aug 13 01:25:13.902303 containerd[1567]: time="2025-08-13T01:25:13.902256112Z" level=info msg="ImageCreate event name:\"registry.k8s.io/etcd:3.5.21-0\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:13.903210 containerd[1567]: time="2025-08-13T01:25:13.903183423Z" level=info msg="stop pulling image registry.k8s.io/etcd:3.5.21-0: active requests=0, bytes read=58247175" Aug 13 01:25:13.903782 containerd[1567]: time="2025-08-13T01:25:13.903702723Z" level=info msg="ImageCreate event name:\"sha256:499038711c0816eda03a1ad96a8eb0440c005baa6949698223c6176b7f5077e1\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:13.906050 containerd[1567]: time="2025-08-13T01:25:13.906028714Z" level=info msg="ImageCreate event name:\"registry.k8s.io/etcd@sha256:d58c035df557080a27387d687092e3fc2b64c6d0e3162dc51453a115f847d121\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:13.907102 containerd[1567]: time="2025-08-13T01:25:13.906950355Z" level=info msg="Pulled image \"registry.k8s.io/etcd:3.5.21-0\" with image id \"sha256:499038711c0816eda03a1ad96a8eb0440c005baa6949698223c6176b7f5077e1\", repo tag \"registry.k8s.io/etcd:3.5.21-0\", repo digest \"registry.k8s.io/etcd@sha256:d58c035df557080a27387d687092e3fc2b64c6d0e3162dc51453a115f847d121\", size \"58938593\" in 1.913024856s" Aug 13 01:25:13.907102 containerd[1567]: time="2025-08-13T01:25:13.906976465Z" level=info msg="PullImage \"registry.k8s.io/etcd:3.5.21-0\" returns image reference \"sha256:499038711c0816eda03a1ad96a8eb0440c005baa6949698223c6176b7f5077e1\"" Aug 13 01:25:15.884791 systemd[1]: Stopped kubelet.service - kubelet: The Kubernetes Node Agent. Aug 13 01:25:15.884940 systemd[1]: kubelet.service: Consumed 194ms CPU time, 109.2M memory peak. Aug 13 01:25:15.887334 systemd[1]: Starting kubelet.service - kubelet: The Kubernetes Node Agent... Aug 13 01:25:15.916301 systemd[1]: Reload requested from client PID 2247 ('systemctl') (unit session-7.scope)... Aug 13 01:25:15.916392 systemd[1]: Reloading... Aug 13 01:25:16.043794 zram_generator::config[2294]: No configuration found. Aug 13 01:25:16.136451 systemd[1]: /usr/lib/systemd/system/docker.socket:6: ListenStream= references a path below legacy directory /var/run/, updating /var/run/docker.sock → /run/docker.sock; please update the unit file accordingly. Aug 13 01:25:16.243212 systemd[1]: Reloading finished in 326 ms. Aug 13 01:25:16.307508 systemd[1]: Stopping kubelet.service - kubelet: The Kubernetes Node Agent... Aug 13 01:25:16.311157 systemd[1]: kubelet.service: Deactivated successfully. Aug 13 01:25:16.311412 systemd[1]: Stopped kubelet.service - kubelet: The Kubernetes Node Agent. Aug 13 01:25:16.311448 systemd[1]: kubelet.service: Consumed 140ms CPU time, 98.3M memory peak. Aug 13 01:25:16.314068 systemd[1]: Starting kubelet.service - kubelet: The Kubernetes Node Agent... Aug 13 01:25:16.484432 systemd[1]: Started kubelet.service - kubelet: The Kubernetes Node Agent. Aug 13 01:25:16.491006 (kubelet)[2347]: kubelet.service: Referenced but unset environment variable evaluates to an empty string: KUBELET_EXTRA_ARGS Aug 13 01:25:16.528491 kubelet[2347]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 13 01:25:16.528491 kubelet[2347]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 13 01:25:16.528491 kubelet[2347]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 13 01:25:16.528795 kubelet[2347]: I0813 01:25:16.528576 2347 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 13 01:25:16.770818 kubelet[2347]: I0813 01:25:16.770445 2347 server.go:530] "Kubelet version" kubeletVersion="v1.33.0" Aug 13 01:25:16.770818 kubelet[2347]: I0813 01:25:16.770468 2347 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 13 01:25:16.770959 kubelet[2347]: I0813 01:25:16.770908 2347 server.go:956] "Client rotation is on, will bootstrap in background" Aug 13 01:25:16.804365 kubelet[2347]: E0813 01:25:16.804312 2347 certificate_manager.go:596] "Failed while requesting a signed certificate from the control plane" err="cannot create certificate signing request: Post \"https://172.234.25.215:6443/apis/certificates.k8s.io/v1/certificatesigningrequests\": dial tcp 172.234.25.215:6443: connect: connection refused" logger="kubernetes.io/kube-apiserver-client-kubelet.UnhandledError" Aug 13 01:25:16.805240 kubelet[2347]: I0813 01:25:16.805211 2347 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" Aug 13 01:25:16.812545 kubelet[2347]: I0813 01:25:16.812518 2347 server.go:1446] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 13 01:25:16.817744 kubelet[2347]: I0813 01:25:16.816988 2347 server.go:782] "--cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /" Aug 13 01:25:16.817744 kubelet[2347]: I0813 01:25:16.817275 2347 container_manager_linux.go:267] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 13 01:25:16.817744 kubelet[2347]: I0813 01:25:16.817307 2347 container_manager_linux.go:272] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"172-234-25-215","RuntimeCgroupsName":"","SystemCgroupsName":"","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":false,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":null,"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":-1,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 13 01:25:16.817744 kubelet[2347]: I0813 01:25:16.817478 2347 topology_manager.go:138] "Creating topology manager with none policy" Aug 13 01:25:16.817947 kubelet[2347]: I0813 01:25:16.817490 2347 container_manager_linux.go:303] "Creating device plugin manager" Aug 13 01:25:16.817947 kubelet[2347]: I0813 01:25:16.817594 2347 state_mem.go:36] "Initialized new in-memory state store" Aug 13 01:25:16.820037 kubelet[2347]: I0813 01:25:16.820013 2347 kubelet.go:480] "Attempting to sync node with API server" Aug 13 01:25:16.820098 kubelet[2347]: I0813 01:25:16.820041 2347 kubelet.go:375] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 13 01:25:16.820098 kubelet[2347]: I0813 01:25:16.820071 2347 kubelet.go:386] "Adding apiserver pod source" Aug 13 01:25:16.820098 kubelet[2347]: I0813 01:25:16.820092 2347 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 13 01:25:16.831669 kubelet[2347]: E0813 01:25:16.831634 2347 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: Get \"https://172.234.25.215:6443/api/v1/services?fieldSelector=spec.clusterIP%21%3DNone&limit=500&resourceVersion=0\": dial tcp 172.234.25.215:6443: connect: connection refused" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 13 01:25:16.832335 kubelet[2347]: E0813 01:25:16.831772 2347 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: Get \"https://172.234.25.215:6443/api/v1/nodes?fieldSelector=metadata.name%3D172-234-25-215&limit=500&resourceVersion=0\": dial tcp 172.234.25.215:6443: connect: connection refused" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 13 01:25:16.832335 kubelet[2347]: I0813 01:25:16.831864 2347 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="containerd" version="v2.0.4" apiVersion="v1" Aug 13 01:25:16.832335 kubelet[2347]: I0813 01:25:16.832236 2347 kubelet.go:935] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 13 01:25:16.832979 kubelet[2347]: W0813 01:25:16.832950 2347 probe.go:272] Flexvolume plugin directory at /opt/libexec/kubernetes/kubelet-plugins/volume/exec/ does not exist. Recreating. Aug 13 01:25:16.835355 kubelet[2347]: I0813 01:25:16.835329 2347 watchdog_linux.go:99] "Systemd watchdog is not enabled" Aug 13 01:25:16.835403 kubelet[2347]: I0813 01:25:16.835387 2347 server.go:1289] "Started kubelet" Aug 13 01:25:16.836024 kubelet[2347]: I0813 01:25:16.835977 2347 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 13 01:25:16.841750 kubelet[2347]: I0813 01:25:16.841319 2347 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 13 01:25:16.841750 kubelet[2347]: I0813 01:25:16.841644 2347 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 13 01:25:16.842227 kubelet[2347]: I0813 01:25:16.842193 2347 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 13 01:25:16.843925 kubelet[2347]: I0813 01:25:16.843909 2347 server.go:317] "Adding debug handlers to kubelet server" Aug 13 01:25:16.844766 kubelet[2347]: I0813 01:25:16.844749 2347 dynamic_serving_content.go:135] "Starting controller" name="kubelet-server-cert-files::/var/lib/kubelet/pki/kubelet.crt::/var/lib/kubelet/pki/kubelet.key" Aug 13 01:25:16.847970 kubelet[2347]: I0813 01:25:16.847850 2347 volume_manager.go:297] "Starting Kubelet Volume Manager" Aug 13 01:25:16.848762 kubelet[2347]: E0813 01:25:16.848103 2347 kubelet_node_status.go:466] "Error getting the current node from lister" err="node \"172-234-25-215\" not found" Aug 13 01:25:16.848895 kubelet[2347]: I0813 01:25:16.848856 2347 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Aug 13 01:25:16.848967 kubelet[2347]: I0813 01:25:16.848935 2347 reconciler.go:26] "Reconciler: start to sync state" Aug 13 01:25:16.852928 kubelet[2347]: E0813 01:25:16.849263 2347 event.go:368] "Unable to write event (may retry after sleeping)" err="Post \"https://172.234.25.215:6443/api/v1/namespaces/default/events\": dial tcp 172.234.25.215:6443: connect: connection refused" event="&Event{ObjectMeta:{172-234-25-215.185b2f25409e4e40 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:172-234-25-215,UID:172-234-25-215,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:172-234-25-215,},FirstTimestamp:2025-08-13 01:25:16.835352128 +0000 UTC m=+0.339548541,LastTimestamp:2025-08-13 01:25:16.835352128 +0000 UTC m=+0.339548541,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:172-234-25-215,}" Aug 13 01:25:16.853363 kubelet[2347]: E0813 01:25:16.853280 2347 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: Get \"https://172.234.25.215:6443/apis/storage.k8s.io/v1/csidrivers?limit=500&resourceVersion=0\": dial tcp 172.234.25.215:6443: connect: connection refused" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Aug 13 01:25:16.853485 kubelet[2347]: E0813 01:25:16.853344 2347 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://172.234.25.215:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/172-234-25-215?timeout=10s\": dial tcp 172.234.25.215:6443: connect: connection refused" interval="200ms" Aug 13 01:25:16.853656 kubelet[2347]: I0813 01:25:16.853634 2347 factory.go:223] Registration of the systemd container factory successfully Aug 13 01:25:16.853782 kubelet[2347]: I0813 01:25:16.853757 2347 factory.go:221] Registration of the crio container factory failed: Get "http://%2Fvar%2Frun%2Fcrio%2Fcrio.sock/info": dial unix /var/run/crio/crio.sock: connect: no such file or directory Aug 13 01:25:16.856211 kubelet[2347]: I0813 01:25:16.856185 2347 factory.go:223] Registration of the containerd container factory successfully Aug 13 01:25:16.858107 kubelet[2347]: E0813 01:25:16.858030 2347 kubelet.go:1600] "Image garbage collection failed once. Stats initialization may not have completed yet" err="invalid capacity 0 on image filesystem" Aug 13 01:25:16.871740 kubelet[2347]: I0813 01:25:16.871403 2347 cpu_manager.go:221] "Starting CPU manager" policy="none" Aug 13 01:25:16.871740 kubelet[2347]: I0813 01:25:16.871422 2347 cpu_manager.go:222] "Reconciling" reconcilePeriod="10s" Aug 13 01:25:16.871740 kubelet[2347]: I0813 01:25:16.871438 2347 state_mem.go:36] "Initialized new in-memory state store" Aug 13 01:25:16.873255 kubelet[2347]: I0813 01:25:16.873067 2347 policy_none.go:49] "None policy: Start" Aug 13 01:25:16.873255 kubelet[2347]: I0813 01:25:16.873087 2347 memory_manager.go:186] "Starting memorymanager" policy="None" Aug 13 01:25:16.873255 kubelet[2347]: I0813 01:25:16.873098 2347 state_mem.go:35] "Initializing new in-memory state store" Aug 13 01:25:16.882593 kubelet[2347]: I0813 01:25:16.882443 2347 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Aug 13 01:25:16.882547 systemd[1]: Created slice kubepods.slice - libcontainer container kubepods.slice. Aug 13 01:25:16.884733 kubelet[2347]: I0813 01:25:16.883920 2347 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Aug 13 01:25:16.884733 kubelet[2347]: I0813 01:25:16.883943 2347 status_manager.go:230] "Starting to sync pod status with apiserver" Aug 13 01:25:16.884733 kubelet[2347]: I0813 01:25:16.883959 2347 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 13 01:25:16.884733 kubelet[2347]: I0813 01:25:16.883965 2347 kubelet.go:2436] "Starting kubelet main sync loop" Aug 13 01:25:16.884733 kubelet[2347]: E0813 01:25:16.884012 2347 kubelet.go:2460] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Aug 13 01:25:16.886245 kubelet[2347]: E0813 01:25:16.886226 2347 reflector.go:200] "Failed to watch" err="failed to list *v1.RuntimeClass: Get \"https://172.234.25.215:6443/apis/node.k8s.io/v1/runtimeclasses?limit=500&resourceVersion=0\": dial tcp 172.234.25.215:6443: connect: connection refused" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Aug 13 01:25:16.894124 systemd[1]: Created slice kubepods-burstable.slice - libcontainer container kubepods-burstable.slice. Aug 13 01:25:16.897464 systemd[1]: Created slice kubepods-besteffort.slice - libcontainer container kubepods-besteffort.slice. Aug 13 01:25:16.912808 kubelet[2347]: E0813 01:25:16.912780 2347 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 13 01:25:16.913160 kubelet[2347]: I0813 01:25:16.913147 2347 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 13 01:25:16.913438 kubelet[2347]: I0813 01:25:16.913416 2347 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 13 01:25:16.913948 kubelet[2347]: I0813 01:25:16.913936 2347 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 13 01:25:16.914987 kubelet[2347]: E0813 01:25:16.914791 2347 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="no imagefs label for configured runtime" Aug 13 01:25:16.915303 kubelet[2347]: E0813 01:25:16.915291 2347 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"172-234-25-215\" not found" Aug 13 01:25:16.995364 systemd[1]: Created slice kubepods-burstable-pod96c354cf60eb29e83ce001f635cc0212.slice - libcontainer container kubepods-burstable-pod96c354cf60eb29e83ce001f635cc0212.slice. Aug 13 01:25:17.001579 kubelet[2347]: E0813 01:25:17.001541 2347 kubelet.go:3305] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"172-234-25-215\" not found" node="172-234-25-215" Aug 13 01:25:17.005121 systemd[1]: Created slice kubepods-burstable-pod03fee088624a1f74688c8facb00ef7f7.slice - libcontainer container kubepods-burstable-pod03fee088624a1f74688c8facb00ef7f7.slice. Aug 13 01:25:17.013062 kubelet[2347]: E0813 01:25:17.012961 2347 kubelet.go:3305] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"172-234-25-215\" not found" node="172-234-25-215" Aug 13 01:25:17.016085 kubelet[2347]: I0813 01:25:17.016037 2347 kubelet_node_status.go:75] "Attempting to register node" node="172-234-25-215" Aug 13 01:25:17.016295 kubelet[2347]: E0813 01:25:17.016264 2347 kubelet_node_status.go:107] "Unable to register node with API server" err="Post \"https://172.234.25.215:6443/api/v1/nodes\": dial tcp 172.234.25.215:6443: connect: connection refused" node="172-234-25-215" Aug 13 01:25:17.018786 systemd[1]: Created slice kubepods-burstable-pod5c228d00d203ca7627372d21a4f939a9.slice - libcontainer container kubepods-burstable-pod5c228d00d203ca7627372d21a4f939a9.slice. Aug 13 01:25:17.021038 kubelet[2347]: E0813 01:25:17.020930 2347 kubelet.go:3305] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"172-234-25-215\" not found" node="172-234-25-215" Aug 13 01:25:17.050192 kubelet[2347]: I0813 01:25:17.050170 2347 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/96c354cf60eb29e83ce001f635cc0212-kubeconfig\") pod \"kube-controller-manager-172-234-25-215\" (UID: \"96c354cf60eb29e83ce001f635cc0212\") " pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:17.050292 kubelet[2347]: I0813 01:25:17.050277 2347 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/03fee088624a1f74688c8facb00ef7f7-kubeconfig\") pod \"kube-scheduler-172-234-25-215\" (UID: \"03fee088624a1f74688c8facb00ef7f7\") " pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:25:17.050398 kubelet[2347]: I0813 01:25:17.050358 2347 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"k8s-certs\" (UniqueName: \"kubernetes.io/host-path/5c228d00d203ca7627372d21a4f939a9-k8s-certs\") pod \"kube-apiserver-172-234-25-215\" (UID: \"5c228d00d203ca7627372d21a4f939a9\") " pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:25:17.050398 kubelet[2347]: I0813 01:25:17.050393 2347 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-certs\" (UniqueName: \"kubernetes.io/host-path/96c354cf60eb29e83ce001f635cc0212-ca-certs\") pod \"kube-controller-manager-172-234-25-215\" (UID: \"96c354cf60eb29e83ce001f635cc0212\") " pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:17.050525 kubelet[2347]: I0813 01:25:17.050413 2347 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"flexvolume-dir\" (UniqueName: \"kubernetes.io/host-path/96c354cf60eb29e83ce001f635cc0212-flexvolume-dir\") pod \"kube-controller-manager-172-234-25-215\" (UID: \"96c354cf60eb29e83ce001f635cc0212\") " pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:17.050525 kubelet[2347]: I0813 01:25:17.050438 2347 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"usr-share-ca-certificates\" (UniqueName: \"kubernetes.io/host-path/96c354cf60eb29e83ce001f635cc0212-usr-share-ca-certificates\") pod \"kube-controller-manager-172-234-25-215\" (UID: \"96c354cf60eb29e83ce001f635cc0212\") " pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:17.050525 kubelet[2347]: I0813 01:25:17.050455 2347 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-certs\" (UniqueName: \"kubernetes.io/host-path/5c228d00d203ca7627372d21a4f939a9-ca-certs\") pod \"kube-apiserver-172-234-25-215\" (UID: \"5c228d00d203ca7627372d21a4f939a9\") " pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:25:17.050525 kubelet[2347]: I0813 01:25:17.050473 2347 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"usr-share-ca-certificates\" (UniqueName: \"kubernetes.io/host-path/5c228d00d203ca7627372d21a4f939a9-usr-share-ca-certificates\") pod \"kube-apiserver-172-234-25-215\" (UID: \"5c228d00d203ca7627372d21a4f939a9\") " pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:25:17.050525 kubelet[2347]: I0813 01:25:17.050496 2347 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"k8s-certs\" (UniqueName: \"kubernetes.io/host-path/96c354cf60eb29e83ce001f635cc0212-k8s-certs\") pod \"kube-controller-manager-172-234-25-215\" (UID: \"96c354cf60eb29e83ce001f635cc0212\") " pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:17.054640 kubelet[2347]: E0813 01:25:17.054587 2347 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://172.234.25.215:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/172-234-25-215?timeout=10s\": dial tcp 172.234.25.215:6443: connect: connection refused" interval="400ms" Aug 13 01:25:17.218399 kubelet[2347]: I0813 01:25:17.218338 2347 kubelet_node_status.go:75] "Attempting to register node" node="172-234-25-215" Aug 13 01:25:17.218623 kubelet[2347]: E0813 01:25:17.218588 2347 kubelet_node_status.go:107] "Unable to register node with API server" err="Post \"https://172.234.25.215:6443/api/v1/nodes\": dial tcp 172.234.25.215:6443: connect: connection refused" node="172-234-25-215" Aug 13 01:25:17.302532 kubelet[2347]: E0813 01:25:17.302452 2347 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:17.303301 containerd[1567]: time="2025-08-13T01:25:17.303215692Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:kube-controller-manager-172-234-25-215,Uid:96c354cf60eb29e83ce001f635cc0212,Namespace:kube-system,Attempt:0,}" Aug 13 01:25:17.316733 kubelet[2347]: E0813 01:25:17.314573 2347 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:17.316905 containerd[1567]: time="2025-08-13T01:25:17.316873049Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:kube-scheduler-172-234-25-215,Uid:03fee088624a1f74688c8facb00ef7f7,Namespace:kube-system,Attempt:0,}" Aug 13 01:25:17.324335 kubelet[2347]: E0813 01:25:17.324309 2347 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:17.324600 containerd[1567]: time="2025-08-13T01:25:17.324578062Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:kube-apiserver-172-234-25-215,Uid:5c228d00d203ca7627372d21a4f939a9,Namespace:kube-system,Attempt:0,}" Aug 13 01:25:17.328955 containerd[1567]: time="2025-08-13T01:25:17.328857995Z" level=info msg="connecting to shim 7d84ed886c0b1326d79ecfd0bcd1de121c3247f9cff6c203229e46496e35dcdb" address="unix:///run/containerd/s/d1921e69e13675f63883a3f813b8fe6e24aba1e75c9bd277aef989df03a34464" namespace=k8s.io protocol=ttrpc version=3 Aug 13 01:25:17.351255 containerd[1567]: time="2025-08-13T01:25:17.351213006Z" level=info msg="connecting to shim 6df0c80144a1579c42fd874e4fdabc1ad503793034fc23e7d723fce185be0263" address="unix:///run/containerd/s/d87dad9077c77421fdfc8556f5ef79c39a21c3306a7e390726065ff4b435f439" namespace=k8s.io protocol=ttrpc version=3 Aug 13 01:25:17.367881 containerd[1567]: time="2025-08-13T01:25:17.367236204Z" level=info msg="connecting to shim 43b994a67168ff22c568e118bfdf83cbf8755570148c8488e918964ae398e1ce" address="unix:///run/containerd/s/c81b187c485bea9e43f5ceeaad923be30ffaf3e6c4eb3b896ddd4c66916e5eb3" namespace=k8s.io protocol=ttrpc version=3 Aug 13 01:25:17.377854 systemd[1]: Started cri-containerd-7d84ed886c0b1326d79ecfd0bcd1de121c3247f9cff6c203229e46496e35dcdb.scope - libcontainer container 7d84ed886c0b1326d79ecfd0bcd1de121c3247f9cff6c203229e46496e35dcdb. Aug 13 01:25:17.390913 systemd[1]: Started cri-containerd-6df0c80144a1579c42fd874e4fdabc1ad503793034fc23e7d723fce185be0263.scope - libcontainer container 6df0c80144a1579c42fd874e4fdabc1ad503793034fc23e7d723fce185be0263. Aug 13 01:25:17.405032 systemd[1]: Started cri-containerd-43b994a67168ff22c568e118bfdf83cbf8755570148c8488e918964ae398e1ce.scope - libcontainer container 43b994a67168ff22c568e118bfdf83cbf8755570148c8488e918964ae398e1ce. Aug 13 01:25:17.458213 kubelet[2347]: E0813 01:25:17.458166 2347 controller.go:145] "Failed to ensure lease exists, will retry" err="Get \"https://172.234.25.215:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/172-234-25-215?timeout=10s\": dial tcp 172.234.25.215:6443: connect: connection refused" interval="800ms" Aug 13 01:25:17.470207 containerd[1567]: time="2025-08-13T01:25:17.469938795Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:kube-controller-manager-172-234-25-215,Uid:96c354cf60eb29e83ce001f635cc0212,Namespace:kube-system,Attempt:0,} returns sandbox id \"7d84ed886c0b1326d79ecfd0bcd1de121c3247f9cff6c203229e46496e35dcdb\"" Aug 13 01:25:17.470937 kubelet[2347]: E0813 01:25:17.470921 2347 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:17.475332 containerd[1567]: time="2025-08-13T01:25:17.475151028Z" level=info msg="CreateContainer within sandbox \"7d84ed886c0b1326d79ecfd0bcd1de121c3247f9cff6c203229e46496e35dcdb\" for container &ContainerMetadata{Name:kube-controller-manager,Attempt:0,}" Aug 13 01:25:17.481217 containerd[1567]: time="2025-08-13T01:25:17.480999311Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:kube-scheduler-172-234-25-215,Uid:03fee088624a1f74688c8facb00ef7f7,Namespace:kube-system,Attempt:0,} returns sandbox id \"6df0c80144a1579c42fd874e4fdabc1ad503793034fc23e7d723fce185be0263\"" Aug 13 01:25:17.482327 kubelet[2347]: E0813 01:25:17.482312 2347 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:17.483528 containerd[1567]: time="2025-08-13T01:25:17.483440972Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:kube-apiserver-172-234-25-215,Uid:5c228d00d203ca7627372d21a4f939a9,Namespace:kube-system,Attempt:0,} returns sandbox id \"43b994a67168ff22c568e118bfdf83cbf8755570148c8488e918964ae398e1ce\"" Aug 13 01:25:17.484426 kubelet[2347]: E0813 01:25:17.484410 2347 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:17.485195 containerd[1567]: time="2025-08-13T01:25:17.485173103Z" level=info msg="Container e175264c33fcc3cbfc4ab096894b87893d82a9339dff54de92520cdc084ea674: CDI devices from CRI Config.CDIDevices: []" Aug 13 01:25:17.485816 containerd[1567]: time="2025-08-13T01:25:17.485763853Z" level=info msg="CreateContainer within sandbox \"6df0c80144a1579c42fd874e4fdabc1ad503793034fc23e7d723fce185be0263\" for container &ContainerMetadata{Name:kube-scheduler,Attempt:0,}" Aug 13 01:25:17.488825 containerd[1567]: time="2025-08-13T01:25:17.488445774Z" level=info msg="CreateContainer within sandbox \"43b994a67168ff22c568e118bfdf83cbf8755570148c8488e918964ae398e1ce\" for container &ContainerMetadata{Name:kube-apiserver,Attempt:0,}" Aug 13 01:25:17.491206 containerd[1567]: time="2025-08-13T01:25:17.491173426Z" level=info msg="CreateContainer within sandbox \"7d84ed886c0b1326d79ecfd0bcd1de121c3247f9cff6c203229e46496e35dcdb\" for &ContainerMetadata{Name:kube-controller-manager,Attempt:0,} returns container id \"e175264c33fcc3cbfc4ab096894b87893d82a9339dff54de92520cdc084ea674\"" Aug 13 01:25:17.492010 containerd[1567]: time="2025-08-13T01:25:17.491990236Z" level=info msg="StartContainer for \"e175264c33fcc3cbfc4ab096894b87893d82a9339dff54de92520cdc084ea674\"" Aug 13 01:25:17.494757 containerd[1567]: time="2025-08-13T01:25:17.494685827Z" level=info msg="connecting to shim e175264c33fcc3cbfc4ab096894b87893d82a9339dff54de92520cdc084ea674" address="unix:///run/containerd/s/d1921e69e13675f63883a3f813b8fe6e24aba1e75c9bd277aef989df03a34464" protocol=ttrpc version=3 Aug 13 01:25:17.495033 containerd[1567]: time="2025-08-13T01:25:17.494989578Z" level=info msg="Container d8abadf258f601795311223ac256947ee2260f53a2d3f5c3b4649ca912a66738: CDI devices from CRI Config.CDIDevices: []" Aug 13 01:25:17.498842 containerd[1567]: time="2025-08-13T01:25:17.498800999Z" level=info msg="Container b2a7ee63e6d55f3bec9cbaaad91d3d9cd19ee86000685da949918d72785c9953: CDI devices from CRI Config.CDIDevices: []" Aug 13 01:25:17.502865 containerd[1567]: time="2025-08-13T01:25:17.502806711Z" level=info msg="CreateContainer within sandbox \"6df0c80144a1579c42fd874e4fdabc1ad503793034fc23e7d723fce185be0263\" for &ContainerMetadata{Name:kube-scheduler,Attempt:0,} returns container id \"d8abadf258f601795311223ac256947ee2260f53a2d3f5c3b4649ca912a66738\"" Aug 13 01:25:17.503300 containerd[1567]: time="2025-08-13T01:25:17.503245142Z" level=info msg="StartContainer for \"d8abadf258f601795311223ac256947ee2260f53a2d3f5c3b4649ca912a66738\"" Aug 13 01:25:17.504240 containerd[1567]: time="2025-08-13T01:25:17.504212492Z" level=info msg="connecting to shim d8abadf258f601795311223ac256947ee2260f53a2d3f5c3b4649ca912a66738" address="unix:///run/containerd/s/d87dad9077c77421fdfc8556f5ef79c39a21c3306a7e390726065ff4b435f439" protocol=ttrpc version=3 Aug 13 01:25:17.508211 containerd[1567]: time="2025-08-13T01:25:17.508143154Z" level=info msg="CreateContainer within sandbox \"43b994a67168ff22c568e118bfdf83cbf8755570148c8488e918964ae398e1ce\" for &ContainerMetadata{Name:kube-apiserver,Attempt:0,} returns container id \"b2a7ee63e6d55f3bec9cbaaad91d3d9cd19ee86000685da949918d72785c9953\"" Aug 13 01:25:17.508838 containerd[1567]: time="2025-08-13T01:25:17.508820454Z" level=info msg="StartContainer for \"b2a7ee63e6d55f3bec9cbaaad91d3d9cd19ee86000685da949918d72785c9953\"" Aug 13 01:25:17.511667 containerd[1567]: time="2025-08-13T01:25:17.511647856Z" level=info msg="connecting to shim b2a7ee63e6d55f3bec9cbaaad91d3d9cd19ee86000685da949918d72785c9953" address="unix:///run/containerd/s/c81b187c485bea9e43f5ceeaad923be30ffaf3e6c4eb3b896ddd4c66916e5eb3" protocol=ttrpc version=3 Aug 13 01:25:17.519843 systemd[1]: Started cri-containerd-e175264c33fcc3cbfc4ab096894b87893d82a9339dff54de92520cdc084ea674.scope - libcontainer container e175264c33fcc3cbfc4ab096894b87893d82a9339dff54de92520cdc084ea674. Aug 13 01:25:17.527843 systemd[1]: Started cri-containerd-d8abadf258f601795311223ac256947ee2260f53a2d3f5c3b4649ca912a66738.scope - libcontainer container d8abadf258f601795311223ac256947ee2260f53a2d3f5c3b4649ca912a66738. Aug 13 01:25:17.540845 systemd[1]: Started cri-containerd-b2a7ee63e6d55f3bec9cbaaad91d3d9cd19ee86000685da949918d72785c9953.scope - libcontainer container b2a7ee63e6d55f3bec9cbaaad91d3d9cd19ee86000685da949918d72785c9953. Aug 13 01:25:17.615886 containerd[1567]: time="2025-08-13T01:25:17.615790278Z" level=info msg="StartContainer for \"e175264c33fcc3cbfc4ab096894b87893d82a9339dff54de92520cdc084ea674\" returns successfully" Aug 13 01:25:17.621481 containerd[1567]: time="2025-08-13T01:25:17.621393121Z" level=info msg="StartContainer for \"d8abadf258f601795311223ac256947ee2260f53a2d3f5c3b4649ca912a66738\" returns successfully" Aug 13 01:25:17.623037 kubelet[2347]: I0813 01:25:17.622699 2347 kubelet_node_status.go:75] "Attempting to register node" node="172-234-25-215" Aug 13 01:25:17.623680 kubelet[2347]: E0813 01:25:17.623648 2347 kubelet_node_status.go:107] "Unable to register node with API server" err="Post \"https://172.234.25.215:6443/api/v1/nodes\": dial tcp 172.234.25.215:6443: connect: connection refused" node="172-234-25-215" Aug 13 01:25:17.631049 containerd[1567]: time="2025-08-13T01:25:17.631023126Z" level=info msg="StartContainer for \"b2a7ee63e6d55f3bec9cbaaad91d3d9cd19ee86000685da949918d72785c9953\" returns successfully" Aug 13 01:25:17.897026 kubelet[2347]: E0813 01:25:17.896931 2347 kubelet.go:3305] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"172-234-25-215\" not found" node="172-234-25-215" Aug 13 01:25:17.897111 kubelet[2347]: E0813 01:25:17.897041 2347 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:17.899313 kubelet[2347]: E0813 01:25:17.899292 2347 kubelet.go:3305] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"172-234-25-215\" not found" node="172-234-25-215" Aug 13 01:25:17.899391 kubelet[2347]: E0813 01:25:17.899373 2347 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:17.902215 kubelet[2347]: E0813 01:25:17.902194 2347 kubelet.go:3305] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"172-234-25-215\" not found" node="172-234-25-215" Aug 13 01:25:17.902295 kubelet[2347]: E0813 01:25:17.902277 2347 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:18.425414 kubelet[2347]: I0813 01:25:18.425390 2347 kubelet_node_status.go:75] "Attempting to register node" node="172-234-25-215" Aug 13 01:25:18.905544 kubelet[2347]: E0813 01:25:18.905400 2347 kubelet.go:3305] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"172-234-25-215\" not found" node="172-234-25-215" Aug 13 01:25:18.906156 kubelet[2347]: E0813 01:25:18.905519 2347 kubelet.go:3305] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"172-234-25-215\" not found" node="172-234-25-215" Aug 13 01:25:18.906156 kubelet[2347]: E0813 01:25:18.905895 2347 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:18.906344 kubelet[2347]: E0813 01:25:18.906286 2347 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:19.333879 kubelet[2347]: E0813 01:25:19.333843 2347 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"172-234-25-215\" not found" node="172-234-25-215" Aug 13 01:25:19.379596 kubelet[2347]: I0813 01:25:19.379489 2347 kubelet_node_status.go:78] "Successfully registered node" node="172-234-25-215" Aug 13 01:25:19.449477 kubelet[2347]: I0813 01:25:19.449429 2347 kubelet.go:3309] "Creating a mirror pod for static pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:19.454226 kubelet[2347]: E0813 01:25:19.454175 2347 kubelet.go:3311] "Failed creating a mirror pod" err="pods \"kube-controller-manager-172-234-25-215\" is forbidden: no PriorityClass with name system-node-critical was found" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:19.454226 kubelet[2347]: I0813 01:25:19.454210 2347 kubelet.go:3309] "Creating a mirror pod for static pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:25:19.455457 kubelet[2347]: E0813 01:25:19.455428 2347 kubelet.go:3311] "Failed creating a mirror pod" err="pods \"kube-scheduler-172-234-25-215\" is forbidden: no PriorityClass with name system-node-critical was found" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:25:19.455457 kubelet[2347]: I0813 01:25:19.455447 2347 kubelet.go:3309] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:25:19.456803 kubelet[2347]: E0813 01:25:19.456565 2347 kubelet.go:3311] "Failed creating a mirror pod" err="pods \"kube-apiserver-172-234-25-215\" is forbidden: no PriorityClass with name system-node-critical was found" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:25:19.826132 kubelet[2347]: I0813 01:25:19.826103 2347 apiserver.go:52] "Watching apiserver" Aug 13 01:25:19.849528 kubelet[2347]: I0813 01:25:19.849507 2347 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Aug 13 01:25:19.904020 kubelet[2347]: I0813 01:25:19.903991 2347 kubelet.go:3309] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:25:19.905782 kubelet[2347]: E0813 01:25:19.905759 2347 kubelet.go:3311] "Failed creating a mirror pod" err="pods \"kube-apiserver-172-234-25-215\" is forbidden: no PriorityClass with name system-node-critical was found" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:25:19.906086 kubelet[2347]: E0813 01:25:19.905879 2347 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:20.438186 kubelet[2347]: I0813 01:25:20.438148 2347 kubelet.go:3309] "Creating a mirror pod for static pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:25:20.442822 kubelet[2347]: E0813 01:25:20.442672 2347 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:20.906271 kubelet[2347]: E0813 01:25:20.906237 2347 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:21.443275 systemd[1]: Reload requested from client PID 2625 ('systemctl') (unit session-7.scope)... Aug 13 01:25:21.443296 systemd[1]: Reloading... Aug 13 01:25:21.535794 zram_generator::config[2667]: No configuration found. Aug 13 01:25:21.640534 systemd[1]: /usr/lib/systemd/system/docker.socket:6: ListenStream= references a path below legacy directory /var/run/, updating /var/run/docker.sock → /run/docker.sock; please update the unit file accordingly. Aug 13 01:25:21.756353 systemd[1]: Reloading finished in 312 ms. Aug 13 01:25:21.781583 systemd[1]: Stopping kubelet.service - kubelet: The Kubernetes Node Agent... Aug 13 01:25:21.797020 systemd[1]: kubelet.service: Deactivated successfully. Aug 13 01:25:21.797376 systemd[1]: Stopped kubelet.service - kubelet: The Kubernetes Node Agent. Aug 13 01:25:21.797430 systemd[1]: kubelet.service: Consumed 725ms CPU time, 132.6M memory peak. Aug 13 01:25:21.799169 systemd[1]: Starting kubelet.service - kubelet: The Kubernetes Node Agent... Aug 13 01:25:21.966055 systemd[1]: Started kubelet.service - kubelet: The Kubernetes Node Agent. Aug 13 01:25:21.973991 (kubelet)[2720]: kubelet.service: Referenced but unset environment variable evaluates to an empty string: KUBELET_EXTRA_ARGS Aug 13 01:25:22.011866 kubelet[2720]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 13 01:25:22.011866 kubelet[2720]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 13 01:25:22.011866 kubelet[2720]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 13 01:25:22.011866 kubelet[2720]: I0813 01:25:22.010787 2720 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 13 01:25:22.017931 kubelet[2720]: I0813 01:25:22.017904 2720 server.go:530] "Kubelet version" kubeletVersion="v1.33.0" Aug 13 01:25:22.017931 kubelet[2720]: I0813 01:25:22.017923 2720 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 13 01:25:22.018077 kubelet[2720]: I0813 01:25:22.018057 2720 server.go:956] "Client rotation is on, will bootstrap in background" Aug 13 01:25:22.018932 kubelet[2720]: I0813 01:25:22.018912 2720 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Aug 13 01:25:22.020736 kubelet[2720]: I0813 01:25:22.020543 2720 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/pki/ca.crt" Aug 13 01:25:22.024383 kubelet[2720]: I0813 01:25:22.024366 2720 server.go:1446] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 13 01:25:22.031353 kubelet[2720]: I0813 01:25:22.030941 2720 server.go:782] "--cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /" Aug 13 01:25:22.031353 kubelet[2720]: I0813 01:25:22.031154 2720 container_manager_linux.go:267] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 13 01:25:22.031353 kubelet[2720]: I0813 01:25:22.031170 2720 container_manager_linux.go:272] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"172-234-25-215","RuntimeCgroupsName":"","SystemCgroupsName":"","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":false,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":null,"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":-1,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 13 01:25:22.031353 kubelet[2720]: I0813 01:25:22.031279 2720 topology_manager.go:138] "Creating topology manager with none policy" Aug 13 01:25:22.031747 kubelet[2720]: I0813 01:25:22.031287 2720 container_manager_linux.go:303] "Creating device plugin manager" Aug 13 01:25:22.031804 kubelet[2720]: I0813 01:25:22.031321 2720 state_mem.go:36] "Initialized new in-memory state store" Aug 13 01:25:22.031979 kubelet[2720]: I0813 01:25:22.031968 2720 kubelet.go:480] "Attempting to sync node with API server" Aug 13 01:25:22.032042 kubelet[2720]: I0813 01:25:22.032033 2720 kubelet.go:375] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 13 01:25:22.032105 kubelet[2720]: I0813 01:25:22.032097 2720 kubelet.go:386] "Adding apiserver pod source" Aug 13 01:25:22.032165 kubelet[2720]: I0813 01:25:22.032155 2720 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 13 01:25:22.033731 kubelet[2720]: I0813 01:25:22.033682 2720 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="containerd" version="v2.0.4" apiVersion="v1" Aug 13 01:25:22.034302 kubelet[2720]: I0813 01:25:22.034274 2720 kubelet.go:935] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 13 01:25:22.037840 kubelet[2720]: I0813 01:25:22.037825 2720 watchdog_linux.go:99] "Systemd watchdog is not enabled" Aug 13 01:25:22.037970 kubelet[2720]: I0813 01:25:22.037922 2720 server.go:1289] "Started kubelet" Aug 13 01:25:22.040952 kubelet[2720]: I0813 01:25:22.040938 2720 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 13 01:25:22.051666 kubelet[2720]: I0813 01:25:22.051598 2720 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Aug 13 01:25:22.053826 kubelet[2720]: E0813 01:25:22.053809 2720 kubelet.go:1600] "Image garbage collection failed once. Stats initialization may not have completed yet" err="invalid capacity 0 on image filesystem" Aug 13 01:25:22.054968 kubelet[2720]: I0813 01:25:22.054949 2720 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 13 01:25:22.055669 kubelet[2720]: I0813 01:25:22.055655 2720 server.go:317] "Adding debug handlers to kubelet server" Aug 13 01:25:22.055907 kubelet[2720]: I0813 01:25:22.055896 2720 volume_manager.go:297] "Starting Kubelet Volume Manager" Aug 13 01:25:22.059440 kubelet[2720]: I0813 01:25:22.055936 2720 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Aug 13 01:25:22.059440 kubelet[2720]: I0813 01:25:22.059162 2720 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 13 01:25:22.059440 kubelet[2720]: I0813 01:25:22.059253 2720 reconciler.go:26] "Reconciler: start to sync state" Aug 13 01:25:22.059750 kubelet[2720]: I0813 01:25:22.059736 2720 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 13 01:25:22.060910 kubelet[2720]: I0813 01:25:22.060895 2720 dynamic_serving_content.go:135] "Starting controller" name="kubelet-server-cert-files::/var/lib/kubelet/pki/kubelet.crt::/var/lib/kubelet/pki/kubelet.key" Aug 13 01:25:22.063573 kubelet[2720]: I0813 01:25:22.063557 2720 factory.go:223] Registration of the systemd container factory successfully Aug 13 01:25:22.063896 kubelet[2720]: I0813 01:25:22.063739 2720 factory.go:221] Registration of the crio container factory failed: Get "http://%2Fvar%2Frun%2Fcrio%2Fcrio.sock/info": dial unix /var/run/crio/crio.sock: connect: no such file or directory Aug 13 01:25:22.066248 kubelet[2720]: I0813 01:25:22.066159 2720 factory.go:223] Registration of the containerd container factory successfully Aug 13 01:25:22.072444 kubelet[2720]: I0813 01:25:22.072380 2720 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Aug 13 01:25:22.072444 kubelet[2720]: I0813 01:25:22.072397 2720 status_manager.go:230] "Starting to sync pod status with apiserver" Aug 13 01:25:22.072444 kubelet[2720]: I0813 01:25:22.072411 2720 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 13 01:25:22.072444 kubelet[2720]: I0813 01:25:22.072418 2720 kubelet.go:2436] "Starting kubelet main sync loop" Aug 13 01:25:22.072555 kubelet[2720]: E0813 01:25:22.072460 2720 kubelet.go:2460] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Aug 13 01:25:22.106438 kubelet[2720]: I0813 01:25:22.106413 2720 cpu_manager.go:221] "Starting CPU manager" policy="none" Aug 13 01:25:22.106438 kubelet[2720]: I0813 01:25:22.106426 2720 cpu_manager.go:222] "Reconciling" reconcilePeriod="10s" Aug 13 01:25:22.106438 kubelet[2720]: I0813 01:25:22.106442 2720 state_mem.go:36] "Initialized new in-memory state store" Aug 13 01:25:22.106565 kubelet[2720]: I0813 01:25:22.106536 2720 state_mem.go:88] "Updated default CPUSet" cpuSet="" Aug 13 01:25:22.106565 kubelet[2720]: I0813 01:25:22.106545 2720 state_mem.go:96] "Updated CPUSet assignments" assignments={} Aug 13 01:25:22.106565 kubelet[2720]: I0813 01:25:22.106558 2720 policy_none.go:49] "None policy: Start" Aug 13 01:25:22.106565 kubelet[2720]: I0813 01:25:22.106566 2720 memory_manager.go:186] "Starting memorymanager" policy="None" Aug 13 01:25:22.106647 kubelet[2720]: I0813 01:25:22.106574 2720 state_mem.go:35] "Initializing new in-memory state store" Aug 13 01:25:22.106647 kubelet[2720]: I0813 01:25:22.106642 2720 state_mem.go:75] "Updated machine memory state" Aug 13 01:25:22.111608 kubelet[2720]: E0813 01:25:22.111058 2720 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 13 01:25:22.111608 kubelet[2720]: I0813 01:25:22.111218 2720 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 13 01:25:22.111608 kubelet[2720]: I0813 01:25:22.111226 2720 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 13 01:25:22.111608 kubelet[2720]: I0813 01:25:22.111423 2720 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 13 01:25:22.112476 kubelet[2720]: E0813 01:25:22.112458 2720 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="no imagefs label for configured runtime" Aug 13 01:25:22.173913 kubelet[2720]: I0813 01:25:22.173638 2720 kubelet.go:3309] "Creating a mirror pod for static pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:25:22.173913 kubelet[2720]: I0813 01:25:22.173695 2720 kubelet.go:3309] "Creating a mirror pod for static pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:22.173913 kubelet[2720]: I0813 01:25:22.173843 2720 kubelet.go:3309] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:25:22.178961 kubelet[2720]: E0813 01:25:22.178522 2720 kubelet.go:3311] "Failed creating a mirror pod" err="pods \"kube-scheduler-172-234-25-215\" already exists" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:25:22.218143 kubelet[2720]: I0813 01:25:22.218123 2720 kubelet_node_status.go:75] "Attempting to register node" node="172-234-25-215" Aug 13 01:25:22.223198 kubelet[2720]: I0813 01:25:22.223166 2720 kubelet_node_status.go:124] "Node was previously registered" node="172-234-25-215" Aug 13 01:25:22.223376 kubelet[2720]: I0813 01:25:22.223315 2720 kubelet_node_status.go:78] "Successfully registered node" node="172-234-25-215" Aug 13 01:25:22.260889 kubelet[2720]: I0813 01:25:22.260869 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/03fee088624a1f74688c8facb00ef7f7-kubeconfig\") pod \"kube-scheduler-172-234-25-215\" (UID: \"03fee088624a1f74688c8facb00ef7f7\") " pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:25:22.262214 kubelet[2720]: I0813 01:25:22.262110 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-certs\" (UniqueName: \"kubernetes.io/host-path/5c228d00d203ca7627372d21a4f939a9-ca-certs\") pod \"kube-apiserver-172-234-25-215\" (UID: \"5c228d00d203ca7627372d21a4f939a9\") " pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:25:22.262370 kubelet[2720]: I0813 01:25:22.262327 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"k8s-certs\" (UniqueName: \"kubernetes.io/host-path/5c228d00d203ca7627372d21a4f939a9-k8s-certs\") pod \"kube-apiserver-172-234-25-215\" (UID: \"5c228d00d203ca7627372d21a4f939a9\") " pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:25:22.262439 kubelet[2720]: I0813 01:25:22.262351 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"flexvolume-dir\" (UniqueName: \"kubernetes.io/host-path/96c354cf60eb29e83ce001f635cc0212-flexvolume-dir\") pod \"kube-controller-manager-172-234-25-215\" (UID: \"96c354cf60eb29e83ce001f635cc0212\") " pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:22.262622 kubelet[2720]: I0813 01:25:22.262607 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"k8s-certs\" (UniqueName: \"kubernetes.io/host-path/96c354cf60eb29e83ce001f635cc0212-k8s-certs\") pod \"kube-controller-manager-172-234-25-215\" (UID: \"96c354cf60eb29e83ce001f635cc0212\") " pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:22.262789 kubelet[2720]: I0813 01:25:22.262665 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubeconfig\" (UniqueName: \"kubernetes.io/host-path/96c354cf60eb29e83ce001f635cc0212-kubeconfig\") pod \"kube-controller-manager-172-234-25-215\" (UID: \"96c354cf60eb29e83ce001f635cc0212\") " pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:22.262789 kubelet[2720]: I0813 01:25:22.262681 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"usr-share-ca-certificates\" (UniqueName: \"kubernetes.io/host-path/96c354cf60eb29e83ce001f635cc0212-usr-share-ca-certificates\") pod \"kube-controller-manager-172-234-25-215\" (UID: \"96c354cf60eb29e83ce001f635cc0212\") " pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:22.262789 kubelet[2720]: I0813 01:25:22.262696 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"usr-share-ca-certificates\" (UniqueName: \"kubernetes.io/host-path/5c228d00d203ca7627372d21a4f939a9-usr-share-ca-certificates\") pod \"kube-apiserver-172-234-25-215\" (UID: \"5c228d00d203ca7627372d21a4f939a9\") " pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:25:22.262789 kubelet[2720]: I0813 01:25:22.262708 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-certs\" (UniqueName: \"kubernetes.io/host-path/96c354cf60eb29e83ce001f635cc0212-ca-certs\") pod \"kube-controller-manager-172-234-25-215\" (UID: \"96c354cf60eb29e83ce001f635cc0212\") " pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:22.480046 kubelet[2720]: E0813 01:25:22.480013 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:22.481646 kubelet[2720]: E0813 01:25:22.480677 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:22.481646 kubelet[2720]: E0813 01:25:22.480287 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:23.033614 kubelet[2720]: I0813 01:25:23.033351 2720 apiserver.go:52] "Watching apiserver" Aug 13 01:25:23.060084 kubelet[2720]: I0813 01:25:23.060058 2720 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Aug 13 01:25:23.093784 kubelet[2720]: E0813 01:25:23.093749 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:23.095896 kubelet[2720]: I0813 01:25:23.095870 2720 kubelet.go:3309] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:25:23.096201 kubelet[2720]: I0813 01:25:23.096060 2720 kubelet.go:3309] "Creating a mirror pod for static pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:23.102807 kubelet[2720]: E0813 01:25:23.102776 2720 kubelet.go:3311] "Failed creating a mirror pod" err="pods \"kube-apiserver-172-234-25-215\" already exists" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:25:23.103171 kubelet[2720]: E0813 01:25:23.102977 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:23.105395 kubelet[2720]: E0813 01:25:23.105283 2720 kubelet.go:3311] "Failed creating a mirror pod" err="pods \"kube-controller-manager-172-234-25-215\" already exists" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:23.105469 kubelet[2720]: E0813 01:25:23.105451 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:23.131743 kubelet[2720]: I0813 01:25:23.131676 2720 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-scheduler-172-234-25-215" podStartSLOduration=3.131662794 podStartE2EDuration="3.131662794s" podCreationTimestamp="2025-08-13 01:25:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-08-13 01:25:23.12281157 +0000 UTC m=+1.143360953" watchObservedRunningTime="2025-08-13 01:25:23.131662794 +0000 UTC m=+1.152212187" Aug 13 01:25:23.139514 kubelet[2720]: I0813 01:25:23.139473 2720 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-172-234-25-215" podStartSLOduration=1.139465288 podStartE2EDuration="1.139465288s" podCreationTimestamp="2025-08-13 01:25:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-08-13 01:25:23.139009798 +0000 UTC m=+1.159559181" watchObservedRunningTime="2025-08-13 01:25:23.139465288 +0000 UTC m=+1.160014681" Aug 13 01:25:23.139596 kubelet[2720]: I0813 01:25:23.139538 2720 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-controller-manager-172-234-25-215" podStartSLOduration=1.139535178 podStartE2EDuration="1.139535178s" podCreationTimestamp="2025-08-13 01:25:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-08-13 01:25:23.132066194 +0000 UTC m=+1.152615577" watchObservedRunningTime="2025-08-13 01:25:23.139535178 +0000 UTC m=+1.160084561" Aug 13 01:25:23.924383 systemd[1]: systemd-hostnamed.service: Deactivated successfully. Aug 13 01:25:24.094870 kubelet[2720]: E0813 01:25:24.094830 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:24.095517 kubelet[2720]: E0813 01:25:24.095491 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:24.095979 kubelet[2720]: E0813 01:25:24.095958 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:25.096177 kubelet[2720]: E0813 01:25:25.096150 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:27.050640 kubelet[2720]: I0813 01:25:27.050603 2720 kuberuntime_manager.go:1746] "Updating runtime config through cri with podcidr" CIDR="192.168.0.0/24" Aug 13 01:25:27.051065 containerd[1567]: time="2025-08-13T01:25:27.050905969Z" level=info msg="No cni config template is specified, wait for other system components to drop the config." Aug 13 01:25:27.051323 kubelet[2720]: I0813 01:25:27.051262 2720 kubelet_network.go:61] "Updating Pod CIDR" originalPodCIDR="" newPodCIDR="192.168.0.0/24" Aug 13 01:25:28.089952 systemd[1]: Created slice kubepods-besteffort-podc2cae147_08c3_44c9_8d6e_2eb951559e9d.slice - libcontainer container kubepods-besteffort-podc2cae147_08c3_44c9_8d6e_2eb951559e9d.slice. Aug 13 01:25:28.096699 kubelet[2720]: I0813 01:25:28.096678 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"xtables-lock\" (UniqueName: \"kubernetes.io/host-path/c2cae147-08c3-44c9-8d6e-2eb951559e9d-xtables-lock\") pod \"kube-proxy-czb9t\" (UID: \"c2cae147-08c3-44c9-8d6e-2eb951559e9d\") " pod="kube-system/kube-proxy-czb9t" Aug 13 01:25:28.097091 kubelet[2720]: I0813 01:25:28.096705 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-proxy\" (UniqueName: \"kubernetes.io/configmap/c2cae147-08c3-44c9-8d6e-2eb951559e9d-kube-proxy\") pod \"kube-proxy-czb9t\" (UID: \"c2cae147-08c3-44c9-8d6e-2eb951559e9d\") " pod="kube-system/kube-proxy-czb9t" Aug 13 01:25:28.097091 kubelet[2720]: I0813 01:25:28.096741 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2cae147-08c3-44c9-8d6e-2eb951559e9d-lib-modules\") pod \"kube-proxy-czb9t\" (UID: \"c2cae147-08c3-44c9-8d6e-2eb951559e9d\") " pod="kube-system/kube-proxy-czb9t" Aug 13 01:25:28.097091 kubelet[2720]: I0813 01:25:28.096756 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49v6h\" (UniqueName: \"kubernetes.io/projected/c2cae147-08c3-44c9-8d6e-2eb951559e9d-kube-api-access-49v6h\") pod \"kube-proxy-czb9t\" (UID: \"c2cae147-08c3-44c9-8d6e-2eb951559e9d\") " pod="kube-system/kube-proxy-czb9t" Aug 13 01:25:28.254608 systemd[1]: Created slice kubepods-besteffort-pod4090dbb9_5cbe_4af6_993c_ea44b6e69aaa.slice - libcontainer container kubepods-besteffort-pod4090dbb9_5cbe_4af6_993c_ea44b6e69aaa.slice. Aug 13 01:25:28.297972 kubelet[2720]: I0813 01:25:28.297945 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xxjx\" (UniqueName: \"kubernetes.io/projected/4090dbb9-5cbe-4af6-993c-ea44b6e69aaa-kube-api-access-8xxjx\") pod \"tigera-operator-747864d56d-vb4bt\" (UID: \"4090dbb9-5cbe-4af6-993c-ea44b6e69aaa\") " pod="tigera-operator/tigera-operator-747864d56d-vb4bt" Aug 13 01:25:28.298129 kubelet[2720]: I0813 01:25:28.297975 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-calico\" (UniqueName: \"kubernetes.io/host-path/4090dbb9-5cbe-4af6-993c-ea44b6e69aaa-var-lib-calico\") pod \"tigera-operator-747864d56d-vb4bt\" (UID: \"4090dbb9-5cbe-4af6-993c-ea44b6e69aaa\") " pod="tigera-operator/tigera-operator-747864d56d-vb4bt" Aug 13 01:25:28.397280 kubelet[2720]: E0813 01:25:28.396930 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:28.397478 containerd[1567]: time="2025-08-13T01:25:28.397424335Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:kube-proxy-czb9t,Uid:c2cae147-08c3-44c9-8d6e-2eb951559e9d,Namespace:kube-system,Attempt:0,}" Aug 13 01:25:28.419446 containerd[1567]: time="2025-08-13T01:25:28.419184014Z" level=info msg="connecting to shim 23e526e680e61ae7e941cc509509cd39acc7d69ce570bad007f6f667be120c7c" address="unix:///run/containerd/s/2c4c6350c03ddd02a786e989710cb4f1baf2e4237738af399e3f2acce6983c4b" namespace=k8s.io protocol=ttrpc version=3 Aug 13 01:25:28.447839 systemd[1]: Started cri-containerd-23e526e680e61ae7e941cc509509cd39acc7d69ce570bad007f6f667be120c7c.scope - libcontainer container 23e526e680e61ae7e941cc509509cd39acc7d69ce570bad007f6f667be120c7c. Aug 13 01:25:28.473242 containerd[1567]: time="2025-08-13T01:25:28.473214361Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:kube-proxy-czb9t,Uid:c2cae147-08c3-44c9-8d6e-2eb951559e9d,Namespace:kube-system,Attempt:0,} returns sandbox id \"23e526e680e61ae7e941cc509509cd39acc7d69ce570bad007f6f667be120c7c\"" Aug 13 01:25:28.474075 kubelet[2720]: E0813 01:25:28.474056 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:28.479400 containerd[1567]: time="2025-08-13T01:25:28.479353825Z" level=info msg="CreateContainer within sandbox \"23e526e680e61ae7e941cc509509cd39acc7d69ce570bad007f6f667be120c7c\" for container &ContainerMetadata{Name:kube-proxy,Attempt:0,}" Aug 13 01:25:28.487819 kubelet[2720]: E0813 01:25:28.487797 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:28.491455 containerd[1567]: time="2025-08-13T01:25:28.490708374Z" level=info msg="Container 6369622dc3b5e4883e8463ceb04a4d4c703a3350ba099bf1c41d2444aaba6313: CDI devices from CRI Config.CDIDevices: []" Aug 13 01:25:28.502960 containerd[1567]: time="2025-08-13T01:25:28.502936012Z" level=info msg="CreateContainer within sandbox \"23e526e680e61ae7e941cc509509cd39acc7d69ce570bad007f6f667be120c7c\" for &ContainerMetadata{Name:kube-proxy,Attempt:0,} returns container id \"6369622dc3b5e4883e8463ceb04a4d4c703a3350ba099bf1c41d2444aaba6313\"" Aug 13 01:25:28.505640 containerd[1567]: time="2025-08-13T01:25:28.505523419Z" level=info msg="StartContainer for \"6369622dc3b5e4883e8463ceb04a4d4c703a3350ba099bf1c41d2444aaba6313\"" Aug 13 01:25:28.510202 containerd[1567]: time="2025-08-13T01:25:28.509993095Z" level=info msg="connecting to shim 6369622dc3b5e4883e8463ceb04a4d4c703a3350ba099bf1c41d2444aaba6313" address="unix:///run/containerd/s/2c4c6350c03ddd02a786e989710cb4f1baf2e4237738af399e3f2acce6983c4b" protocol=ttrpc version=3 Aug 13 01:25:28.535846 systemd[1]: Started cri-containerd-6369622dc3b5e4883e8463ceb04a4d4c703a3350ba099bf1c41d2444aaba6313.scope - libcontainer container 6369622dc3b5e4883e8463ceb04a4d4c703a3350ba099bf1c41d2444aaba6313. Aug 13 01:25:28.557520 containerd[1567]: time="2025-08-13T01:25:28.557477158Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:tigera-operator-747864d56d-vb4bt,Uid:4090dbb9-5cbe-4af6-993c-ea44b6e69aaa,Namespace:tigera-operator,Attempt:0,}" Aug 13 01:25:28.581323 containerd[1567]: time="2025-08-13T01:25:28.581223065Z" level=info msg="StartContainer for \"6369622dc3b5e4883e8463ceb04a4d4c703a3350ba099bf1c41d2444aaba6313\" returns successfully" Aug 13 01:25:28.584749 containerd[1567]: time="2025-08-13T01:25:28.584654052Z" level=info msg="connecting to shim 19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec" address="unix:///run/containerd/s/a94b5c387e26d251dcc589f61d330576e2b79cf5c5bd9024576ab8ed9e16a24d" namespace=k8s.io protocol=ttrpc version=3 Aug 13 01:25:28.613834 systemd[1]: Started cri-containerd-19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec.scope - libcontainer container 19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec. Aug 13 01:25:28.667198 containerd[1567]: time="2025-08-13T01:25:28.666316041Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:tigera-operator-747864d56d-vb4bt,Uid:4090dbb9-5cbe-4af6-993c-ea44b6e69aaa,Namespace:tigera-operator,Attempt:0,} returns sandbox id \"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\"" Aug 13 01:25:28.669932 containerd[1567]: time="2025-08-13T01:25:28.669574878Z" level=info msg="PullImage \"quay.io/tigera/operator:v1.38.3\"" Aug 13 01:25:29.104367 kubelet[2720]: E0813 01:25:29.103911 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:29.104367 kubelet[2720]: E0813 01:25:29.104194 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:29.129983 kubelet[2720]: I0813 01:25:29.129929 2720 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-proxy-czb9t" podStartSLOduration=1.12991556 podStartE2EDuration="1.12991556s" podCreationTimestamp="2025-08-13 01:25:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2025-08-13 01:25:29.118823249 +0000 UTC m=+7.139372632" watchObservedRunningTime="2025-08-13 01:25:29.12991556 +0000 UTC m=+7.150464943" Aug 13 01:25:29.207420 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount1050817851.mount: Deactivated successfully. Aug 13 01:25:29.254051 kubelet[2720]: E0813 01:25:29.254017 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:30.000321 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount2492826154.mount: Deactivated successfully. Aug 13 01:25:30.109498 kubelet[2720]: E0813 01:25:30.109473 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:30.110600 kubelet[2720]: E0813 01:25:30.110223 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:30.412349 containerd[1567]: time="2025-08-13T01:25:30.412077017Z" level=info msg="ImageCreate event name:\"quay.io/tigera/operator:v1.38.3\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:30.413557 containerd[1567]: time="2025-08-13T01:25:30.413527886Z" level=info msg="stop pulling image quay.io/tigera/operator:v1.38.3: active requests=0, bytes read=25056543" Aug 13 01:25:30.414735 containerd[1567]: time="2025-08-13T01:25:30.414428345Z" level=info msg="ImageCreate event name:\"sha256:8bde16470b09d1963e19456806d73180c9778a6c2b3c1fda2335c67c1cd4ce93\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:30.417548 containerd[1567]: time="2025-08-13T01:25:30.417527342Z" level=info msg="ImageCreate event name:\"quay.io/tigera/operator@sha256:dbf1bad0def7b5955dc8e4aeee96e23ead0bc5822f6872518e685cd0ed484121\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:30.418291 containerd[1567]: time="2025-08-13T01:25:30.418269972Z" level=info msg="Pulled image \"quay.io/tigera/operator:v1.38.3\" with image id \"sha256:8bde16470b09d1963e19456806d73180c9778a6c2b3c1fda2335c67c1cd4ce93\", repo tag \"quay.io/tigera/operator:v1.38.3\", repo digest \"quay.io/tigera/operator@sha256:dbf1bad0def7b5955dc8e4aeee96e23ead0bc5822f6872518e685cd0ed484121\", size \"25052538\" in 1.747809635s" Aug 13 01:25:30.418367 containerd[1567]: time="2025-08-13T01:25:30.418353212Z" level=info msg="PullImage \"quay.io/tigera/operator:v1.38.3\" returns image reference \"sha256:8bde16470b09d1963e19456806d73180c9778a6c2b3c1fda2335c67c1cd4ce93\"" Aug 13 01:25:30.421209 containerd[1567]: time="2025-08-13T01:25:30.421185940Z" level=info msg="CreateContainer within sandbox \"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" for container &ContainerMetadata{Name:tigera-operator,Attempt:0,}" Aug 13 01:25:30.428739 containerd[1567]: time="2025-08-13T01:25:30.428239424Z" level=info msg="Container 86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382: CDI devices from CRI Config.CDIDevices: []" Aug 13 01:25:30.431484 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount3333504359.mount: Deactivated successfully. Aug 13 01:25:30.435600 containerd[1567]: time="2025-08-13T01:25:30.435577078Z" level=info msg="CreateContainer within sandbox \"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" for &ContainerMetadata{Name:tigera-operator,Attempt:0,} returns container id \"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\"" Aug 13 01:25:30.436074 containerd[1567]: time="2025-08-13T01:25:30.436017538Z" level=info msg="StartContainer for \"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\"" Aug 13 01:25:30.437118 containerd[1567]: time="2025-08-13T01:25:30.437095967Z" level=info msg="connecting to shim 86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382" address="unix:///run/containerd/s/a94b5c387e26d251dcc589f61d330576e2b79cf5c5bd9024576ab8ed9e16a24d" protocol=ttrpc version=3 Aug 13 01:25:30.458847 systemd[1]: Started cri-containerd-86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382.scope - libcontainer container 86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382. Aug 13 01:25:30.491377 containerd[1567]: time="2025-08-13T01:25:30.491331023Z" level=info msg="StartContainer for \"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\" returns successfully" Aug 13 01:25:30.644832 kubelet[2720]: E0813 01:25:30.644799 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:31.111159 kubelet[2720]: E0813 01:25:31.111126 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:31.111897 kubelet[2720]: E0813 01:25:31.111615 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:35.869934 sudo[1802]: pam_unix(sudo:session): session closed for user root Aug 13 01:25:35.920105 sshd[1801]: Connection closed by 147.75.109.163 port 51660 Aug 13 01:25:35.921896 sshd-session[1799]: pam_unix(sshd:session): session closed for user core Aug 13 01:25:35.928572 systemd-logind[1543]: Session 7 logged out. Waiting for processes to exit. Aug 13 01:25:35.929452 systemd[1]: sshd@6-172.234.25.215:22-147.75.109.163:51660.service: Deactivated successfully. Aug 13 01:25:35.935353 systemd[1]: session-7.scope: Deactivated successfully. Aug 13 01:25:35.936087 systemd[1]: session-7.scope: Consumed 3.868s CPU time, 227.4M memory peak. Aug 13 01:25:35.941963 systemd-logind[1543]: Removed session 7. Aug 13 01:25:38.523450 update_engine[1544]: I20250813 01:25:38.522851 1544 update_attempter.cc:509] Updating boot flags... Aug 13 01:25:39.042274 kubelet[2720]: I0813 01:25:39.041944 2720 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="tigera-operator/tigera-operator-747864d56d-vb4bt" podStartSLOduration=9.291460823 podStartE2EDuration="11.041931175s" podCreationTimestamp="2025-08-13 01:25:28 +0000 UTC" firstStartedPulling="2025-08-13 01:25:28.668534439 +0000 UTC m=+6.689083822" lastFinishedPulling="2025-08-13 01:25:30.419004791 +0000 UTC m=+8.439554174" observedRunningTime="2025-08-13 01:25:31.13341629 +0000 UTC m=+9.153965673" watchObservedRunningTime="2025-08-13 01:25:39.041931175 +0000 UTC m=+17.062480558" Aug 13 01:25:39.054185 systemd[1]: Created slice kubepods-besteffort-podd0c18708_6c9b_42ce_bbc4_7fe0748e1776.slice - libcontainer container kubepods-besteffort-podd0c18708_6c9b_42ce_bbc4_7fe0748e1776.slice. Aug 13 01:25:39.066756 kubelet[2720]: I0813 01:25:39.066326 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mglqs\" (UniqueName: \"kubernetes.io/projected/d0c18708-6c9b-42ce-bbc4-7fe0748e1776-kube-api-access-mglqs\") pod \"calico-typha-5bf7f956b7-szw5v\" (UID: \"d0c18708-6c9b-42ce-bbc4-7fe0748e1776\") " pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:25:39.067126 kubelet[2720]: I0813 01:25:39.067110 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"typha-certs\" (UniqueName: \"kubernetes.io/secret/d0c18708-6c9b-42ce-bbc4-7fe0748e1776-typha-certs\") pod \"calico-typha-5bf7f956b7-szw5v\" (UID: \"d0c18708-6c9b-42ce-bbc4-7fe0748e1776\") " pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:25:39.067205 kubelet[2720]: I0813 01:25:39.067192 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tigera-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0c18708-6c9b-42ce-bbc4-7fe0748e1776-tigera-ca-bundle\") pod \"calico-typha-5bf7f956b7-szw5v\" (UID: \"d0c18708-6c9b-42ce-bbc4-7fe0748e1776\") " pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:25:39.151707 systemd[1]: Created slice kubepods-besteffort-pod4f334296_e8c0_427e_a090_d8fbd3582f26.slice - libcontainer container kubepods-besteffort-pod4f334296_e8c0_427e_a090_d8fbd3582f26.slice. Aug 13 01:25:39.168080 kubelet[2720]: I0813 01:25:39.168040 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-log-dir\" (UniqueName: \"kubernetes.io/host-path/4f334296-e8c0-427e-a090-d8fbd3582f26-cni-log-dir\") pod \"calico-node-2p8hj\" (UID: \"4f334296-e8c0-427e-a090-d8fbd3582f26\") " pod="calico-system/calico-node-2p8hj" Aug 13 01:25:39.168255 kubelet[2720]: I0813 01:25:39.168206 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-net-dir\" (UniqueName: \"kubernetes.io/host-path/4f334296-e8c0-427e-a090-d8fbd3582f26-cni-net-dir\") pod \"calico-node-2p8hj\" (UID: \"4f334296-e8c0-427e-a090-d8fbd3582f26\") " pod="calico-system/calico-node-2p8hj" Aug 13 01:25:39.168255 kubelet[2720]: I0813 01:25:39.168226 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"policysync\" (UniqueName: \"kubernetes.io/host-path/4f334296-e8c0-427e-a090-d8fbd3582f26-policysync\") pod \"calico-node-2p8hj\" (UID: \"4f334296-e8c0-427e-a090-d8fbd3582f26\") " pod="calico-system/calico-node-2p8hj" Aug 13 01:25:39.168755 kubelet[2720]: I0813 01:25:39.168338 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"xtables-lock\" (UniqueName: \"kubernetes.io/host-path/4f334296-e8c0-427e-a090-d8fbd3582f26-xtables-lock\") pod \"calico-node-2p8hj\" (UID: \"4f334296-e8c0-427e-a090-d8fbd3582f26\") " pod="calico-system/calico-node-2p8hj" Aug 13 01:25:39.168755 kubelet[2720]: I0813 01:25:39.168371 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-calico\" (UniqueName: \"kubernetes.io/host-path/4f334296-e8c0-427e-a090-d8fbd3582f26-var-lib-calico\") pod \"calico-node-2p8hj\" (UID: \"4f334296-e8c0-427e-a090-d8fbd3582f26\") " pod="calico-system/calico-node-2p8hj" Aug 13 01:25:39.170736 kubelet[2720]: I0813 01:25:39.169182 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"flexvol-driver-host\" (UniqueName: \"kubernetes.io/host-path/4f334296-e8c0-427e-a090-d8fbd3582f26-flexvol-driver-host\") pod \"calico-node-2p8hj\" (UID: \"4f334296-e8c0-427e-a090-d8fbd3582f26\") " pod="calico-system/calico-node-2p8hj" Aug 13 01:25:39.170736 kubelet[2720]: I0813 01:25:39.169212 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-certs\" (UniqueName: \"kubernetes.io/secret/4f334296-e8c0-427e-a090-d8fbd3582f26-node-certs\") pod \"calico-node-2p8hj\" (UID: \"4f334296-e8c0-427e-a090-d8fbd3582f26\") " pod="calico-system/calico-node-2p8hj" Aug 13 01:25:39.170736 kubelet[2720]: I0813 01:25:39.169243 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-run-calico\" (UniqueName: \"kubernetes.io/host-path/4f334296-e8c0-427e-a090-d8fbd3582f26-var-run-calico\") pod \"calico-node-2p8hj\" (UID: \"4f334296-e8c0-427e-a090-d8fbd3582f26\") " pod="calico-system/calico-node-2p8hj" Aug 13 01:25:39.170736 kubelet[2720]: I0813 01:25:39.169263 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-bin-dir\" (UniqueName: \"kubernetes.io/host-path/4f334296-e8c0-427e-a090-d8fbd3582f26-cni-bin-dir\") pod \"calico-node-2p8hj\" (UID: \"4f334296-e8c0-427e-a090-d8fbd3582f26\") " pod="calico-system/calico-node-2p8hj" Aug 13 01:25:39.170736 kubelet[2720]: I0813 01:25:39.169286 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tigera-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f334296-e8c0-427e-a090-d8fbd3582f26-tigera-ca-bundle\") pod \"calico-node-2p8hj\" (UID: \"4f334296-e8c0-427e-a090-d8fbd3582f26\") " pod="calico-system/calico-node-2p8hj" Aug 13 01:25:39.170914 kubelet[2720]: I0813 01:25:39.169303 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl8ww\" (UniqueName: \"kubernetes.io/projected/4f334296-e8c0-427e-a090-d8fbd3582f26-kube-api-access-jl8ww\") pod \"calico-node-2p8hj\" (UID: \"4f334296-e8c0-427e-a090-d8fbd3582f26\") " pod="calico-system/calico-node-2p8hj" Aug 13 01:25:39.170914 kubelet[2720]: I0813 01:25:39.169336 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f334296-e8c0-427e-a090-d8fbd3582f26-lib-modules\") pod \"calico-node-2p8hj\" (UID: \"4f334296-e8c0-427e-a090-d8fbd3582f26\") " pod="calico-system/calico-node-2p8hj" Aug 13 01:25:39.271739 kubelet[2720]: E0813 01:25:39.271508 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.271739 kubelet[2720]: W0813 01:25:39.271560 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.271739 kubelet[2720]: E0813 01:25:39.271587 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.273306 kubelet[2720]: E0813 01:25:39.273289 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.273410 kubelet[2720]: W0813 01:25:39.273396 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.273700 kubelet[2720]: E0813 01:25:39.273687 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.276075 kubelet[2720]: E0813 01:25:39.276044 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.276075 kubelet[2720]: W0813 01:25:39.276064 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.276150 kubelet[2720]: E0813 01:25:39.276097 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.277825 kubelet[2720]: E0813 01:25:39.276625 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.277825 kubelet[2720]: W0813 01:25:39.276637 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.277825 kubelet[2720]: E0813 01:25:39.276646 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.277825 kubelet[2720]: E0813 01:25:39.276987 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.277825 kubelet[2720]: W0813 01:25:39.276995 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.277825 kubelet[2720]: E0813 01:25:39.277002 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.277825 kubelet[2720]: E0813 01:25:39.277374 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.277825 kubelet[2720]: W0813 01:25:39.277382 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.277825 kubelet[2720]: E0813 01:25:39.277391 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.277825 kubelet[2720]: E0813 01:25:39.277757 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.278078 kubelet[2720]: W0813 01:25:39.277765 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.278078 kubelet[2720]: E0813 01:25:39.277773 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.278078 kubelet[2720]: E0813 01:25:39.278061 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.278078 kubelet[2720]: W0813 01:25:39.278069 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.278078 kubelet[2720]: E0813 01:25:39.278076 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.278452 kubelet[2720]: E0813 01:25:39.278327 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.278452 kubelet[2720]: W0813 01:25:39.278339 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.278452 kubelet[2720]: E0813 01:25:39.278348 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.279798 kubelet[2720]: E0813 01:25:39.279784 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.279945 kubelet[2720]: W0813 01:25:39.279851 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.279945 kubelet[2720]: E0813 01:25:39.279867 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.280192 kubelet[2720]: E0813 01:25:39.280180 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.280941 kubelet[2720]: W0813 01:25:39.280914 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.281039 kubelet[2720]: E0813 01:25:39.281026 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.283404 kubelet[2720]: E0813 01:25:39.283369 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.283404 kubelet[2720]: W0813 01:25:39.283381 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.283404 kubelet[2720]: E0813 01:25:39.283391 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.289754 kubelet[2720]: E0813 01:25:39.289734 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.289754 kubelet[2720]: W0813 01:25:39.289750 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.289861 kubelet[2720]: E0813 01:25:39.289762 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.290188 kubelet[2720]: E0813 01:25:39.290164 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.290188 kubelet[2720]: W0813 01:25:39.290180 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.290734 kubelet[2720]: E0813 01:25:39.290316 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.290964 kubelet[2720]: E0813 01:25:39.290943 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.290964 kubelet[2720]: W0813 01:25:39.290958 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.291300 kubelet[2720]: E0813 01:25:39.291276 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.291987 kubelet[2720]: E0813 01:25:39.291949 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.292192 kubelet[2720]: W0813 01:25:39.292160 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.292192 kubelet[2720]: E0813 01:25:39.292177 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.293052 kubelet[2720]: E0813 01:25:39.292922 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.293052 kubelet[2720]: W0813 01:25:39.292938 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.293613 kubelet[2720]: E0813 01:25:39.293167 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.293613 kubelet[2720]: E0813 01:25:39.293460 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.293613 kubelet[2720]: W0813 01:25:39.293468 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.293613 kubelet[2720]: E0813 01:25:39.293487 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.294631 kubelet[2720]: E0813 01:25:39.294613 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.294631 kubelet[2720]: W0813 01:25:39.294627 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.294799 kubelet[2720]: E0813 01:25:39.294638 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.295002 kubelet[2720]: E0813 01:25:39.294931 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.295002 kubelet[2720]: W0813 01:25:39.294942 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.295002 kubelet[2720]: E0813 01:25:39.294950 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.319806 kubelet[2720]: E0813 01:25:39.319768 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:25:39.362284 kubelet[2720]: E0813 01:25:39.362040 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:39.362694 containerd[1567]: time="2025-08-13T01:25:39.362613352Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-typha-5bf7f956b7-szw5v,Uid:d0c18708-6c9b-42ce-bbc4-7fe0748e1776,Namespace:calico-system,Attempt:0,}" Aug 13 01:25:39.364123 kubelet[2720]: E0813 01:25:39.364045 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.364123 kubelet[2720]: W0813 01:25:39.364084 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.364123 kubelet[2720]: E0813 01:25:39.364099 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.364649 kubelet[2720]: E0813 01:25:39.364282 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.364649 kubelet[2720]: W0813 01:25:39.364290 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.364649 kubelet[2720]: E0813 01:25:39.364319 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.364649 kubelet[2720]: E0813 01:25:39.364505 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.364649 kubelet[2720]: W0813 01:25:39.364513 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.364649 kubelet[2720]: E0813 01:25:39.364520 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.365476 kubelet[2720]: E0813 01:25:39.365031 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.365476 kubelet[2720]: W0813 01:25:39.365040 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.365476 kubelet[2720]: E0813 01:25:39.365048 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.365476 kubelet[2720]: E0813 01:25:39.365327 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.365476 kubelet[2720]: W0813 01:25:39.365334 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.365476 kubelet[2720]: E0813 01:25:39.365342 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.365849 kubelet[2720]: E0813 01:25:39.365642 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.365849 kubelet[2720]: W0813 01:25:39.365650 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.365849 kubelet[2720]: E0813 01:25:39.365658 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.366892 kubelet[2720]: E0813 01:25:39.366838 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.366892 kubelet[2720]: W0813 01:25:39.366853 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.367104 kubelet[2720]: E0813 01:25:39.366862 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.367216 kubelet[2720]: E0813 01:25:39.367133 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.367216 kubelet[2720]: W0813 01:25:39.367147 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.367216 kubelet[2720]: E0813 01:25:39.367155 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.367381 kubelet[2720]: E0813 01:25:39.367341 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.367381 kubelet[2720]: W0813 01:25:39.367356 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.367381 kubelet[2720]: E0813 01:25:39.367363 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.367636 kubelet[2720]: E0813 01:25:39.367521 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.367636 kubelet[2720]: W0813 01:25:39.367535 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.367636 kubelet[2720]: E0813 01:25:39.367562 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.367918 kubelet[2720]: E0813 01:25:39.367751 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.367918 kubelet[2720]: W0813 01:25:39.367758 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.367918 kubelet[2720]: E0813 01:25:39.367765 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.367989 kubelet[2720]: E0813 01:25:39.367948 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.367989 kubelet[2720]: W0813 01:25:39.367956 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.367989 kubelet[2720]: E0813 01:25:39.367963 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.368384 kubelet[2720]: E0813 01:25:39.368153 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.368384 kubelet[2720]: W0813 01:25:39.368164 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.368384 kubelet[2720]: E0813 01:25:39.368182 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.368384 kubelet[2720]: E0813 01:25:39.368332 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.368384 kubelet[2720]: W0813 01:25:39.368340 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.368384 kubelet[2720]: E0813 01:25:39.368347 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.368792 kubelet[2720]: E0813 01:25:39.368747 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.368792 kubelet[2720]: W0813 01:25:39.368761 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.368792 kubelet[2720]: E0813 01:25:39.368770 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.369208 kubelet[2720]: E0813 01:25:39.369125 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.369208 kubelet[2720]: W0813 01:25:39.369142 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.369208 kubelet[2720]: E0813 01:25:39.369150 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.369434 kubelet[2720]: E0813 01:25:39.369361 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.369434 kubelet[2720]: W0813 01:25:39.369381 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.369434 kubelet[2720]: E0813 01:25:39.369389 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.369842 kubelet[2720]: E0813 01:25:39.369634 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.369842 kubelet[2720]: W0813 01:25:39.369642 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.369842 kubelet[2720]: E0813 01:25:39.369649 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.370200 kubelet[2720]: E0813 01:25:39.369914 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.370200 kubelet[2720]: W0813 01:25:39.369922 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.370200 kubelet[2720]: E0813 01:25:39.369929 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.370200 kubelet[2720]: E0813 01:25:39.370094 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.370200 kubelet[2720]: W0813 01:25:39.370101 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.370200 kubelet[2720]: E0813 01:25:39.370108 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.371409 kubelet[2720]: E0813 01:25:39.371364 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.371409 kubelet[2720]: W0813 01:25:39.371380 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.371409 kubelet[2720]: E0813 01:25:39.371411 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.371843 kubelet[2720]: I0813 01:25:39.371435 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzwc2\" (UniqueName: \"kubernetes.io/projected/1e4523fc-9f2d-4edf-945c-d7ef00a14fdf-kube-api-access-dzwc2\") pod \"csi-node-driver-8bzzs\" (UID: \"1e4523fc-9f2d-4edf-945c-d7ef00a14fdf\") " pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:25:39.371868 kubelet[2720]: E0813 01:25:39.371841 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.371868 kubelet[2720]: W0813 01:25:39.371851 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.371868 kubelet[2720]: E0813 01:25:39.371860 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.371923 kubelet[2720]: I0813 01:25:39.371887 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1e4523fc-9f2d-4edf-945c-d7ef00a14fdf-kubelet-dir\") pod \"csi-node-driver-8bzzs\" (UID: \"1e4523fc-9f2d-4edf-945c-d7ef00a14fdf\") " pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:25:39.372125 kubelet[2720]: E0813 01:25:39.372091 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.372125 kubelet[2720]: W0813 01:25:39.372107 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.372125 kubelet[2720]: E0813 01:25:39.372114 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.372301 kubelet[2720]: I0813 01:25:39.372131 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1e4523fc-9f2d-4edf-945c-d7ef00a14fdf-registration-dir\") pod \"csi-node-driver-8bzzs\" (UID: \"1e4523fc-9f2d-4edf-945c-d7ef00a14fdf\") " pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:25:39.372301 kubelet[2720]: E0813 01:25:39.372286 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.372301 kubelet[2720]: W0813 01:25:39.372293 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.372479 kubelet[2720]: E0813 01:25:39.372302 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.372479 kubelet[2720]: I0813 01:25:39.372350 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1e4523fc-9f2d-4edf-945c-d7ef00a14fdf-socket-dir\") pod \"csi-node-driver-8bzzs\" (UID: \"1e4523fc-9f2d-4edf-945c-d7ef00a14fdf\") " pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:25:39.372670 kubelet[2720]: E0813 01:25:39.372539 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.372670 kubelet[2720]: W0813 01:25:39.372547 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.372670 kubelet[2720]: E0813 01:25:39.372556 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.372670 kubelet[2720]: I0813 01:25:39.372580 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"varrun\" (UniqueName: \"kubernetes.io/host-path/1e4523fc-9f2d-4edf-945c-d7ef00a14fdf-varrun\") pod \"csi-node-driver-8bzzs\" (UID: \"1e4523fc-9f2d-4edf-945c-d7ef00a14fdf\") " pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:25:39.376752 kubelet[2720]: E0813 01:25:39.374853 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.376752 kubelet[2720]: W0813 01:25:39.374867 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.376752 kubelet[2720]: E0813 01:25:39.374877 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.376752 kubelet[2720]: E0813 01:25:39.375059 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.376752 kubelet[2720]: W0813 01:25:39.375066 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.376752 kubelet[2720]: E0813 01:25:39.375074 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.376752 kubelet[2720]: E0813 01:25:39.375236 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.376752 kubelet[2720]: W0813 01:25:39.375243 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.376752 kubelet[2720]: E0813 01:25:39.375250 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.376752 kubelet[2720]: E0813 01:25:39.375422 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.376956 kubelet[2720]: W0813 01:25:39.375430 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.376956 kubelet[2720]: E0813 01:25:39.375437 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.376956 kubelet[2720]: E0813 01:25:39.375611 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.376956 kubelet[2720]: W0813 01:25:39.375618 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.376956 kubelet[2720]: E0813 01:25:39.375625 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.376956 kubelet[2720]: E0813 01:25:39.375828 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.376956 kubelet[2720]: W0813 01:25:39.375838 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.376956 kubelet[2720]: E0813 01:25:39.375849 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.376956 kubelet[2720]: E0813 01:25:39.376003 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.376956 kubelet[2720]: W0813 01:25:39.376010 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.377177 kubelet[2720]: E0813 01:25:39.376017 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.377177 kubelet[2720]: E0813 01:25:39.376190 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.377177 kubelet[2720]: W0813 01:25:39.376197 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.377177 kubelet[2720]: E0813 01:25:39.376204 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.377177 kubelet[2720]: E0813 01:25:39.376386 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.377177 kubelet[2720]: W0813 01:25:39.376394 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.377177 kubelet[2720]: E0813 01:25:39.376401 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.377177 kubelet[2720]: E0813 01:25:39.376572 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.377177 kubelet[2720]: W0813 01:25:39.376580 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.377177 kubelet[2720]: E0813 01:25:39.376587 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.392962 containerd[1567]: time="2025-08-13T01:25:39.392893605Z" level=info msg="connecting to shim 7091a88bf8ed2063aabec236166d769ff0994e69d63c60dc74e004d6f390e7e0" address="unix:///run/containerd/s/ade707ff47009547b42de9f9a80f56e616487f1b7812fb1279a94a48eb83783c" namespace=k8s.io protocol=ttrpc version=3 Aug 13 01:25:39.431100 systemd[1]: Started cri-containerd-7091a88bf8ed2063aabec236166d769ff0994e69d63c60dc74e004d6f390e7e0.scope - libcontainer container 7091a88bf8ed2063aabec236166d769ff0994e69d63c60dc74e004d6f390e7e0. Aug 13 01:25:39.456993 containerd[1567]: time="2025-08-13T01:25:39.456964400Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-node-2p8hj,Uid:4f334296-e8c0-427e-a090-d8fbd3582f26,Namespace:calico-system,Attempt:0,}" Aug 13 01:25:39.473906 kubelet[2720]: E0813 01:25:39.473807 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.473906 kubelet[2720]: W0813 01:25:39.473828 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.473906 kubelet[2720]: E0813 01:25:39.473845 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.474239 kubelet[2720]: E0813 01:25:39.474091 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.474239 kubelet[2720]: W0813 01:25:39.474104 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.474239 kubelet[2720]: E0813 01:25:39.474113 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.474662 kubelet[2720]: E0813 01:25:39.474338 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.474662 kubelet[2720]: W0813 01:25:39.474346 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.474662 kubelet[2720]: E0813 01:25:39.474354 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.474662 kubelet[2720]: E0813 01:25:39.474575 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.474662 kubelet[2720]: W0813 01:25:39.474583 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.474662 kubelet[2720]: E0813 01:25:39.474590 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.475375 kubelet[2720]: E0813 01:25:39.474816 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.475375 kubelet[2720]: W0813 01:25:39.474825 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.475375 kubelet[2720]: E0813 01:25:39.474833 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.475375 kubelet[2720]: E0813 01:25:39.475038 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.475375 kubelet[2720]: W0813 01:25:39.475046 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.475375 kubelet[2720]: E0813 01:25:39.475055 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.475375 kubelet[2720]: E0813 01:25:39.475230 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.475375 kubelet[2720]: W0813 01:25:39.475239 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.475375 kubelet[2720]: E0813 01:25:39.475246 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.476618 kubelet[2720]: E0813 01:25:39.475447 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.476618 kubelet[2720]: W0813 01:25:39.475455 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.476618 kubelet[2720]: E0813 01:25:39.475463 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.476618 kubelet[2720]: E0813 01:25:39.475917 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.476618 kubelet[2720]: W0813 01:25:39.475925 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.476618 kubelet[2720]: E0813 01:25:39.475933 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.477098 kubelet[2720]: E0813 01:25:39.476927 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.477098 kubelet[2720]: W0813 01:25:39.476935 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.477098 kubelet[2720]: E0813 01:25:39.476942 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.477153 kubelet[2720]: E0813 01:25:39.477119 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.477153 kubelet[2720]: W0813 01:25:39.477128 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.477153 kubelet[2720]: E0813 01:25:39.477135 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.477739 kubelet[2720]: E0813 01:25:39.477342 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.477739 kubelet[2720]: W0813 01:25:39.477354 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.477739 kubelet[2720]: E0813 01:25:39.477362 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.477739 kubelet[2720]: E0813 01:25:39.477537 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.477739 kubelet[2720]: W0813 01:25:39.477544 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.477739 kubelet[2720]: E0813 01:25:39.477551 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.478915 kubelet[2720]: E0813 01:25:39.477827 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.478915 kubelet[2720]: W0813 01:25:39.477834 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.478915 kubelet[2720]: E0813 01:25:39.477841 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.478915 kubelet[2720]: E0813 01:25:39.478013 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.478915 kubelet[2720]: W0813 01:25:39.478021 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.478915 kubelet[2720]: E0813 01:25:39.478028 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.478915 kubelet[2720]: E0813 01:25:39.478234 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.478915 kubelet[2720]: W0813 01:25:39.478241 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.478915 kubelet[2720]: E0813 01:25:39.478248 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.478915 kubelet[2720]: E0813 01:25:39.478431 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.480879 kubelet[2720]: W0813 01:25:39.478439 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.480879 kubelet[2720]: E0813 01:25:39.478445 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.480879 kubelet[2720]: E0813 01:25:39.478615 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.480879 kubelet[2720]: W0813 01:25:39.478621 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.480879 kubelet[2720]: E0813 01:25:39.478628 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.480879 kubelet[2720]: E0813 01:25:39.479229 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.480879 kubelet[2720]: W0813 01:25:39.479246 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.480879 kubelet[2720]: E0813 01:25:39.480205 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.481733 kubelet[2720]: E0813 01:25:39.481600 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.481733 kubelet[2720]: W0813 01:25:39.481613 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.481733 kubelet[2720]: E0813 01:25:39.481641 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.482868 kubelet[2720]: E0813 01:25:39.482855 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.483137 kubelet[2720]: W0813 01:25:39.482937 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.483137 kubelet[2720]: E0813 01:25:39.482949 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.484408 kubelet[2720]: E0813 01:25:39.484372 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.484408 kubelet[2720]: W0813 01:25:39.484385 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.484408 kubelet[2720]: E0813 01:25:39.484395 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.485271 kubelet[2720]: E0813 01:25:39.484989 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.485271 kubelet[2720]: W0813 01:25:39.485021 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.485271 kubelet[2720]: E0813 01:25:39.485031 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.485921 kubelet[2720]: E0813 01:25:39.485888 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.485921 kubelet[2720]: W0813 01:25:39.485899 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.485921 kubelet[2720]: E0813 01:25:39.485909 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.487199 kubelet[2720]: E0813 01:25:39.486648 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.487199 kubelet[2720]: W0813 01:25:39.487164 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.487199 kubelet[2720]: E0813 01:25:39.487179 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.488271 containerd[1567]: time="2025-08-13T01:25:39.488061364Z" level=info msg="connecting to shim 5105f19a4fa4aed619cbcaf7a7ad5ac97da1bec5f3e441a9c5640cdaf472ed9d" address="unix:///run/containerd/s/80cc981cbd846a3b6e47f304d0dd0986382b4ac30c826ac9de5859d4834de77a" namespace=k8s.io protocol=ttrpc version=3 Aug 13 01:25:39.508245 kubelet[2720]: E0813 01:25:39.508219 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:39.508245 kubelet[2720]: W0813 01:25:39.508238 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:39.508245 kubelet[2720]: E0813 01:25:39.508249 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:39.522017 systemd[1]: Started cri-containerd-5105f19a4fa4aed619cbcaf7a7ad5ac97da1bec5f3e441a9c5640cdaf472ed9d.scope - libcontainer container 5105f19a4fa4aed619cbcaf7a7ad5ac97da1bec5f3e441a9c5640cdaf472ed9d. Aug 13 01:25:39.542122 containerd[1567]: time="2025-08-13T01:25:39.542073972Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-typha-5bf7f956b7-szw5v,Uid:d0c18708-6c9b-42ce-bbc4-7fe0748e1776,Namespace:calico-system,Attempt:0,} returns sandbox id \"7091a88bf8ed2063aabec236166d769ff0994e69d63c60dc74e004d6f390e7e0\"" Aug 13 01:25:39.543701 kubelet[2720]: E0813 01:25:39.543600 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:39.547020 containerd[1567]: time="2025-08-13T01:25:39.546965080Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/typha:v3.30.2\"" Aug 13 01:25:39.576503 containerd[1567]: time="2025-08-13T01:25:39.576479313Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-node-2p8hj,Uid:4f334296-e8c0-427e-a090-d8fbd3582f26,Namespace:calico-system,Attempt:0,} returns sandbox id \"5105f19a4fa4aed619cbcaf7a7ad5ac97da1bec5f3e441a9c5640cdaf472ed9d\"" Aug 13 01:25:40.466365 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount3623203230.mount: Deactivated successfully. Aug 13 01:25:40.924031 containerd[1567]: time="2025-08-13T01:25:40.923911378Z" level=info msg="ImageCreate event name:\"ghcr.io/flatcar/calico/typha:v3.30.2\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:40.924917 containerd[1567]: time="2025-08-13T01:25:40.924864257Z" level=info msg="stop pulling image ghcr.io/flatcar/calico/typha:v3.30.2: active requests=0, bytes read=35233364" Aug 13 01:25:40.926345 containerd[1567]: time="2025-08-13T01:25:40.925358648Z" level=info msg="ImageCreate event name:\"sha256:b3baa600c7ff9cd50dc12f2529ef263aaa346dbeca13c77c6553d661fd216b54\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:40.926849 containerd[1567]: time="2025-08-13T01:25:40.926812747Z" level=info msg="ImageCreate event name:\"ghcr.io/flatcar/calico/typha@sha256:da29d745efe5eb7d25f765d3aa439f3fe60710a458efe39c285e58b02bd961af\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:40.927439 containerd[1567]: time="2025-08-13T01:25:40.927411638Z" level=info msg="Pulled image \"ghcr.io/flatcar/calico/typha:v3.30.2\" with image id \"sha256:b3baa600c7ff9cd50dc12f2529ef263aaa346dbeca13c77c6553d661fd216b54\", repo tag \"ghcr.io/flatcar/calico/typha:v3.30.2\", repo digest \"ghcr.io/flatcar/calico/typha@sha256:da29d745efe5eb7d25f765d3aa439f3fe60710a458efe39c285e58b02bd961af\", size \"35233218\" in 1.380408008s" Aug 13 01:25:40.927515 containerd[1567]: time="2025-08-13T01:25:40.927495028Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/typha:v3.30.2\" returns image reference \"sha256:b3baa600c7ff9cd50dc12f2529ef263aaa346dbeca13c77c6553d661fd216b54\"" Aug 13 01:25:40.929631 containerd[1567]: time="2025-08-13T01:25:40.929604497Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/pod2daemon-flexvol:v3.30.2\"" Aug 13 01:25:40.944740 containerd[1567]: time="2025-08-13T01:25:40.944684364Z" level=info msg="CreateContainer within sandbox \"7091a88bf8ed2063aabec236166d769ff0994e69d63c60dc74e004d6f390e7e0\" for container &ContainerMetadata{Name:calico-typha,Attempt:0,}" Aug 13 01:25:40.949429 containerd[1567]: time="2025-08-13T01:25:40.949389774Z" level=info msg="Container a6a57ab1f557ef6f8b80fa519cc494d4ad4ce164e33088aa825a9cc7c2fe5f7b: CDI devices from CRI Config.CDIDevices: []" Aug 13 01:25:40.954594 containerd[1567]: time="2025-08-13T01:25:40.954529392Z" level=info msg="CreateContainer within sandbox \"7091a88bf8ed2063aabec236166d769ff0994e69d63c60dc74e004d6f390e7e0\" for &ContainerMetadata{Name:calico-typha,Attempt:0,} returns container id \"a6a57ab1f557ef6f8b80fa519cc494d4ad4ce164e33088aa825a9cc7c2fe5f7b\"" Aug 13 01:25:40.955331 containerd[1567]: time="2025-08-13T01:25:40.955203453Z" level=info msg="StartContainer for \"a6a57ab1f557ef6f8b80fa519cc494d4ad4ce164e33088aa825a9cc7c2fe5f7b\"" Aug 13 01:25:40.956764 containerd[1567]: time="2025-08-13T01:25:40.956707022Z" level=info msg="connecting to shim a6a57ab1f557ef6f8b80fa519cc494d4ad4ce164e33088aa825a9cc7c2fe5f7b" address="unix:///run/containerd/s/ade707ff47009547b42de9f9a80f56e616487f1b7812fb1279a94a48eb83783c" protocol=ttrpc version=3 Aug 13 01:25:40.977890 systemd[1]: Started cri-containerd-a6a57ab1f557ef6f8b80fa519cc494d4ad4ce164e33088aa825a9cc7c2fe5f7b.scope - libcontainer container a6a57ab1f557ef6f8b80fa519cc494d4ad4ce164e33088aa825a9cc7c2fe5f7b. Aug 13 01:25:41.040139 containerd[1567]: time="2025-08-13T01:25:41.040057009Z" level=info msg="StartContainer for \"a6a57ab1f557ef6f8b80fa519cc494d4ad4ce164e33088aa825a9cc7c2fe5f7b\" returns successfully" Aug 13 01:25:41.073191 kubelet[2720]: E0813 01:25:41.073148 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:25:41.134438 kubelet[2720]: E0813 01:25:41.134387 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:41.180586 kubelet[2720]: E0813 01:25:41.180443 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.180586 kubelet[2720]: W0813 01:25:41.180462 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.180586 kubelet[2720]: E0813 01:25:41.180480 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.181001 kubelet[2720]: E0813 01:25:41.180938 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.181001 kubelet[2720]: W0813 01:25:41.180950 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.181001 kubelet[2720]: E0813 01:25:41.180960 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.186269 kubelet[2720]: E0813 01:25:41.185787 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.186269 kubelet[2720]: W0813 01:25:41.185800 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.186269 kubelet[2720]: E0813 01:25:41.185813 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.186604 kubelet[2720]: E0813 01:25:41.186472 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.186604 kubelet[2720]: W0813 01:25:41.186484 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.186604 kubelet[2720]: E0813 01:25:41.186493 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.186865 kubelet[2720]: E0813 01:25:41.186764 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.186865 kubelet[2720]: W0813 01:25:41.186775 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.186865 kubelet[2720]: E0813 01:25:41.186784 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.187082 kubelet[2720]: E0813 01:25:41.186987 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.187082 kubelet[2720]: W0813 01:25:41.186998 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.187082 kubelet[2720]: E0813 01:25:41.187006 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.187306 kubelet[2720]: E0813 01:25:41.187207 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.187306 kubelet[2720]: W0813 01:25:41.187217 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.187306 kubelet[2720]: E0813 01:25:41.187225 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.187457 kubelet[2720]: E0813 01:25:41.187424 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.187457 kubelet[2720]: W0813 01:25:41.187434 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.187457 kubelet[2720]: E0813 01:25:41.187444 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.187782 kubelet[2720]: E0813 01:25:41.187771 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.187875 kubelet[2720]: W0813 01:25:41.187830 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.187875 kubelet[2720]: E0813 01:25:41.187842 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.188117 kubelet[2720]: E0813 01:25:41.188067 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.188117 kubelet[2720]: W0813 01:25:41.188077 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.188117 kubelet[2720]: E0813 01:25:41.188085 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.188373 kubelet[2720]: E0813 01:25:41.188322 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.188373 kubelet[2720]: W0813 01:25:41.188331 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.188373 kubelet[2720]: E0813 01:25:41.188339 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.188594 kubelet[2720]: E0813 01:25:41.188564 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.188594 kubelet[2720]: W0813 01:25:41.188573 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.188594 kubelet[2720]: E0813 01:25:41.188580 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.188973 kubelet[2720]: E0813 01:25:41.188918 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.188973 kubelet[2720]: W0813 01:25:41.188929 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.188973 kubelet[2720]: E0813 01:25:41.188936 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.190935 kubelet[2720]: E0813 01:25:41.190921 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.191053 kubelet[2720]: W0813 01:25:41.191001 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.191053 kubelet[2720]: E0813 01:25:41.191014 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.191338 kubelet[2720]: E0813 01:25:41.191307 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.191338 kubelet[2720]: W0813 01:25:41.191318 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.191338 kubelet[2720]: E0813 01:25:41.191326 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.191764 kubelet[2720]: E0813 01:25:41.191748 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.191838 kubelet[2720]: W0813 01:25:41.191811 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.191838 kubelet[2720]: E0813 01:25:41.191823 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.192176 kubelet[2720]: E0813 01:25:41.192147 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.192176 kubelet[2720]: W0813 01:25:41.192157 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.192176 kubelet[2720]: E0813 01:25:41.192165 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.192499 kubelet[2720]: E0813 01:25:41.192471 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.192499 kubelet[2720]: W0813 01:25:41.192481 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.192499 kubelet[2720]: E0813 01:25:41.192489 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.192837 kubelet[2720]: E0813 01:25:41.192809 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.192837 kubelet[2720]: W0813 01:25:41.192818 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.192837 kubelet[2720]: E0813 01:25:41.192825 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.193177 kubelet[2720]: E0813 01:25:41.193145 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.193177 kubelet[2720]: W0813 01:25:41.193156 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.193177 kubelet[2720]: E0813 01:25:41.193166 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.193475 kubelet[2720]: E0813 01:25:41.193445 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.193475 kubelet[2720]: W0813 01:25:41.193457 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.193475 kubelet[2720]: E0813 01:25:41.193464 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.193785 kubelet[2720]: E0813 01:25:41.193755 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.193785 kubelet[2720]: W0813 01:25:41.193765 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.193785 kubelet[2720]: E0813 01:25:41.193774 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.194079 kubelet[2720]: E0813 01:25:41.194050 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.194079 kubelet[2720]: W0813 01:25:41.194061 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.194079 kubelet[2720]: E0813 01:25:41.194069 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.194966 kubelet[2720]: E0813 01:25:41.194934 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.194966 kubelet[2720]: W0813 01:25:41.194945 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.194966 kubelet[2720]: E0813 01:25:41.194954 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.195456 kubelet[2720]: E0813 01:25:41.195426 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.195456 kubelet[2720]: W0813 01:25:41.195438 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.195456 kubelet[2720]: E0813 01:25:41.195446 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.195772 kubelet[2720]: E0813 01:25:41.195742 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.195772 kubelet[2720]: W0813 01:25:41.195753 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.195772 kubelet[2720]: E0813 01:25:41.195761 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.197042 kubelet[2720]: E0813 01:25:41.197009 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.197042 kubelet[2720]: W0813 01:25:41.197021 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.197042 kubelet[2720]: E0813 01:25:41.197030 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.197531 kubelet[2720]: E0813 01:25:41.197500 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.197531 kubelet[2720]: W0813 01:25:41.197511 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.197531 kubelet[2720]: E0813 01:25:41.197519 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.197867 kubelet[2720]: E0813 01:25:41.197838 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.197867 kubelet[2720]: W0813 01:25:41.197848 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.197867 kubelet[2720]: E0813 01:25:41.197856 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.198197 kubelet[2720]: E0813 01:25:41.198167 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.198197 kubelet[2720]: W0813 01:25:41.198177 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.198197 kubelet[2720]: E0813 01:25:41.198185 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.198520 kubelet[2720]: E0813 01:25:41.198457 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.198520 kubelet[2720]: W0813 01:25:41.198466 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.198520 kubelet[2720]: E0813 01:25:41.198474 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.198828 kubelet[2720]: E0813 01:25:41.198793 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.198828 kubelet[2720]: W0813 01:25:41.198823 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.198896 kubelet[2720]: E0813 01:25:41.198848 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.199985 kubelet[2720]: E0813 01:25:41.199956 2720 driver-call.go:262] Failed to unmarshal output for command: init, output: "", error: unexpected end of JSON input Aug 13 01:25:41.199985 kubelet[2720]: W0813 01:25:41.199979 2720 driver-call.go:149] FlexVolume: driver call failed: executable: /opt/libexec/kubernetes/kubelet-plugins/volume/exec/nodeagent~uds/uds, args: [init], error: executable file not found in $PATH, output: "" Aug 13 01:25:41.200051 kubelet[2720]: E0813 01:25:41.199999 2720 plugins.go:703] "Error dynamically probing plugins" err="error creating Flexvolume plugin from directory nodeagent~uds, skipping. Error: unexpected end of JSON input" Aug 13 01:25:41.527496 containerd[1567]: time="2025-08-13T01:25:41.527434561Z" level=info msg="ImageCreate event name:\"ghcr.io/flatcar/calico/pod2daemon-flexvol:v3.30.2\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:41.528126 containerd[1567]: time="2025-08-13T01:25:41.528095610Z" level=info msg="stop pulling image ghcr.io/flatcar/calico/pod2daemon-flexvol:v3.30.2: active requests=0, bytes read=4446956" Aug 13 01:25:41.528746 containerd[1567]: time="2025-08-13T01:25:41.528587940Z" level=info msg="ImageCreate event name:\"sha256:639615519fa6f7bc4b4756066ba9780068fd291eacc36c120f6c555e62f2b00e\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:41.530056 containerd[1567]: time="2025-08-13T01:25:41.530024730Z" level=info msg="ImageCreate event name:\"ghcr.io/flatcar/calico/pod2daemon-flexvol@sha256:972be127eaecd7d1a2d5393b8d14f1ae8f88550bee83e0519e9590c7e15eb41b\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:41.530697 containerd[1567]: time="2025-08-13T01:25:41.530662950Z" level=info msg="Pulled image \"ghcr.io/flatcar/calico/pod2daemon-flexvol:v3.30.2\" with image id \"sha256:639615519fa6f7bc4b4756066ba9780068fd291eacc36c120f6c555e62f2b00e\", repo tag \"ghcr.io/flatcar/calico/pod2daemon-flexvol:v3.30.2\", repo digest \"ghcr.io/flatcar/calico/pod2daemon-flexvol@sha256:972be127eaecd7d1a2d5393b8d14f1ae8f88550bee83e0519e9590c7e15eb41b\", size \"5939619\" in 601.025563ms" Aug 13 01:25:41.530697 containerd[1567]: time="2025-08-13T01:25:41.530695560Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/pod2daemon-flexvol:v3.30.2\" returns image reference \"sha256:639615519fa6f7bc4b4756066ba9780068fd291eacc36c120f6c555e62f2b00e\"" Aug 13 01:25:41.535087 containerd[1567]: time="2025-08-13T01:25:41.535029619Z" level=info msg="CreateContainer within sandbox \"5105f19a4fa4aed619cbcaf7a7ad5ac97da1bec5f3e441a9c5640cdaf472ed9d\" for container &ContainerMetadata{Name:flexvol-driver,Attempt:0,}" Aug 13 01:25:41.544958 containerd[1567]: time="2025-08-13T01:25:41.544381468Z" level=info msg="Container d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6: CDI devices from CRI Config.CDIDevices: []" Aug 13 01:25:41.548686 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount4198855924.mount: Deactivated successfully. Aug 13 01:25:41.553402 containerd[1567]: time="2025-08-13T01:25:41.553373776Z" level=info msg="CreateContainer within sandbox \"5105f19a4fa4aed619cbcaf7a7ad5ac97da1bec5f3e441a9c5640cdaf472ed9d\" for &ContainerMetadata{Name:flexvol-driver,Attempt:0,} returns container id \"d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6\"" Aug 13 01:25:41.554653 containerd[1567]: time="2025-08-13T01:25:41.554624537Z" level=info msg="StartContainer for \"d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6\"" Aug 13 01:25:41.556301 containerd[1567]: time="2025-08-13T01:25:41.556263246Z" level=info msg="connecting to shim d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6" address="unix:///run/containerd/s/80cc981cbd846a3b6e47f304d0dd0986382b4ac30c826ac9de5859d4834de77a" protocol=ttrpc version=3 Aug 13 01:25:41.579864 systemd[1]: Started cri-containerd-d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6.scope - libcontainer container d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6. Aug 13 01:25:41.618831 containerd[1567]: time="2025-08-13T01:25:41.618793288Z" level=info msg="StartContainer for \"d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6\" returns successfully" Aug 13 01:25:41.632842 systemd[1]: cri-containerd-d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6.scope: Deactivated successfully. Aug 13 01:25:41.636373 containerd[1567]: time="2025-08-13T01:25:41.636345555Z" level=info msg="received exit event container_id:\"d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6\" id:\"d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6\" pid:3440 exited_at:{seconds:1755048341 nanos:636051125}" Aug 13 01:25:41.636535 containerd[1567]: time="2025-08-13T01:25:41.636511625Z" level=info msg="TaskExit event in podsandbox handler container_id:\"d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6\" id:\"d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6\" pid:3440 exited_at:{seconds:1755048341 nanos:636051125}" Aug 13 01:25:41.658873 systemd[1]: run-containerd-io.containerd.runtime.v2.task-k8s.io-d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6-rootfs.mount: Deactivated successfully. Aug 13 01:25:42.136371 kubelet[2720]: I0813 01:25:42.136308 2720 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Aug 13 01:25:42.137150 kubelet[2720]: E0813 01:25:42.136585 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:42.139133 containerd[1567]: time="2025-08-13T01:25:42.139080390Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/cni:v3.30.2\"" Aug 13 01:25:42.153675 kubelet[2720]: I0813 01:25:42.153149 2720 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="calico-system/calico-typha-5bf7f956b7-szw5v" podStartSLOduration=1.770881243 podStartE2EDuration="3.153133799s" podCreationTimestamp="2025-08-13 01:25:39 +0000 UTC" firstStartedPulling="2025-08-13 01:25:39.546064601 +0000 UTC m=+17.566613994" lastFinishedPulling="2025-08-13 01:25:40.928317167 +0000 UTC m=+18.948866550" observedRunningTime="2025-08-13 01:25:41.156611672 +0000 UTC m=+19.177161055" watchObservedRunningTime="2025-08-13 01:25:42.153133799 +0000 UTC m=+20.173683182" Aug 13 01:25:43.073776 kubelet[2720]: E0813 01:25:43.073687 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:25:43.846736 containerd[1567]: time="2025-08-13T01:25:43.846568091Z" level=info msg="ImageCreate event name:\"ghcr.io/flatcar/calico/cni:v3.30.2\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:43.847436 containerd[1567]: time="2025-08-13T01:25:43.847394862Z" level=info msg="stop pulling image ghcr.io/flatcar/calico/cni:v3.30.2: active requests=0, bytes read=70436221" Aug 13 01:25:43.847625 containerd[1567]: time="2025-08-13T01:25:43.847605112Z" level=info msg="ImageCreate event name:\"sha256:77a357d0d33e3016e61153f7d2b7de72371579c4aaeb767fb7ef0af606fe1630\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:43.849134 containerd[1567]: time="2025-08-13T01:25:43.849100392Z" level=info msg="ImageCreate event name:\"ghcr.io/flatcar/calico/cni@sha256:50686775cc60acb78bd92a66fa2d84e1700b2d8e43a718fbadbf35e59baefb4d\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:25:43.849731 containerd[1567]: time="2025-08-13T01:25:43.849694762Z" level=info msg="Pulled image \"ghcr.io/flatcar/calico/cni:v3.30.2\" with image id \"sha256:77a357d0d33e3016e61153f7d2b7de72371579c4aaeb767fb7ef0af606fe1630\", repo tag \"ghcr.io/flatcar/calico/cni:v3.30.2\", repo digest \"ghcr.io/flatcar/calico/cni@sha256:50686775cc60acb78bd92a66fa2d84e1700b2d8e43a718fbadbf35e59baefb4d\", size \"71928924\" in 1.710579972s" Aug 13 01:25:43.849819 containerd[1567]: time="2025-08-13T01:25:43.849803521Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/cni:v3.30.2\" returns image reference \"sha256:77a357d0d33e3016e61153f7d2b7de72371579c4aaeb767fb7ef0af606fe1630\"" Aug 13 01:25:43.853707 containerd[1567]: time="2025-08-13T01:25:43.853685031Z" level=info msg="CreateContainer within sandbox \"5105f19a4fa4aed619cbcaf7a7ad5ac97da1bec5f3e441a9c5640cdaf472ed9d\" for container &ContainerMetadata{Name:install-cni,Attempt:0,}" Aug 13 01:25:43.862379 containerd[1567]: time="2025-08-13T01:25:43.862281400Z" level=info msg="Container d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252: CDI devices from CRI Config.CDIDevices: []" Aug 13 01:25:43.864819 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount2375425066.mount: Deactivated successfully. Aug 13 01:25:43.873745 containerd[1567]: time="2025-08-13T01:25:43.873701250Z" level=info msg="CreateContainer within sandbox \"5105f19a4fa4aed619cbcaf7a7ad5ac97da1bec5f3e441a9c5640cdaf472ed9d\" for &ContainerMetadata{Name:install-cni,Attempt:0,} returns container id \"d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252\"" Aug 13 01:25:43.874852 containerd[1567]: time="2025-08-13T01:25:43.874271430Z" level=info msg="StartContainer for \"d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252\"" Aug 13 01:25:43.876466 containerd[1567]: time="2025-08-13T01:25:43.876432660Z" level=info msg="connecting to shim d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252" address="unix:///run/containerd/s/80cc981cbd846a3b6e47f304d0dd0986382b4ac30c826ac9de5859d4834de77a" protocol=ttrpc version=3 Aug 13 01:25:43.900834 systemd[1]: Started cri-containerd-d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252.scope - libcontainer container d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252. Aug 13 01:25:43.943937 containerd[1567]: time="2025-08-13T01:25:43.943878536Z" level=info msg="StartContainer for \"d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252\" returns successfully" Aug 13 01:25:44.521363 containerd[1567]: time="2025-08-13T01:25:44.521315718Z" level=error msg="failed to reload cni configuration after receiving fs change event(WRITE \"/etc/cni/net.d/calico-kubeconfig\")" error="cni config load failed: no network config found in /etc/cni/net.d: cni plugin not initialized: failed to load cni config" Aug 13 01:25:44.523889 systemd[1]: cri-containerd-d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252.scope: Deactivated successfully. Aug 13 01:25:44.524189 systemd[1]: cri-containerd-d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252.scope: Consumed 528ms CPU time, 192.9M memory peak, 171.2M written to disk. Aug 13 01:25:44.526605 containerd[1567]: time="2025-08-13T01:25:44.526574878Z" level=info msg="TaskExit event in podsandbox handler container_id:\"d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252\" id:\"d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252\" pid:3500 exited_at:{seconds:1755048344 nanos:526109018}" Aug 13 01:25:44.526862 containerd[1567]: time="2025-08-13T01:25:44.526814438Z" level=info msg="received exit event container_id:\"d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252\" id:\"d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252\" pid:3500 exited_at:{seconds:1755048344 nanos:526109018}" Aug 13 01:25:44.553189 systemd[1]: run-containerd-io.containerd.runtime.v2.task-k8s.io-d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252-rootfs.mount: Deactivated successfully. Aug 13 01:25:44.601293 kubelet[2720]: I0813 01:25:44.600114 2720 kubelet_node_status.go:501] "Fast updating node status as it just became ready" Aug 13 01:25:44.637975 systemd[1]: Created slice kubepods-burstable-pod567c4932_37b3_4053_b427_2472bc4c195b.slice - libcontainer container kubepods-burstable-pod567c4932_37b3_4053_b427_2472bc4c195b.slice. Aug 13 01:25:44.649798 systemd[1]: Created slice kubepods-burstable-pode3a3eaf4_0523_46b9_8d19_1303011f36f8.slice - libcontainer container kubepods-burstable-pode3a3eaf4_0523_46b9_8d19_1303011f36f8.slice. Aug 13 01:25:44.663882 systemd[1]: Created slice kubepods-besteffort-podffb60df5_70ce_4ebe_ad70_a3fd8de49793.slice - libcontainer container kubepods-besteffort-podffb60df5_70ce_4ebe_ad70_a3fd8de49793.slice. Aug 13 01:25:44.674893 systemd[1]: Created slice kubepods-besteffort-pod308de2f1_eb6e_47d8_b49a_85db02a09390.slice - libcontainer container kubepods-besteffort-pod308de2f1_eb6e_47d8_b49a_85db02a09390.slice. Aug 13 01:25:44.685418 systemd[1]: Created slice kubepods-besteffort-pod7a6f8cd3_7ee5_4677_9b3c_6e44d08257c6.slice - libcontainer container kubepods-besteffort-pod7a6f8cd3_7ee5_4677_9b3c_6e44d08257c6.slice. Aug 13 01:25:44.695154 systemd[1]: Created slice kubepods-besteffort-pod045c69cc_698d_4c00_8a26_dcea7fba5a2e.slice - libcontainer container kubepods-besteffort-pod045c69cc_698d_4c00_8a26_dcea7fba5a2e.slice. Aug 13 01:25:44.701677 systemd[1]: Created slice kubepods-besteffort-podbac4ab4b_8a28_40be_9dfc_ab98956ae605.slice - libcontainer container kubepods-besteffort-podbac4ab4b_8a28_40be_9dfc_ab98956ae605.slice. Aug 13 01:25:44.718596 kubelet[2720]: I0813 01:25:44.718111 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2299t\" (UniqueName: \"kubernetes.io/projected/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-kube-api-access-2299t\") pod \"goldmane-768f4c5c69-mf54f\" (UID: \"7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6\") " pod="calico-system/goldmane-768f4c5c69-mf54f" Aug 13 01:25:44.718856 kubelet[2720]: I0813 01:25:44.718788 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e3a3eaf4-0523-46b9-8d19-1303011f36f8-config-volume\") pod \"coredns-674b8bbfcf-jlpx4\" (UID: \"e3a3eaf4-0523-46b9-8d19-1303011f36f8\") " pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:25:44.718856 kubelet[2720]: I0813 01:25:44.718821 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whisker-backend-key-pair\" (UniqueName: \"kubernetes.io/secret/308de2f1-eb6e-47d8-b49a-85db02a09390-whisker-backend-key-pair\") pod \"whisker-99dd966c4-29sst\" (UID: \"308de2f1-eb6e-47d8-b49a-85db02a09390\") " pod="calico-system/whisker-99dd966c4-29sst" Aug 13 01:25:44.719273 kubelet[2720]: I0813 01:25:44.718862 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtd9k\" (UniqueName: \"kubernetes.io/projected/bac4ab4b-8a28-40be-9dfc-ab98956ae605-kube-api-access-wtd9k\") pod \"calico-apiserver-7566fdf5fb-n55b2\" (UID: \"bac4ab4b-8a28-40be-9dfc-ab98956ae605\") " pod="calico-apiserver/calico-apiserver-7566fdf5fb-n55b2" Aug 13 01:25:44.719273 kubelet[2720]: I0813 01:25:44.718888 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"goldmane-key-pair\" (UniqueName: \"kubernetes.io/secret/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-goldmane-key-pair\") pod \"goldmane-768f4c5c69-mf54f\" (UID: \"7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6\") " pod="calico-system/goldmane-768f4c5c69-mf54f" Aug 13 01:25:44.719273 kubelet[2720]: I0813 01:25:44.718907 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whisker-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/308de2f1-eb6e-47d8-b49a-85db02a09390-whisker-ca-bundle\") pod \"whisker-99dd966c4-29sst\" (UID: \"308de2f1-eb6e-47d8-b49a-85db02a09390\") " pod="calico-system/whisker-99dd966c4-29sst" Aug 13 01:25:44.719273 kubelet[2720]: I0813 01:25:44.718925 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncbl9\" (UniqueName: \"kubernetes.io/projected/045c69cc-698d-4c00-8a26-dcea7fba5a2e-kube-api-access-ncbl9\") pod \"calico-apiserver-7566fdf5fb-vbzh6\" (UID: \"045c69cc-698d-4c00-8a26-dcea7fba5a2e\") " pod="calico-apiserver/calico-apiserver-7566fdf5fb-vbzh6" Aug 13 01:25:44.719273 kubelet[2720]: I0813 01:25:44.718940 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-config\") pod \"goldmane-768f4c5c69-mf54f\" (UID: \"7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6\") " pod="calico-system/goldmane-768f4c5c69-mf54f" Aug 13 01:25:44.719393 kubelet[2720]: I0813 01:25:44.718958 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdpj9\" (UniqueName: \"kubernetes.io/projected/e3a3eaf4-0523-46b9-8d19-1303011f36f8-kube-api-access-jdpj9\") pod \"coredns-674b8bbfcf-jlpx4\" (UID: \"e3a3eaf4-0523-46b9-8d19-1303011f36f8\") " pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:25:44.719393 kubelet[2720]: I0813 01:25:44.718975 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"calico-apiserver-certs\" (UniqueName: \"kubernetes.io/secret/bac4ab4b-8a28-40be-9dfc-ab98956ae605-calico-apiserver-certs\") pod \"calico-apiserver-7566fdf5fb-n55b2\" (UID: \"bac4ab4b-8a28-40be-9dfc-ab98956ae605\") " pod="calico-apiserver/calico-apiserver-7566fdf5fb-n55b2" Aug 13 01:25:44.719393 kubelet[2720]: I0813 01:25:44.718998 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lncr6\" (UniqueName: \"kubernetes.io/projected/567c4932-37b3-4053-b427-2472bc4c195b-kube-api-access-lncr6\") pod \"coredns-674b8bbfcf-b2ds8\" (UID: \"567c4932-37b3-4053-b427-2472bc4c195b\") " pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:25:44.719393 kubelet[2720]: I0813 01:25:44.719016 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69trf\" (UniqueName: \"kubernetes.io/projected/ffb60df5-70ce-4ebe-ad70-a3fd8de49793-kube-api-access-69trf\") pod \"calico-kube-controllers-b57d47f84-lznkx\" (UID: \"ffb60df5-70ce-4ebe-ad70-a3fd8de49793\") " pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:25:44.719393 kubelet[2720]: I0813 01:25:44.719033 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"calico-apiserver-certs\" (UniqueName: \"kubernetes.io/secret/045c69cc-698d-4c00-8a26-dcea7fba5a2e-calico-apiserver-certs\") pod \"calico-apiserver-7566fdf5fb-vbzh6\" (UID: \"045c69cc-698d-4c00-8a26-dcea7fba5a2e\") " pod="calico-apiserver/calico-apiserver-7566fdf5fb-vbzh6" Aug 13 01:25:44.719499 kubelet[2720]: I0813 01:25:44.719049 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tigera-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ffb60df5-70ce-4ebe-ad70-a3fd8de49793-tigera-ca-bundle\") pod \"calico-kube-controllers-b57d47f84-lznkx\" (UID: \"ffb60df5-70ce-4ebe-ad70-a3fd8de49793\") " pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:25:44.719499 kubelet[2720]: I0813 01:25:44.719065 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5x8w\" (UniqueName: \"kubernetes.io/projected/308de2f1-eb6e-47d8-b49a-85db02a09390-kube-api-access-t5x8w\") pod \"whisker-99dd966c4-29sst\" (UID: \"308de2f1-eb6e-47d8-b49a-85db02a09390\") " pod="calico-system/whisker-99dd966c4-29sst" Aug 13 01:25:44.719499 kubelet[2720]: I0813 01:25:44.719085 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/567c4932-37b3-4053-b427-2472bc4c195b-config-volume\") pod \"coredns-674b8bbfcf-b2ds8\" (UID: \"567c4932-37b3-4053-b427-2472bc4c195b\") " pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:25:44.719499 kubelet[2720]: I0813 01:25:44.719101 2720 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"goldmane-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-goldmane-ca-bundle\") pod \"goldmane-768f4c5c69-mf54f\" (UID: \"7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6\") " pod="calico-system/goldmane-768f4c5c69-mf54f" Aug 13 01:25:44.949380 kubelet[2720]: E0813 01:25:44.949282 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:44.950146 containerd[1567]: time="2025-08-13T01:25:44.950105097Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:25:44.958410 kubelet[2720]: E0813 01:25:44.957629 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:44.959648 containerd[1567]: time="2025-08-13T01:25:44.959623106Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:25:44.969594 containerd[1567]: time="2025-08-13T01:25:44.969562405Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:25:44.980547 containerd[1567]: time="2025-08-13T01:25:44.980522116Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:whisker-99dd966c4-29sst,Uid:308de2f1-eb6e-47d8-b49a-85db02a09390,Namespace:calico-system,Attempt:0,}" Aug 13 01:25:44.995118 containerd[1567]: time="2025-08-13T01:25:44.995082455Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:goldmane-768f4c5c69-mf54f,Uid:7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6,Namespace:calico-system,Attempt:0,}" Aug 13 01:25:45.008372 containerd[1567]: time="2025-08-13T01:25:45.008342655Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-apiserver-7566fdf5fb-vbzh6,Uid:045c69cc-698d-4c00-8a26-dcea7fba5a2e,Namespace:calico-apiserver,Attempt:0,}" Aug 13 01:25:45.010888 containerd[1567]: time="2025-08-13T01:25:45.009224055Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-apiserver-7566fdf5fb-n55b2,Uid:bac4ab4b-8a28-40be-9dfc-ab98956ae605,Namespace:calico-apiserver,Attempt:0,}" Aug 13 01:25:45.081965 systemd[1]: Created slice kubepods-besteffort-pod1e4523fc_9f2d_4edf_945c_d7ef00a14fdf.slice - libcontainer container kubepods-besteffort-pod1e4523fc_9f2d_4edf_945c_d7ef00a14fdf.slice. Aug 13 01:25:45.086210 containerd[1567]: time="2025-08-13T01:25:45.085945636Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:25:45.159414 containerd[1567]: time="2025-08-13T01:25:45.159369916Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/node:v3.30.2\"" Aug 13 01:25:45.201697 containerd[1567]: time="2025-08-13T01:25:45.201559256Z" level=error msg="Failed to destroy network for sandbox \"f387f9b4b01dd25968dd8f456e3a047b78051b06e1a17c49cff40df46362d959\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.202864 containerd[1567]: time="2025-08-13T01:25:45.202839996Z" level=error msg="Failed to destroy network for sandbox \"01fd2e4c040629cf1e50a99f26b39b6d2441b3eb11a11c17f9684dd7c353b7db\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.204944 containerd[1567]: time="2025-08-13T01:25:45.204902536Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"f387f9b4b01dd25968dd8f456e3a047b78051b06e1a17c49cff40df46362d959\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.206343 containerd[1567]: time="2025-08-13T01:25:45.206134606Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"01fd2e4c040629cf1e50a99f26b39b6d2441b3eb11a11c17f9684dd7c353b7db\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.206418 kubelet[2720]: E0813 01:25:45.205589 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f387f9b4b01dd25968dd8f456e3a047b78051b06e1a17c49cff40df46362d959\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.206418 kubelet[2720]: E0813 01:25:45.206291 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f387f9b4b01dd25968dd8f456e3a047b78051b06e1a17c49cff40df46362d959\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:25:45.206418 kubelet[2720]: E0813 01:25:45.206315 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f387f9b4b01dd25968dd8f456e3a047b78051b06e1a17c49cff40df46362d959\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:25:45.206418 kubelet[2720]: E0813 01:25:45.206311 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"01fd2e4c040629cf1e50a99f26b39b6d2441b3eb11a11c17f9684dd7c353b7db\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.206541 kubelet[2720]: E0813 01:25:45.206383 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"01fd2e4c040629cf1e50a99f26b39b6d2441b3eb11a11c17f9684dd7c353b7db\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:25:45.206541 kubelet[2720]: E0813 01:25:45.206404 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"01fd2e4c040629cf1e50a99f26b39b6d2441b3eb11a11c17f9684dd7c353b7db\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:25:45.207050 kubelet[2720]: E0813 01:25:45.206447 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"01fd2e4c040629cf1e50a99f26b39b6d2441b3eb11a11c17f9684dd7c353b7db\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:25:45.207050 kubelet[2720]: E0813 01:25:45.206767 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"f387f9b4b01dd25968dd8f456e3a047b78051b06e1a17c49cff40df46362d959\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:25:45.219133 containerd[1567]: time="2025-08-13T01:25:45.219040286Z" level=error msg="Failed to destroy network for sandbox \"d35804cd1294722447a42b420170f07e370530e1e60dafe299296248d547f5c1\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.219806 containerd[1567]: time="2025-08-13T01:25:45.219698426Z" level=error msg="Failed to destroy network for sandbox \"39cc53273b09988196eb7246f8a019b6092e3c2a3b54c4326726d60bbd2f7ec8\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.221549 containerd[1567]: time="2025-08-13T01:25:45.221499276Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:whisker-99dd966c4-29sst,Uid:308de2f1-eb6e-47d8-b49a-85db02a09390,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"d35804cd1294722447a42b420170f07e370530e1e60dafe299296248d547f5c1\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.222529 containerd[1567]: time="2025-08-13T01:25:45.222504586Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"39cc53273b09988196eb7246f8a019b6092e3c2a3b54c4326726d60bbd2f7ec8\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.223302 kubelet[2720]: E0813 01:25:45.223235 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"39cc53273b09988196eb7246f8a019b6092e3c2a3b54c4326726d60bbd2f7ec8\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.223302 kubelet[2720]: E0813 01:25:45.223273 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"39cc53273b09988196eb7246f8a019b6092e3c2a3b54c4326726d60bbd2f7ec8\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:25:45.223302 kubelet[2720]: E0813 01:25:45.223290 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"39cc53273b09988196eb7246f8a019b6092e3c2a3b54c4326726d60bbd2f7ec8\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:25:45.223528 kubelet[2720]: E0813 01:25:45.223329 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"39cc53273b09988196eb7246f8a019b6092e3c2a3b54c4326726d60bbd2f7ec8\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:25:45.224020 kubelet[2720]: E0813 01:25:45.223598 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d35804cd1294722447a42b420170f07e370530e1e60dafe299296248d547f5c1\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.224020 kubelet[2720]: E0813 01:25:45.223672 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d35804cd1294722447a42b420170f07e370530e1e60dafe299296248d547f5c1\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/whisker-99dd966c4-29sst" Aug 13 01:25:45.224020 kubelet[2720]: E0813 01:25:45.223690 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d35804cd1294722447a42b420170f07e370530e1e60dafe299296248d547f5c1\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/whisker-99dd966c4-29sst" Aug 13 01:25:45.224351 kubelet[2720]: E0813 01:25:45.223869 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"whisker-99dd966c4-29sst_calico-system(308de2f1-eb6e-47d8-b49a-85db02a09390)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"whisker-99dd966c4-29sst_calico-system(308de2f1-eb6e-47d8-b49a-85db02a09390)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"d35804cd1294722447a42b420170f07e370530e1e60dafe299296248d547f5c1\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/whisker-99dd966c4-29sst" podUID="308de2f1-eb6e-47d8-b49a-85db02a09390" Aug 13 01:25:45.241882 containerd[1567]: time="2025-08-13T01:25:45.241641416Z" level=error msg="Failed to destroy network for sandbox \"0f42113ae4b55541054397da0336e1c6eb0799b909fb1f9b006019c226cbf2c3\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.243543 containerd[1567]: time="2025-08-13T01:25:45.243449186Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-apiserver-7566fdf5fb-n55b2,Uid:bac4ab4b-8a28-40be-9dfc-ab98956ae605,Namespace:calico-apiserver,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"0f42113ae4b55541054397da0336e1c6eb0799b909fb1f9b006019c226cbf2c3\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.244262 kubelet[2720]: E0813 01:25:45.244049 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0f42113ae4b55541054397da0336e1c6eb0799b909fb1f9b006019c226cbf2c3\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.245370 kubelet[2720]: E0813 01:25:45.244361 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0f42113ae4b55541054397da0336e1c6eb0799b909fb1f9b006019c226cbf2c3\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-apiserver/calico-apiserver-7566fdf5fb-n55b2" Aug 13 01:25:45.245370 kubelet[2720]: E0813 01:25:45.245159 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0f42113ae4b55541054397da0336e1c6eb0799b909fb1f9b006019c226cbf2c3\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-apiserver/calico-apiserver-7566fdf5fb-n55b2" Aug 13 01:25:45.245370 kubelet[2720]: E0813 01:25:45.245263 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-apiserver-7566fdf5fb-n55b2_calico-apiserver(bac4ab4b-8a28-40be-9dfc-ab98956ae605)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-apiserver-7566fdf5fb-n55b2_calico-apiserver(bac4ab4b-8a28-40be-9dfc-ab98956ae605)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"0f42113ae4b55541054397da0336e1c6eb0799b909fb1f9b006019c226cbf2c3\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-apiserver/calico-apiserver-7566fdf5fb-n55b2" podUID="bac4ab4b-8a28-40be-9dfc-ab98956ae605" Aug 13 01:25:45.273260 containerd[1567]: time="2025-08-13T01:25:45.273226696Z" level=error msg="Failed to destroy network for sandbox \"b4eff5688d593c940591882bf2d4320adc0ac3e200a34b8b24eaa174bfda2ffd\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.274058 containerd[1567]: time="2025-08-13T01:25:45.274027627Z" level=error msg="Failed to destroy network for sandbox \"db878c20deed912c38874e5e3c05f1cd94be84804fe6a53791c3e13c3431d0e6\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.274615 containerd[1567]: time="2025-08-13T01:25:45.274584677Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:goldmane-768f4c5c69-mf54f,Uid:7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"b4eff5688d593c940591882bf2d4320adc0ac3e200a34b8b24eaa174bfda2ffd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.275016 kubelet[2720]: E0813 01:25:45.274988 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b4eff5688d593c940591882bf2d4320adc0ac3e200a34b8b24eaa174bfda2ffd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.275154 kubelet[2720]: E0813 01:25:45.275131 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b4eff5688d593c940591882bf2d4320adc0ac3e200a34b8b24eaa174bfda2ffd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/goldmane-768f4c5c69-mf54f" Aug 13 01:25:45.275263 kubelet[2720]: E0813 01:25:45.275204 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b4eff5688d593c940591882bf2d4320adc0ac3e200a34b8b24eaa174bfda2ffd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/goldmane-768f4c5c69-mf54f" Aug 13 01:25:45.275298 containerd[1567]: time="2025-08-13T01:25:45.275205446Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"db878c20deed912c38874e5e3c05f1cd94be84804fe6a53791c3e13c3431d0e6\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.275447 kubelet[2720]: E0813 01:25:45.275403 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"db878c20deed912c38874e5e3c05f1cd94be84804fe6a53791c3e13c3431d0e6\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.275901 containerd[1567]: time="2025-08-13T01:25:45.275534446Z" level=error msg="Failed to destroy network for sandbox \"176aa8b8e8ee6cb0773137ad19e8a84c890ebf474b9b3dcd233d481bdd6b506d\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.276168 kubelet[2720]: E0813 01:25:45.276143 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"db878c20deed912c38874e5e3c05f1cd94be84804fe6a53791c3e13c3431d0e6\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:25:45.276456 kubelet[2720]: E0813 01:25:45.276311 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"db878c20deed912c38874e5e3c05f1cd94be84804fe6a53791c3e13c3431d0e6\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:25:45.276456 kubelet[2720]: E0813 01:25:45.276358 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"db878c20deed912c38874e5e3c05f1cd94be84804fe6a53791c3e13c3431d0e6\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:25:45.276456 kubelet[2720]: E0813 01:25:45.275659 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"goldmane-768f4c5c69-mf54f_calico-system(7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"goldmane-768f4c5c69-mf54f_calico-system(7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"b4eff5688d593c940591882bf2d4320adc0ac3e200a34b8b24eaa174bfda2ffd\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/goldmane-768f4c5c69-mf54f" podUID="7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6" Aug 13 01:25:45.277294 containerd[1567]: time="2025-08-13T01:25:45.276923056Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-apiserver-7566fdf5fb-vbzh6,Uid:045c69cc-698d-4c00-8a26-dcea7fba5a2e,Namespace:calico-apiserver,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"176aa8b8e8ee6cb0773137ad19e8a84c890ebf474b9b3dcd233d481bdd6b506d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.277592 kubelet[2720]: E0813 01:25:45.277553 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"176aa8b8e8ee6cb0773137ad19e8a84c890ebf474b9b3dcd233d481bdd6b506d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:45.277793 kubelet[2720]: E0813 01:25:45.277666 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"176aa8b8e8ee6cb0773137ad19e8a84c890ebf474b9b3dcd233d481bdd6b506d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-apiserver/calico-apiserver-7566fdf5fb-vbzh6" Aug 13 01:25:45.277793 kubelet[2720]: E0813 01:25:45.277687 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"176aa8b8e8ee6cb0773137ad19e8a84c890ebf474b9b3dcd233d481bdd6b506d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-apiserver/calico-apiserver-7566fdf5fb-vbzh6" Aug 13 01:25:45.277793 kubelet[2720]: E0813 01:25:45.277765 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-apiserver-7566fdf5fb-vbzh6_calico-apiserver(045c69cc-698d-4c00-8a26-dcea7fba5a2e)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-apiserver-7566fdf5fb-vbzh6_calico-apiserver(045c69cc-698d-4c00-8a26-dcea7fba5a2e)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"176aa8b8e8ee6cb0773137ad19e8a84c890ebf474b9b3dcd233d481bdd6b506d\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-apiserver/calico-apiserver-7566fdf5fb-vbzh6" podUID="045c69cc-698d-4c00-8a26-dcea7fba5a2e" Aug 13 01:25:45.439002 kubelet[2720]: I0813 01:25:45.438842 2720 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Aug 13 01:25:45.439366 kubelet[2720]: E0813 01:25:45.439318 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:45.864136 systemd[1]: run-netns-cni\x2d6e6a6ecf\x2d36fc\x2d1189\x2d548c\x2de0703974fe8e.mount: Deactivated successfully. Aug 13 01:25:45.864868 systemd[1]: run-netns-cni\x2df24e4f56\x2d5ef1\x2d0c19\x2d782f\x2d6b76d47c0ed3.mount: Deactivated successfully. Aug 13 01:25:45.864943 systemd[1]: run-netns-cni\x2df5e7f40d\x2dad3b\x2ddd5d\x2d4dae\x2db3f423796dd8.mount: Deactivated successfully. Aug 13 01:25:45.865012 systemd[1]: run-netns-cni\x2d174052a3\x2d29de\x2d3d96\x2d5473\x2d33c4e9c957c7.mount: Deactivated successfully. Aug 13 01:25:46.161551 kubelet[2720]: E0813 01:25:46.161424 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:47.446708 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount4008541472.mount: Deactivated successfully. Aug 13 01:25:47.448162 containerd[1567]: time="2025-08-13T01:25:47.447449955Z" level=error msg="PullImage \"ghcr.io/flatcar/calico/node:v3.30.2\" failed" error="rpc error: code = Unknown desc = failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount4008541472: write /var/lib/containerd/tmpmounts/containerd-mount4008541472/usr/bin/calico-node: no space left on device" Aug 13 01:25:47.448162 containerd[1567]: time="2025-08-13T01:25:47.447551286Z" level=info msg="stop pulling image ghcr.io/flatcar/calico/node:v3.30.2: active requests=0, bytes read=158500163" Aug 13 01:25:47.449027 kubelet[2720]: E0813 01:25:47.448989 2720 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount4008541472: write /var/lib/containerd/tmpmounts/containerd-mount4008541472/usr/bin/calico-node: no space left on device" image="ghcr.io/flatcar/calico/node:v3.30.2" Aug 13 01:25:47.449294 kubelet[2720]: E0813 01:25:47.449042 2720 kuberuntime_image.go:42] "Failed to pull image" err="failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount4008541472: write /var/lib/containerd/tmpmounts/containerd-mount4008541472/usr/bin/calico-node: no space left on device" image="ghcr.io/flatcar/calico/node:v3.30.2" Aug 13 01:25:47.449475 kubelet[2720]: E0813 01:25:47.449277 2720 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:calico-node,Image:ghcr.io/flatcar/calico/node:v3.30.2,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:DATASTORE_TYPE,Value:kubernetes,ValueFrom:nil,},EnvVar{Name:WAIT_FOR_DATASTORE,Value:true,ValueFrom:nil,},EnvVar{Name:CLUSTER_TYPE,Value:k8s,operator,bgp,ValueFrom:nil,},EnvVar{Name:CALICO_DISABLE_FILE_LOGGING,Value:false,ValueFrom:nil,},EnvVar{Name:FELIX_DEFAULTENDPOINTTOHOSTACTION,Value:ACCEPT,ValueFrom:nil,},EnvVar{Name:FELIX_HEALTHENABLED,Value:true,ValueFrom:nil,},EnvVar{Name:FELIX_HEALTHPORT,Value:9099,ValueFrom:nil,},EnvVar{Name:NODENAME,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:spec.nodeName,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:FELIX_TYPHAK8SNAMESPACE,Value:calico-system,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHAK8SSERVICENAME,Value:calico-typha,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHACAFILE,Value:/etc/pki/tls/certs/tigera-ca-bundle.crt,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHACERTFILE,Value:/node-certs/tls.crt,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHAKEYFILE,Value:/node-certs/tls.key,ValueFrom:nil,},EnvVar{Name:NO_DEFAULT_POOLS,Value:true,ValueFrom:nil,},EnvVar{Name:FELIX_FLOWLOGSGOLDMANESERVER,Value:goldmane.calico-system.svc:7443,ValueFrom:nil,},EnvVar{Name:FELIX_FLOWLOGSFLUSHINTERVAL,Value:15,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHACN,Value:typha-server,ValueFrom:nil,},EnvVar{Name:CALICO_MANAGE_CNI,Value:true,ValueFrom:nil,},EnvVar{Name:CALICO_NETWORKING_BACKEND,Value:bird,ValueFrom:nil,},EnvVar{Name:IP,Value:autodetect,ValueFrom:nil,},EnvVar{Name:IP_AUTODETECTION_METHOD,Value:first-found,ValueFrom:nil,},EnvVar{Name:IP6,Value:none,ValueFrom:nil,},EnvVar{Name:FELIX_IPV6SUPPORT,Value:false,ValueFrom:nil,},EnvVar{Name:KUBERNETES_SERVICE_HOST,Value:10.96.0.1,ValueFrom:nil,},EnvVar{Name:KUBERNETES_SERVICE_PORT,Value:443,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:cni-log-dir,ReadOnly:false,MountPath:/var/log/calico/cni,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:cni-net-dir,ReadOnly:false,MountPath:/host/etc/cni/net.d,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:lib-modules,ReadOnly:true,MountPath:/lib/modules,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:node-certs,ReadOnly:true,MountPath:/node-certs,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:policysync,ReadOnly:false,MountPath:/var/run/nodeagent,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:tigera-ca-bundle,ReadOnly:true,MountPath:/etc/pki/tls/certs,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:tigera-ca-bundle,ReadOnly:true,MountPath:/etc/pki/tls/cert.pem,SubPath:ca-bundle.crt,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:var-lib-calico,ReadOnly:false,MountPath:/var/lib/calico,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:var-run-calico,ReadOnly:false,MountPath:/var/run/calico,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:xtables-lock,ReadOnly:false,MountPath:/run/xtables.lock,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-jl8ww,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/liveness,Port:{0 9099 },Host:localhost,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:10,PeriodSeconds:60,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:&ExecAction{Command:[/bin/calico-node -bird-ready -felix-ready],},HTTPGet:nil,TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:10,PeriodSeconds:30,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:&Lifecycle{PostStart:nil,PreStop:&LifecycleHandler{Exec:&ExecAction{Command:[/bin/calico-node -shutdown],},HTTPGet:nil,TCPSocket:nil,Sleep:nil,},StopSignal:nil,},TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:*true,SELinuxOptions:nil,RunAsUser:*0,RunAsNonRoot:*false,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*true,RunAsGroup:*0,ProcMount:nil,WindowsOptions:nil,SeccompProfile:&SeccompProfile{Type:RuntimeDefault,LocalhostProfile:nil,},AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod calico-node-2p8hj_calico-system(4f334296-e8c0-427e-a090-d8fbd3582f26): ErrImagePull: failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount4008541472: write /var/lib/containerd/tmpmounts/containerd-mount4008541472/usr/bin/calico-node: no space left on device" logger="UnhandledError" Aug 13 01:25:47.450532 kubelet[2720]: E0813 01:25:47.450477 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"calico-node\" with ErrImagePull: \"failed to pull and unpack image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount4008541472: write /var/lib/containerd/tmpmounts/containerd-mount4008541472/usr/bin/calico-node: no space left on device\"" pod="calico-system/calico-node-2p8hj" podUID="4f334296-e8c0-427e-a090-d8fbd3582f26" Aug 13 01:25:48.168779 kubelet[2720]: E0813 01:25:48.168588 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"calico-node\" with ImagePullBackOff: \"Back-off pulling image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": ErrImagePull: failed to pull and unpack image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount4008541472: write /var/lib/containerd/tmpmounts/containerd-mount4008541472/usr/bin/calico-node: no space left on device\"" pod="calico-system/calico-node-2p8hj" podUID="4f334296-e8c0-427e-a090-d8fbd3582f26" Aug 13 01:25:52.203110 kubelet[2720]: I0813 01:25:52.203074 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:25:52.203110 kubelet[2720]: I0813 01:25:52.203113 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:25:52.205349 kubelet[2720]: I0813 01:25:52.205328 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:25:52.214242 kubelet[2720]: I0813 01:25:52.214223 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:25:52.214317 kubelet[2720]: I0813 01:25:52.214299 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/goldmane-768f4c5c69-mf54f","calico-apiserver/calico-apiserver-7566fdf5fb-vbzh6","calico-system/whisker-99dd966c4-29sst","calico-apiserver/calico-apiserver-7566fdf5fb-n55b2","kube-system/coredns-674b8bbfcf-b2ds8","kube-system/coredns-674b8bbfcf-jlpx4","calico-system/calico-kube-controllers-b57d47f84-lznkx","calico-system/calico-node-2p8hj","calico-system/csi-node-driver-8bzzs","tigera-operator/tigera-operator-747864d56d-vb4bt","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:25:52.218982 kubelet[2720]: I0813 01:25:52.218966 2720 eviction_manager.go:629] "Eviction manager: pod is evicted successfully" pod="calico-system/goldmane-768f4c5c69-mf54f" Aug 13 01:25:52.219082 kubelet[2720]: I0813 01:25:52.219052 2720 eviction_manager.go:208] "Eviction manager: pods evicted, waiting for pod to be cleaned up" pods=["calico-system/goldmane-768f4c5c69-mf54f"] Aug 13 01:25:52.240159 kubelet[2720]: I0813 01:25:52.240117 2720 kubelet.go:2405] "Pod admission denied" podUID="20b712df-2d16-454b-b194-365ba0d2697b" pod="calico-system/goldmane-768f4c5c69-22c8f" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:52.266731 kubelet[2720]: I0813 01:25:52.266677 2720 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"goldmane-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-goldmane-ca-bundle\") pod \"7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6\" (UID: \"7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6\") " Aug 13 01:25:52.267474 kubelet[2720]: I0813 01:25:52.267229 2720 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-goldmane-ca-bundle" (OuterVolumeSpecName: "goldmane-ca-bundle") pod "7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6" (UID: "7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6"). InnerVolumeSpecName "goldmane-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 13 01:25:52.268824 kubelet[2720]: I0813 01:25:52.266707 2720 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2299t\" (UniqueName: \"kubernetes.io/projected/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-kube-api-access-2299t\") pod \"7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6\" (UID: \"7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6\") " Aug 13 01:25:52.269267 kubelet[2720]: I0813 01:25:52.269221 2720 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-config\") pod \"7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6\" (UID: \"7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6\") " Aug 13 01:25:52.269311 kubelet[2720]: I0813 01:25:52.269269 2720 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"goldmane-key-pair\" (UniqueName: \"kubernetes.io/secret/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-goldmane-key-pair\") pod \"7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6\" (UID: \"7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6\") " Aug 13 01:25:52.269357 kubelet[2720]: I0813 01:25:52.269335 2720 reconciler_common.go:299] "Volume detached for volume \"goldmane-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-goldmane-ca-bundle\") on node \"172-234-25-215\" DevicePath \"\"" Aug 13 01:25:52.275904 kubelet[2720]: I0813 01:25:52.275873 2720 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-config" (OuterVolumeSpecName: "config") pod "7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6" (UID: "7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 13 01:25:52.277573 systemd[1]: var-lib-kubelet-pods-7a6f8cd3\x2d7ee5\x2d4677\x2d9b3c\x2d6e44d08257c6-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d2299t.mount: Deactivated successfully. Aug 13 01:25:52.277685 systemd[1]: var-lib-kubelet-pods-7a6f8cd3\x2d7ee5\x2d4677\x2d9b3c\x2d6e44d08257c6-volumes-kubernetes.io\x7esecret-goldmane\x2dkey\x2dpair.mount: Deactivated successfully. Aug 13 01:25:52.277887 kubelet[2720]: I0813 01:25:52.276854 2720 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-goldmane-key-pair" (OuterVolumeSpecName: "goldmane-key-pair") pod "7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6" (UID: "7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6"). InnerVolumeSpecName "goldmane-key-pair". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 13 01:25:52.277887 kubelet[2720]: I0813 01:25:52.277829 2720 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-kube-api-access-2299t" (OuterVolumeSpecName: "kube-api-access-2299t") pod "7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6" (UID: "7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6"). InnerVolumeSpecName "kube-api-access-2299t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 13 01:25:52.301742 kubelet[2720]: I0813 01:25:52.301053 2720 kubelet.go:2405] "Pod admission denied" podUID="581c6e7b-4f20-47ed-be5a-2521dba64894" pod="calico-system/goldmane-768f4c5c69-kd8m5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:52.329442 kubelet[2720]: I0813 01:25:52.329394 2720 kubelet.go:2405] "Pod admission denied" podUID="83c5015e-28c3-4064-865f-731b02d2c39b" pod="calico-system/goldmane-768f4c5c69-bdp9b" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:52.369688 kubelet[2720]: I0813 01:25:52.369649 2720 reconciler_common.go:299] "Volume detached for volume \"goldmane-key-pair\" (UniqueName: \"kubernetes.io/secret/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-goldmane-key-pair\") on node \"172-234-25-215\" DevicePath \"\"" Aug 13 01:25:52.369688 kubelet[2720]: I0813 01:25:52.369679 2720 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2299t\" (UniqueName: \"kubernetes.io/projected/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-kube-api-access-2299t\") on node \"172-234-25-215\" DevicePath \"\"" Aug 13 01:25:52.369688 kubelet[2720]: I0813 01:25:52.369689 2720 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a6f8cd3-7ee5-4677-9b3c-6e44d08257c6-config\") on node \"172-234-25-215\" DevicePath \"\"" Aug 13 01:25:52.374124 kubelet[2720]: I0813 01:25:52.374093 2720 kubelet.go:2405] "Pod admission denied" podUID="a8e275c1-3250-4c89-ba3b-40eaa7cb94d5" pod="calico-system/goldmane-768f4c5c69-7hxkj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:52.405023 kubelet[2720]: I0813 01:25:52.404961 2720 kubelet.go:2405] "Pod admission denied" podUID="2e288acf-dd67-4878-86de-86e1b96b51a2" pod="calico-system/goldmane-768f4c5c69-s7lnv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:52.441195 kubelet[2720]: I0813 01:25:52.441128 2720 kubelet.go:2405] "Pod admission denied" podUID="7c57dd97-5446-4202-a5ff-cfac6042251e" pod="calico-system/goldmane-768f4c5c69-tjbxn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:52.485336 kubelet[2720]: I0813 01:25:52.484557 2720 kubelet.go:2405] "Pod admission denied" podUID="1b83dbc1-b005-45f9-807d-f8d91b732a14" pod="calico-system/goldmane-768f4c5c69-mrg9h" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:52.507522 kubelet[2720]: I0813 01:25:52.507468 2720 kubelet.go:2405] "Pod admission denied" podUID="96b52d04-0ac7-4837-9126-5118bfa2239e" pod="calico-system/goldmane-768f4c5c69-j6rjm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:52.539226 kubelet[2720]: I0813 01:25:52.539185 2720 kubelet.go:2405] "Pod admission denied" podUID="5ad0640c-7a7d-43df-8bea-55530eeff2eb" pod="calico-system/goldmane-768f4c5c69-7rzl6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:52.691477 kubelet[2720]: I0813 01:25:52.691436 2720 kubelet.go:2405] "Pod admission denied" podUID="0ae06144-cc54-453f-8438-c55db2f8cd08" pod="calico-system/goldmane-768f4c5c69-ddpqz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:53.182259 systemd[1]: Removed slice kubepods-besteffort-pod7a6f8cd3_7ee5_4677_9b3c_6e44d08257c6.slice - libcontainer container kubepods-besteffort-pod7a6f8cd3_7ee5_4677_9b3c_6e44d08257c6.slice. Aug 13 01:25:53.219957 kubelet[2720]: I0813 01:25:53.219915 2720 eviction_manager.go:459] "Eviction manager: pods successfully cleaned up" pods=["calico-system/goldmane-768f4c5c69-mf54f"] Aug 13 01:25:53.233172 kubelet[2720]: I0813 01:25:53.233144 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:25:53.233403 kubelet[2720]: I0813 01:25:53.233187 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:25:53.235966 kubelet[2720]: I0813 01:25:53.235945 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:25:53.247536 kubelet[2720]: I0813 01:25:53.247511 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:25:53.247617 kubelet[2720]: I0813 01:25:53.247587 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/whisker-99dd966c4-29sst","calico-apiserver/calico-apiserver-7566fdf5fb-vbzh6","calico-apiserver/calico-apiserver-7566fdf5fb-n55b2","kube-system/coredns-674b8bbfcf-b2ds8","kube-system/coredns-674b8bbfcf-jlpx4","calico-system/calico-kube-controllers-b57d47f84-lznkx","calico-system/csi-node-driver-8bzzs","calico-system/calico-node-2p8hj","tigera-operator/tigera-operator-747864d56d-vb4bt","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:25:53.254528 kubelet[2720]: I0813 01:25:53.254494 2720 eviction_manager.go:629] "Eviction manager: pod is evicted successfully" pod="calico-system/whisker-99dd966c4-29sst" Aug 13 01:25:53.254976 kubelet[2720]: I0813 01:25:53.254944 2720 eviction_manager.go:208] "Eviction manager: pods evicted, waiting for pod to be cleaned up" pods=["calico-system/whisker-99dd966c4-29sst"] Aug 13 01:25:53.274961 kubelet[2720]: I0813 01:25:53.274924 2720 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5x8w\" (UniqueName: \"kubernetes.io/projected/308de2f1-eb6e-47d8-b49a-85db02a09390-kube-api-access-t5x8w\") pod \"308de2f1-eb6e-47d8-b49a-85db02a09390\" (UID: \"308de2f1-eb6e-47d8-b49a-85db02a09390\") " Aug 13 01:25:53.275083 kubelet[2720]: I0813 01:25:53.275064 2720 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"whisker-backend-key-pair\" (UniqueName: \"kubernetes.io/secret/308de2f1-eb6e-47d8-b49a-85db02a09390-whisker-backend-key-pair\") pod \"308de2f1-eb6e-47d8-b49a-85db02a09390\" (UID: \"308de2f1-eb6e-47d8-b49a-85db02a09390\") " Aug 13 01:25:53.275110 kubelet[2720]: I0813 01:25:53.275087 2720 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"whisker-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/308de2f1-eb6e-47d8-b49a-85db02a09390-whisker-ca-bundle\") pod \"308de2f1-eb6e-47d8-b49a-85db02a09390\" (UID: \"308de2f1-eb6e-47d8-b49a-85db02a09390\") " Aug 13 01:25:53.276512 kubelet[2720]: I0813 01:25:53.276432 2720 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/308de2f1-eb6e-47d8-b49a-85db02a09390-whisker-ca-bundle" (OuterVolumeSpecName: "whisker-ca-bundle") pod "308de2f1-eb6e-47d8-b49a-85db02a09390" (UID: "308de2f1-eb6e-47d8-b49a-85db02a09390"). InnerVolumeSpecName "whisker-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 13 01:25:53.287641 kubelet[2720]: I0813 01:25:53.287186 2720 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/308de2f1-eb6e-47d8-b49a-85db02a09390-kube-api-access-t5x8w" (OuterVolumeSpecName: "kube-api-access-t5x8w") pod "308de2f1-eb6e-47d8-b49a-85db02a09390" (UID: "308de2f1-eb6e-47d8-b49a-85db02a09390"). InnerVolumeSpecName "kube-api-access-t5x8w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 13 01:25:53.287656 systemd[1]: var-lib-kubelet-pods-308de2f1\x2deb6e\x2d47d8\x2db49a\x2d85db02a09390-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dt5x8w.mount: Deactivated successfully. Aug 13 01:25:53.287946 kubelet[2720]: I0813 01:25:53.287848 2720 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/308de2f1-eb6e-47d8-b49a-85db02a09390-whisker-backend-key-pair" (OuterVolumeSpecName: "whisker-backend-key-pair") pod "308de2f1-eb6e-47d8-b49a-85db02a09390" (UID: "308de2f1-eb6e-47d8-b49a-85db02a09390"). InnerVolumeSpecName "whisker-backend-key-pair". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 13 01:25:53.292386 systemd[1]: var-lib-kubelet-pods-308de2f1\x2deb6e\x2d47d8\x2db49a\x2d85db02a09390-volumes-kubernetes.io\x7esecret-whisker\x2dbackend\x2dkey\x2dpair.mount: Deactivated successfully. Aug 13 01:25:53.376330 kubelet[2720]: I0813 01:25:53.376292 2720 reconciler_common.go:299] "Volume detached for volume \"whisker-backend-key-pair\" (UniqueName: \"kubernetes.io/secret/308de2f1-eb6e-47d8-b49a-85db02a09390-whisker-backend-key-pair\") on node \"172-234-25-215\" DevicePath \"\"" Aug 13 01:25:53.376330 kubelet[2720]: I0813 01:25:53.376323 2720 reconciler_common.go:299] "Volume detached for volume \"whisker-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/308de2f1-eb6e-47d8-b49a-85db02a09390-whisker-ca-bundle\") on node \"172-234-25-215\" DevicePath \"\"" Aug 13 01:25:53.376330 kubelet[2720]: I0813 01:25:53.376335 2720 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5x8w\" (UniqueName: \"kubernetes.io/projected/308de2f1-eb6e-47d8-b49a-85db02a09390-kube-api-access-t5x8w\") on node \"172-234-25-215\" DevicePath \"\"" Aug 13 01:25:54.080619 systemd[1]: Removed slice kubepods-besteffort-pod308de2f1_eb6e_47d8_b49a_85db02a09390.slice - libcontainer container kubepods-besteffort-pod308de2f1_eb6e_47d8_b49a_85db02a09390.slice. Aug 13 01:25:54.255995 kubelet[2720]: I0813 01:25:54.255707 2720 eviction_manager.go:459] "Eviction manager: pods successfully cleaned up" pods=["calico-system/whisker-99dd966c4-29sst"] Aug 13 01:25:54.282121 kubelet[2720]: I0813 01:25:54.282084 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:25:54.282686 kubelet[2720]: I0813 01:25:54.282339 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:25:54.289129 kubelet[2720]: I0813 01:25:54.289078 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:25:54.308397 kubelet[2720]: I0813 01:25:54.308344 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:25:54.308627 kubelet[2720]: I0813 01:25:54.308549 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-apiserver/calico-apiserver-7566fdf5fb-n55b2","calico-apiserver/calico-apiserver-7566fdf5fb-vbzh6","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","calico-system/calico-node-2p8hj","calico-system/csi-node-driver-8bzzs","tigera-operator/tigera-operator-747864d56d-vb4bt","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:25:54.315470 kubelet[2720]: I0813 01:25:54.315433 2720 eviction_manager.go:629] "Eviction manager: pod is evicted successfully" pod="calico-apiserver/calico-apiserver-7566fdf5fb-n55b2" Aug 13 01:25:54.316251 kubelet[2720]: I0813 01:25:54.316229 2720 eviction_manager.go:208] "Eviction manager: pods evicted, waiting for pod to be cleaned up" pods=["calico-apiserver/calico-apiserver-7566fdf5fb-n55b2"] Aug 13 01:25:54.382830 kubelet[2720]: I0813 01:25:54.382018 2720 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"calico-apiserver-certs\" (UniqueName: \"kubernetes.io/secret/bac4ab4b-8a28-40be-9dfc-ab98956ae605-calico-apiserver-certs\") pod \"bac4ab4b-8a28-40be-9dfc-ab98956ae605\" (UID: \"bac4ab4b-8a28-40be-9dfc-ab98956ae605\") " Aug 13 01:25:54.382830 kubelet[2720]: I0813 01:25:54.382066 2720 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtd9k\" (UniqueName: \"kubernetes.io/projected/bac4ab4b-8a28-40be-9dfc-ab98956ae605-kube-api-access-wtd9k\") pod \"bac4ab4b-8a28-40be-9dfc-ab98956ae605\" (UID: \"bac4ab4b-8a28-40be-9dfc-ab98956ae605\") " Aug 13 01:25:54.387547 kubelet[2720]: I0813 01:25:54.387305 2720 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bac4ab4b-8a28-40be-9dfc-ab98956ae605-kube-api-access-wtd9k" (OuterVolumeSpecName: "kube-api-access-wtd9k") pod "bac4ab4b-8a28-40be-9dfc-ab98956ae605" (UID: "bac4ab4b-8a28-40be-9dfc-ab98956ae605"). InnerVolumeSpecName "kube-api-access-wtd9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 13 01:25:54.387444 systemd[1]: var-lib-kubelet-pods-bac4ab4b\x2d8a28\x2d40be\x2d9dfc\x2dab98956ae605-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dwtd9k.mount: Deactivated successfully. Aug 13 01:25:54.390629 kubelet[2720]: I0813 01:25:54.390578 2720 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac4ab4b-8a28-40be-9dfc-ab98956ae605-calico-apiserver-certs" (OuterVolumeSpecName: "calico-apiserver-certs") pod "bac4ab4b-8a28-40be-9dfc-ab98956ae605" (UID: "bac4ab4b-8a28-40be-9dfc-ab98956ae605"). InnerVolumeSpecName "calico-apiserver-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 13 01:25:54.390897 systemd[1]: var-lib-kubelet-pods-bac4ab4b\x2d8a28\x2d40be\x2d9dfc\x2dab98956ae605-volumes-kubernetes.io\x7esecret-calico\x2dapiserver\x2dcerts.mount: Deactivated successfully. Aug 13 01:25:54.483096 kubelet[2720]: I0813 01:25:54.482871 2720 reconciler_common.go:299] "Volume detached for volume \"calico-apiserver-certs\" (UniqueName: \"kubernetes.io/secret/bac4ab4b-8a28-40be-9dfc-ab98956ae605-calico-apiserver-certs\") on node \"172-234-25-215\" DevicePath \"\"" Aug 13 01:25:54.483096 kubelet[2720]: I0813 01:25:54.483087 2720 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtd9k\" (UniqueName: \"kubernetes.io/projected/bac4ab4b-8a28-40be-9dfc-ab98956ae605-kube-api-access-wtd9k\") on node \"172-234-25-215\" DevicePath \"\"" Aug 13 01:25:55.184324 systemd[1]: Removed slice kubepods-besteffort-podbac4ab4b_8a28_40be_9dfc_ab98956ae605.slice - libcontainer container kubepods-besteffort-podbac4ab4b_8a28_40be_9dfc_ab98956ae605.slice. Aug 13 01:25:55.316483 kubelet[2720]: I0813 01:25:55.316439 2720 eviction_manager.go:459] "Eviction manager: pods successfully cleaned up" pods=["calico-apiserver/calico-apiserver-7566fdf5fb-n55b2"] Aug 13 01:25:55.326135 kubelet[2720]: I0813 01:25:55.326113 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:25:55.326194 kubelet[2720]: I0813 01:25:55.326153 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:25:55.329186 kubelet[2720]: I0813 01:25:55.329122 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:25:55.341067 kubelet[2720]: I0813 01:25:55.341001 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:25:55.341337 kubelet[2720]: I0813 01:25:55.341228 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-apiserver/calico-apiserver-7566fdf5fb-vbzh6","kube-system/coredns-674b8bbfcf-b2ds8","kube-system/coredns-674b8bbfcf-jlpx4","calico-system/calico-kube-controllers-b57d47f84-lznkx","calico-system/csi-node-driver-8bzzs","calico-system/calico-node-2p8hj","tigera-operator/tigera-operator-747864d56d-vb4bt","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:25:55.347359 kubelet[2720]: I0813 01:25:55.347325 2720 eviction_manager.go:629] "Eviction manager: pod is evicted successfully" pod="calico-apiserver/calico-apiserver-7566fdf5fb-vbzh6" Aug 13 01:25:55.348017 kubelet[2720]: I0813 01:25:55.347342 2720 eviction_manager.go:208] "Eviction manager: pods evicted, waiting for pod to be cleaned up" pods=["calico-apiserver/calico-apiserver-7566fdf5fb-vbzh6"] Aug 13 01:25:55.390232 kubelet[2720]: I0813 01:25:55.390180 2720 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"calico-apiserver-certs\" (UniqueName: \"kubernetes.io/secret/045c69cc-698d-4c00-8a26-dcea7fba5a2e-calico-apiserver-certs\") pod \"045c69cc-698d-4c00-8a26-dcea7fba5a2e\" (UID: \"045c69cc-698d-4c00-8a26-dcea7fba5a2e\") " Aug 13 01:25:55.390395 kubelet[2720]: I0813 01:25:55.390234 2720 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncbl9\" (UniqueName: \"kubernetes.io/projected/045c69cc-698d-4c00-8a26-dcea7fba5a2e-kube-api-access-ncbl9\") pod \"045c69cc-698d-4c00-8a26-dcea7fba5a2e\" (UID: \"045c69cc-698d-4c00-8a26-dcea7fba5a2e\") " Aug 13 01:25:55.395244 kubelet[2720]: I0813 01:25:55.395194 2720 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/045c69cc-698d-4c00-8a26-dcea7fba5a2e-calico-apiserver-certs" (OuterVolumeSpecName: "calico-apiserver-certs") pod "045c69cc-698d-4c00-8a26-dcea7fba5a2e" (UID: "045c69cc-698d-4c00-8a26-dcea7fba5a2e"). InnerVolumeSpecName "calico-apiserver-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 13 01:25:55.395477 systemd[1]: var-lib-kubelet-pods-045c69cc\x2d698d\x2d4c00\x2d8a26\x2ddcea7fba5a2e-volumes-kubernetes.io\x7esecret-calico\x2dapiserver\x2dcerts.mount: Deactivated successfully. Aug 13 01:25:55.398364 kubelet[2720]: I0813 01:25:55.398308 2720 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/045c69cc-698d-4c00-8a26-dcea7fba5a2e-kube-api-access-ncbl9" (OuterVolumeSpecName: "kube-api-access-ncbl9") pod "045c69cc-698d-4c00-8a26-dcea7fba5a2e" (UID: "045c69cc-698d-4c00-8a26-dcea7fba5a2e"). InnerVolumeSpecName "kube-api-access-ncbl9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 13 01:25:55.399011 systemd[1]: var-lib-kubelet-pods-045c69cc\x2d698d\x2d4c00\x2d8a26\x2ddcea7fba5a2e-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2dncbl9.mount: Deactivated successfully. Aug 13 01:25:55.490775 kubelet[2720]: I0813 01:25:55.490736 2720 reconciler_common.go:299] "Volume detached for volume \"calico-apiserver-certs\" (UniqueName: \"kubernetes.io/secret/045c69cc-698d-4c00-8a26-dcea7fba5a2e-calico-apiserver-certs\") on node \"172-234-25-215\" DevicePath \"\"" Aug 13 01:25:55.490775 kubelet[2720]: I0813 01:25:55.490764 2720 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncbl9\" (UniqueName: \"kubernetes.io/projected/045c69cc-698d-4c00-8a26-dcea7fba5a2e-kube-api-access-ncbl9\") on node \"172-234-25-215\" DevicePath \"\"" Aug 13 01:25:56.080102 systemd[1]: Removed slice kubepods-besteffort-pod045c69cc_698d_4c00_8a26_dcea7fba5a2e.slice - libcontainer container kubepods-besteffort-pod045c69cc_698d_4c00_8a26_dcea7fba5a2e.slice. Aug 13 01:25:56.347864 kubelet[2720]: I0813 01:25:56.347754 2720 eviction_manager.go:459] "Eviction manager: pods successfully cleaned up" pods=["calico-apiserver/calico-apiserver-7566fdf5fb-vbzh6"] Aug 13 01:25:56.356447 kubelet[2720]: I0813 01:25:56.356429 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:25:56.356514 kubelet[2720]: I0813 01:25:56.356465 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:25:56.358104 kubelet[2720]: I0813 01:25:56.358073 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:25:56.368415 kubelet[2720]: I0813 01:25:56.368398 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:25:56.368480 kubelet[2720]: I0813 01:25:56.368455 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-b2ds8","kube-system/coredns-674b8bbfcf-jlpx4","calico-system/calico-kube-controllers-b57d47f84-lznkx","calico-system/csi-node-driver-8bzzs","calico-system/calico-node-2p8hj","tigera-operator/tigera-operator-747864d56d-vb4bt","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:25:56.368480 kubelet[2720]: E0813 01:25:56.368477 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:25:56.368698 kubelet[2720]: E0813 01:25:56.368487 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:25:56.368698 kubelet[2720]: E0813 01:25:56.368493 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:25:56.368698 kubelet[2720]: E0813 01:25:56.368498 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:25:56.368698 kubelet[2720]: E0813 01:25:56.368503 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:25:56.370175 containerd[1567]: time="2025-08-13T01:25:56.370136256Z" level=info msg="StopContainer for \"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\" with timeout 2 (s)" Aug 13 01:25:56.370651 containerd[1567]: time="2025-08-13T01:25:56.370428926Z" level=info msg="Stop container \"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\" with signal terminated" Aug 13 01:25:56.393370 systemd[1]: cri-containerd-86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382.scope: Deactivated successfully. Aug 13 01:25:56.394084 systemd[1]: cri-containerd-86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382.scope: Consumed 3.477s CPU time, 86.9M memory peak. Aug 13 01:25:56.395351 containerd[1567]: time="2025-08-13T01:25:56.395327152Z" level=info msg="received exit event container_id:\"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\" id:\"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\" pid:3046 exited_at:{seconds:1755048356 nanos:395017732}" Aug 13 01:25:56.395480 containerd[1567]: time="2025-08-13T01:25:56.395353802Z" level=info msg="TaskExit event in podsandbox handler container_id:\"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\" id:\"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\" pid:3046 exited_at:{seconds:1755048356 nanos:395017732}" Aug 13 01:25:56.413952 systemd[1]: run-containerd-io.containerd.runtime.v2.task-k8s.io-86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382-rootfs.mount: Deactivated successfully. Aug 13 01:25:56.420410 containerd[1567]: time="2025-08-13T01:25:56.420379049Z" level=info msg="StopContainer for \"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\" returns successfully" Aug 13 01:25:56.420908 containerd[1567]: time="2025-08-13T01:25:56.420888939Z" level=info msg="StopPodSandbox for \"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\"" Aug 13 01:25:56.420964 containerd[1567]: time="2025-08-13T01:25:56.420935269Z" level=info msg="Container to stop \"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\" must be in running or unknown state, current state \"CONTAINER_EXITED\"" Aug 13 01:25:56.426613 systemd[1]: cri-containerd-19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec.scope: Deactivated successfully. Aug 13 01:25:56.429772 containerd[1567]: time="2025-08-13T01:25:56.429750742Z" level=info msg="TaskExit event in podsandbox handler container_id:\"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" id:\"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" pid:2880 exit_status:137 exited_at:{seconds:1755048356 nanos:429499201}" Aug 13 01:25:56.452014 systemd[1]: run-containerd-io.containerd.runtime.v2.task-k8s.io-19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec-rootfs.mount: Deactivated successfully. Aug 13 01:25:56.454551 containerd[1567]: time="2025-08-13T01:25:56.454531988Z" level=info msg="shim disconnected" id=19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec namespace=k8s.io Aug 13 01:25:56.455497 containerd[1567]: time="2025-08-13T01:25:56.455413438Z" level=warning msg="cleaning up after shim disconnected" id=19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec namespace=k8s.io Aug 13 01:25:56.455497 containerd[1567]: time="2025-08-13T01:25:56.455428628Z" level=info msg="cleaning up dead shim" namespace=k8s.io Aug 13 01:25:56.465976 containerd[1567]: time="2025-08-13T01:25:56.465943371Z" level=error msg="Failed to handle event container_id:\"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" id:\"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" pid:2880 exit_status:137 exited_at:{seconds:1755048356 nanos:429499201} for 19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec" error="failed to handle container TaskExit event: failed to stop sandbox: failed to delete task: ttrpc: closed" Aug 13 01:25:56.467620 containerd[1567]: time="2025-08-13T01:25:56.466215071Z" level=info msg="received exit event sandbox_id:\"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" exit_status:137 exited_at:{seconds:1755048356 nanos:429499201}" Aug 13 01:25:56.467882 containerd[1567]: time="2025-08-13T01:25:56.467777872Z" level=info msg="TearDown network for sandbox \"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" successfully" Aug 13 01:25:56.467882 containerd[1567]: time="2025-08-13T01:25:56.467797072Z" level=info msg="StopPodSandbox for \"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" returns successfully" Aug 13 01:25:56.467989 systemd[1]: run-containerd-io.containerd.grpc.v1.cri-sandboxes-19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec-shm.mount: Deactivated successfully. Aug 13 01:25:56.473130 kubelet[2720]: I0813 01:25:56.473112 2720 eviction_manager.go:629] "Eviction manager: pod is evicted successfully" pod="tigera-operator/tigera-operator-747864d56d-vb4bt" Aug 13 01:25:56.473279 kubelet[2720]: I0813 01:25:56.473247 2720 eviction_manager.go:208] "Eviction manager: pods evicted, waiting for pod to be cleaned up" pods=["tigera-operator/tigera-operator-747864d56d-vb4bt"] Aug 13 01:25:56.490019 kubelet[2720]: I0813 01:25:56.489990 2720 kubelet.go:2405] "Pod admission denied" podUID="fb01ab40-08d2-4f50-8343-fce96ac87307" pod="tigera-operator/tigera-operator-747864d56d-vcxh4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:56.496406 kubelet[2720]: I0813 01:25:56.496356 2720 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"var-lib-calico\" (UniqueName: \"kubernetes.io/host-path/4090dbb9-5cbe-4af6-993c-ea44b6e69aaa-var-lib-calico\") pod \"4090dbb9-5cbe-4af6-993c-ea44b6e69aaa\" (UID: \"4090dbb9-5cbe-4af6-993c-ea44b6e69aaa\") " Aug 13 01:25:56.496529 kubelet[2720]: I0813 01:25:56.496513 2720 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/4090dbb9-5cbe-4af6-993c-ea44b6e69aaa-var-lib-calico" (OuterVolumeSpecName: "var-lib-calico") pod "4090dbb9-5cbe-4af6-993c-ea44b6e69aaa" (UID: "4090dbb9-5cbe-4af6-993c-ea44b6e69aaa"). InnerVolumeSpecName "var-lib-calico". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Aug 13 01:25:56.497040 kubelet[2720]: I0813 01:25:56.496578 2720 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xxjx\" (UniqueName: \"kubernetes.io/projected/4090dbb9-5cbe-4af6-993c-ea44b6e69aaa-kube-api-access-8xxjx\") pod \"4090dbb9-5cbe-4af6-993c-ea44b6e69aaa\" (UID: \"4090dbb9-5cbe-4af6-993c-ea44b6e69aaa\") " Aug 13 01:25:56.497197 kubelet[2720]: I0813 01:25:56.497142 2720 reconciler_common.go:299] "Volume detached for volume \"var-lib-calico\" (UniqueName: \"kubernetes.io/host-path/4090dbb9-5cbe-4af6-993c-ea44b6e69aaa-var-lib-calico\") on node \"172-234-25-215\" DevicePath \"\"" Aug 13 01:25:56.503269 kubelet[2720]: I0813 01:25:56.503231 2720 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4090dbb9-5cbe-4af6-993c-ea44b6e69aaa-kube-api-access-8xxjx" (OuterVolumeSpecName: "kube-api-access-8xxjx") pod "4090dbb9-5cbe-4af6-993c-ea44b6e69aaa" (UID: "4090dbb9-5cbe-4af6-993c-ea44b6e69aaa"). InnerVolumeSpecName "kube-api-access-8xxjx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 13 01:25:56.504159 systemd[1]: var-lib-kubelet-pods-4090dbb9\x2d5cbe\x2d4af6\x2d993c\x2dea44b6e69aaa-volumes-kubernetes.io\x7eprojected-kube\x2dapi\x2daccess\x2d8xxjx.mount: Deactivated successfully. Aug 13 01:25:56.518279 kubelet[2720]: I0813 01:25:56.518247 2720 kubelet.go:2405] "Pod admission denied" podUID="f784bd58-e5e9-443c-832e-320599657c37" pod="tigera-operator/tigera-operator-747864d56d-xbjwn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:56.537507 kubelet[2720]: I0813 01:25:56.537455 2720 kubelet.go:2405] "Pod admission denied" podUID="003b65dc-a25c-4001-a34d-a0b722ce243a" pod="tigera-operator/tigera-operator-747864d56d-czkhp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:56.560786 kubelet[2720]: I0813 01:25:56.560753 2720 kubelet.go:2405] "Pod admission denied" podUID="31bc9cac-afb5-472b-8e42-590c51b29579" pod="tigera-operator/tigera-operator-747864d56d-gndnk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:56.576895 kubelet[2720]: I0813 01:25:56.576174 2720 kubelet.go:2405] "Pod admission denied" podUID="39085eb7-00c5-4570-b107-3f5deb82b464" pod="tigera-operator/tigera-operator-747864d56d-lh75f" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:56.595617 kubelet[2720]: I0813 01:25:56.595432 2720 kubelet.go:2405] "Pod admission denied" podUID="6e1ab4aa-89d3-4af7-a123-5bdc569bbd0d" pod="tigera-operator/tigera-operator-747864d56d-5px5w" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:56.600580 kubelet[2720]: I0813 01:25:56.599309 2720 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xxjx\" (UniqueName: \"kubernetes.io/projected/4090dbb9-5cbe-4af6-993c-ea44b6e69aaa-kube-api-access-8xxjx\") on node \"172-234-25-215\" DevicePath \"\"" Aug 13 01:25:56.617733 kubelet[2720]: I0813 01:25:56.617681 2720 kubelet.go:2405] "Pod admission denied" podUID="6cab5882-dd9c-4014-b35b-b4f025540f9f" pod="tigera-operator/tigera-operator-747864d56d-frfzr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:56.637849 kubelet[2720]: I0813 01:25:56.637788 2720 kubelet.go:2405] "Pod admission denied" podUID="419036cf-7f7e-4be4-bfb7-7ec4ed322d6d" pod="tigera-operator/tigera-operator-747864d56d-5w98s" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:56.661818 kubelet[2720]: I0813 01:25:56.661775 2720 kubelet.go:2405] "Pod admission denied" podUID="3118bd2a-2073-48b3-a863-8f6c51053cd3" pod="tigera-operator/tigera-operator-747864d56d-fbkrw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:56.739197 kubelet[2720]: I0813 01:25:56.739163 2720 kubelet.go:2405] "Pod admission denied" podUID="6ae1cd54-a798-4ae7-9059-40d2c2390e89" pod="tigera-operator/tigera-operator-747864d56d-dvff4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:56.891836 kubelet[2720]: I0813 01:25:56.891376 2720 kubelet.go:2405] "Pod admission denied" podUID="6e24e128-b55a-4e59-9af3-04facfce8085" pod="tigera-operator/tigera-operator-747864d56d-lb2m7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:57.043210 kubelet[2720]: I0813 01:25:57.043158 2720 kubelet.go:2405] "Pod admission denied" podUID="bbc201f2-53fc-4b72-b2de-913fc2b87e34" pod="tigera-operator/tigera-operator-747864d56d-lmpzf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:57.073946 kubelet[2720]: E0813 01:25:57.073910 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:25:57.075465 containerd[1567]: time="2025-08-13T01:25:57.075055499Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:25:57.075465 containerd[1567]: time="2025-08-13T01:25:57.075270589Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:25:57.155046 containerd[1567]: time="2025-08-13T01:25:57.154946631Z" level=error msg="Failed to destroy network for sandbox \"69a420b9578ff5bea7ef1340ab1b365b9bc338623fcb8adec971781b7b398cc2\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:57.157159 containerd[1567]: time="2025-08-13T01:25:57.157114081Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"69a420b9578ff5bea7ef1340ab1b365b9bc338623fcb8adec971781b7b398cc2\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:57.157350 kubelet[2720]: E0813 01:25:57.157271 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"69a420b9578ff5bea7ef1340ab1b365b9bc338623fcb8adec971781b7b398cc2\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:57.157350 kubelet[2720]: E0813 01:25:57.157323 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"69a420b9578ff5bea7ef1340ab1b365b9bc338623fcb8adec971781b7b398cc2\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:25:57.157350 kubelet[2720]: E0813 01:25:57.157342 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"69a420b9578ff5bea7ef1340ab1b365b9bc338623fcb8adec971781b7b398cc2\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:25:57.157449 kubelet[2720]: E0813 01:25:57.157390 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"69a420b9578ff5bea7ef1340ab1b365b9bc338623fcb8adec971781b7b398cc2\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:25:57.158896 containerd[1567]: time="2025-08-13T01:25:57.158857852Z" level=error msg="Failed to destroy network for sandbox \"1dc59eec918f76f99acd03e84ee8d3d904504b53135e9f44ed608b3dc326a55d\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:57.159711 containerd[1567]: time="2025-08-13T01:25:57.159637532Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"1dc59eec918f76f99acd03e84ee8d3d904504b53135e9f44ed608b3dc326a55d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:57.160127 kubelet[2720]: E0813 01:25:57.160103 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1dc59eec918f76f99acd03e84ee8d3d904504b53135e9f44ed608b3dc326a55d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:57.160177 kubelet[2720]: E0813 01:25:57.160137 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1dc59eec918f76f99acd03e84ee8d3d904504b53135e9f44ed608b3dc326a55d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:25:57.160177 kubelet[2720]: E0813 01:25:57.160154 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1dc59eec918f76f99acd03e84ee8d3d904504b53135e9f44ed608b3dc326a55d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:25:57.160225 kubelet[2720]: E0813 01:25:57.160186 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"1dc59eec918f76f99acd03e84ee8d3d904504b53135e9f44ed608b3dc326a55d\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:25:57.185911 kubelet[2720]: I0813 01:25:57.185878 2720 scope.go:117] "RemoveContainer" containerID="86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382" Aug 13 01:25:57.191322 containerd[1567]: time="2025-08-13T01:25:57.191164470Z" level=info msg="RemoveContainer for \"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\"" Aug 13 01:25:57.195875 systemd[1]: Removed slice kubepods-besteffort-pod4090dbb9_5cbe_4af6_993c_ea44b6e69aaa.slice - libcontainer container kubepods-besteffort-pod4090dbb9_5cbe_4af6_993c_ea44b6e69aaa.slice. Aug 13 01:25:57.196403 systemd[1]: kubepods-besteffort-pod4090dbb9_5cbe_4af6_993c_ea44b6e69aaa.slice: Consumed 3.507s CPU time, 87.2M memory peak. Aug 13 01:25:57.201364 kubelet[2720]: I0813 01:25:57.201072 2720 kubelet.go:2405] "Pod admission denied" podUID="4b763cac-e512-4f44-a0f7-0cf93ac1c0eb" pod="tigera-operator/tigera-operator-747864d56d-qvbtp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:57.208036 containerd[1567]: time="2025-08-13T01:25:57.207992815Z" level=info msg="RemoveContainer for \"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\" returns successfully" Aug 13 01:25:57.208301 kubelet[2720]: I0813 01:25:57.208282 2720 scope.go:117] "RemoveContainer" containerID="86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382" Aug 13 01:25:57.208642 containerd[1567]: time="2025-08-13T01:25:57.208618536Z" level=error msg="ContainerStatus for \"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\" failed" error="rpc error: code = NotFound desc = an error occurred when try to find container \"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\": not found" Aug 13 01:25:57.208942 kubelet[2720]: E0813 01:25:57.208842 2720 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = an error occurred when try to find container \"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\": not found" containerID="86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382" Aug 13 01:25:57.208942 kubelet[2720]: I0813 01:25:57.208866 2720 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"containerd","ID":"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382"} err="failed to get container status \"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\": rpc error: code = NotFound desc = an error occurred when try to find container \"86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382\": not found" Aug 13 01:25:57.342372 kubelet[2720]: I0813 01:25:57.342334 2720 kubelet.go:2405] "Pod admission denied" podUID="56111987-e779-4885-a6b3-f53e7a4c9ac2" pod="tigera-operator/tigera-operator-747864d56d-cnclh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:57.413417 systemd[1]: run-netns-cni\x2dd27e7c3d\x2d22b5\x2d07dd\x2dba65\x2d87c72f0f0e2f.mount: Deactivated successfully. Aug 13 01:25:57.413533 systemd[1]: run-netns-cni\x2d8fdd2b6d\x2dcaad\x2d1d26\x2d3049\x2d6ca2877e41fa.mount: Deactivated successfully. Aug 13 01:25:57.443292 kubelet[2720]: I0813 01:25:57.442792 2720 kubelet.go:2405] "Pod admission denied" podUID="072993b8-893d-47ca-87f0-54fb17e61adc" pod="tigera-operator/tigera-operator-747864d56d-4gw8v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:57.473477 kubelet[2720]: I0813 01:25:57.473407 2720 eviction_manager.go:459] "Eviction manager: pods successfully cleaned up" pods=["tigera-operator/tigera-operator-747864d56d-vb4bt"] Aug 13 01:25:57.489242 kubelet[2720]: I0813 01:25:57.489203 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:25:57.489242 kubelet[2720]: I0813 01:25:57.489238 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:25:57.491948 containerd[1567]: time="2025-08-13T01:25:57.491852972Z" level=info msg="StopPodSandbox for \"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\"" Aug 13 01:25:57.492488 containerd[1567]: time="2025-08-13T01:25:57.492151792Z" level=info msg="TearDown network for sandbox \"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" successfully" Aug 13 01:25:57.492488 containerd[1567]: time="2025-08-13T01:25:57.492165722Z" level=info msg="StopPodSandbox for \"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" returns successfully" Aug 13 01:25:57.493205 containerd[1567]: time="2025-08-13T01:25:57.493184683Z" level=info msg="RemovePodSandbox for \"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\"" Aug 13 01:25:57.493561 containerd[1567]: time="2025-08-13T01:25:57.493363013Z" level=info msg="Forcibly stopping sandbox \"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\"" Aug 13 01:25:57.493561 containerd[1567]: time="2025-08-13T01:25:57.493430063Z" level=info msg="TearDown network for sandbox \"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" successfully" Aug 13 01:25:57.495270 containerd[1567]: time="2025-08-13T01:25:57.495240833Z" level=info msg="Ensure that sandbox 19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec in task-service has been cleanup successfully" Aug 13 01:25:57.497149 containerd[1567]: time="2025-08-13T01:25:57.497117034Z" level=info msg="RemovePodSandbox \"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" returns successfully" Aug 13 01:25:57.497614 kubelet[2720]: I0813 01:25:57.497596 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:25:57.507656 kubelet[2720]: I0813 01:25:57.507618 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:25:57.507756 kubelet[2720]: I0813 01:25:57.507685 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-b2ds8","kube-system/coredns-674b8bbfcf-jlpx4","calico-system/calico-node-2p8hj","calico-system/csi-node-driver-8bzzs","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:25:57.507756 kubelet[2720]: E0813 01:25:57.507727 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:25:57.507756 kubelet[2720]: E0813 01:25:57.507740 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:25:57.507756 kubelet[2720]: E0813 01:25:57.507747 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:25:57.507756 kubelet[2720]: E0813 01:25:57.507752 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:25:57.507756 kubelet[2720]: E0813 01:25:57.507759 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:25:57.507932 kubelet[2720]: E0813 01:25:57.507768 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:25:57.507932 kubelet[2720]: E0813 01:25:57.507776 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:25:57.507932 kubelet[2720]: E0813 01:25:57.507783 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:25:57.507932 kubelet[2720]: E0813 01:25:57.507792 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:25:57.507932 kubelet[2720]: E0813 01:25:57.507799 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:25:57.507932 kubelet[2720]: I0813 01:25:57.507808 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:25:57.513030 containerd[1567]: time="2025-08-13T01:25:57.512991008Z" level=info msg="TaskExit event in podsandbox handler container_id:\"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" id:\"19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec\" pid:2880 exit_status:137 exited_at:{seconds:1755048356 nanos:429499201}" Aug 13 01:25:57.592690 kubelet[2720]: I0813 01:25:57.592661 2720 kubelet.go:2405] "Pod admission denied" podUID="f28b6f3b-f4f7-44c4-b8cc-4b3216fe0cb0" pod="tigera-operator/tigera-operator-747864d56d-vnmmh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:57.742688 kubelet[2720]: I0813 01:25:57.742632 2720 kubelet.go:2405] "Pod admission denied" podUID="0c86ac1d-cedf-4a15-8e9a-b1da871e9d4c" pod="tigera-operator/tigera-operator-747864d56d-p6hcp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:57.892738 kubelet[2720]: I0813 01:25:57.892671 2720 kubelet.go:2405] "Pod admission denied" podUID="339cad6e-d895-4055-9bb2-0d0ff485dd2e" pod="tigera-operator/tigera-operator-747864d56d-4ngjk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:58.043117 kubelet[2720]: I0813 01:25:58.042822 2720 kubelet.go:2405] "Pod admission denied" podUID="8e779645-e2ba-4781-83aa-0f700543a567" pod="tigera-operator/tigera-operator-747864d56d-nmqcm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:58.193659 kubelet[2720]: I0813 01:25:58.193619 2720 kubelet.go:2405] "Pod admission denied" podUID="44edfaf3-faf1-44f3-ad81-2ea84b827808" pod="tigera-operator/tigera-operator-747864d56d-pw9wb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:58.346338 kubelet[2720]: I0813 01:25:58.346210 2720 kubelet.go:2405] "Pod admission denied" podUID="805fd34f-ddc8-4ea8-8583-354a0a568faf" pod="tigera-operator/tigera-operator-747864d56d-wqnh5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:58.494605 kubelet[2720]: I0813 01:25:58.494546 2720 kubelet.go:2405] "Pod admission denied" podUID="6da2df88-2dbd-4d8a-bff2-f507333b913e" pod="tigera-operator/tigera-operator-747864d56d-mjsr8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:58.742772 kubelet[2720]: I0813 01:25:58.742708 2720 kubelet.go:2405] "Pod admission denied" podUID="b70b22af-5f44-45d0-bcf2-04ecdbcc8204" pod="tigera-operator/tigera-operator-747864d56d-xz7qh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:58.890460 kubelet[2720]: I0813 01:25:58.890396 2720 kubelet.go:2405] "Pod admission denied" podUID="4f5fbaa4-ffd4-417b-8f31-1b04e69a4b64" pod="tigera-operator/tigera-operator-747864d56d-fnrlp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:59.044262 kubelet[2720]: I0813 01:25:59.041902 2720 kubelet.go:2405] "Pod admission denied" podUID="cce2e436-bac9-4812-aa43-1830c49b2718" pod="tigera-operator/tigera-operator-747864d56d-jgpmv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:59.073612 containerd[1567]: time="2025-08-13T01:25:59.073554969Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:25:59.129449 containerd[1567]: time="2025-08-13T01:25:59.129403755Z" level=error msg="Failed to destroy network for sandbox \"c8d6407c57412185503250b10b4bbf24669325bb7637bca75119514882049f34\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:59.131951 systemd[1]: run-netns-cni\x2d88b794cd\x2d872e\x2d51f5\x2d10b0\x2d6b514a4041af.mount: Deactivated successfully. Aug 13 01:25:59.132183 containerd[1567]: time="2025-08-13T01:25:59.132063326Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c8d6407c57412185503250b10b4bbf24669325bb7637bca75119514882049f34\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:59.132425 kubelet[2720]: E0813 01:25:59.132384 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c8d6407c57412185503250b10b4bbf24669325bb7637bca75119514882049f34\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:25:59.132546 kubelet[2720]: E0813 01:25:59.132441 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c8d6407c57412185503250b10b4bbf24669325bb7637bca75119514882049f34\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:25:59.132546 kubelet[2720]: E0813 01:25:59.132465 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c8d6407c57412185503250b10b4bbf24669325bb7637bca75119514882049f34\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:25:59.132601 kubelet[2720]: E0813 01:25:59.132550 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c8d6407c57412185503250b10b4bbf24669325bb7637bca75119514882049f34\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:25:59.143277 kubelet[2720]: I0813 01:25:59.143248 2720 kubelet.go:2405] "Pod admission denied" podUID="7dc4255c-4802-4843-a971-a40a8c840cf2" pod="tigera-operator/tigera-operator-747864d56d-9pthq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:59.242684 kubelet[2720]: I0813 01:25:59.242640 2720 kubelet.go:2405] "Pod admission denied" podUID="d4c23ed3-f544-4814-bd1f-df1da3a2efac" pod="tigera-operator/tigera-operator-747864d56d-h4cn5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:59.443261 kubelet[2720]: I0813 01:25:59.443119 2720 kubelet.go:2405] "Pod admission denied" podUID="dbdcc768-822d-43f9-9bc4-3fa8b02b654e" pod="tigera-operator/tigera-operator-747864d56d-6fnj7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:59.542700 kubelet[2720]: I0813 01:25:59.542650 2720 kubelet.go:2405] "Pod admission denied" podUID="43b6b5b6-7d43-414f-9ff1-055d0e07dc05" pod="tigera-operator/tigera-operator-747864d56d-9d9kb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:59.592755 kubelet[2720]: I0813 01:25:59.592694 2720 kubelet.go:2405] "Pod admission denied" podUID="86e3a575-3f8a-4b96-ac90-e4c2933aa10f" pod="tigera-operator/tigera-operator-747864d56d-2lrvr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:59.692436 kubelet[2720]: I0813 01:25:59.692405 2720 kubelet.go:2405] "Pod admission denied" podUID="7a761390-5b3a-4149-b8df-9cc0193457d7" pod="tigera-operator/tigera-operator-747864d56d-rvtpn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:59.793861 kubelet[2720]: I0813 01:25:59.793813 2720 kubelet.go:2405] "Pod admission denied" podUID="c9ed6d63-ebcc-49e6-b9e5-b0fa13ff6f2d" pod="tigera-operator/tigera-operator-747864d56d-n6kfn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:59.894419 kubelet[2720]: I0813 01:25:59.894388 2720 kubelet.go:2405] "Pod admission denied" podUID="b3710549-0a8c-4369-9bf9-31f462777c2f" pod="tigera-operator/tigera-operator-747864d56d-6k7bt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:25:59.994263 kubelet[2720]: I0813 01:25:59.994067 2720 kubelet.go:2405] "Pod admission denied" podUID="ce85e16f-a9de-46e6-ae98-335667b468ae" pod="tigera-operator/tigera-operator-747864d56d-p6rxj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:00.042730 kubelet[2720]: I0813 01:26:00.042671 2720 kubelet.go:2405] "Pod admission denied" podUID="2873e07e-7193-494d-95bf-3230e0cc4b1b" pod="tigera-operator/tigera-operator-747864d56d-28n88" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:00.073603 kubelet[2720]: E0813 01:26:00.073160 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:26:00.074318 containerd[1567]: time="2025-08-13T01:26:00.074113789Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:26:00.081570 containerd[1567]: time="2025-08-13T01:26:00.078860861Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/node:v3.30.2\"" Aug 13 01:26:00.149339 kubelet[2720]: I0813 01:26:00.149299 2720 kubelet.go:2405] "Pod admission denied" podUID="56617596-98d6-466f-9ad2-faea8a03e255" pod="tigera-operator/tigera-operator-747864d56d-f5rgz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:00.168010 containerd[1567]: time="2025-08-13T01:26:00.167974748Z" level=error msg="Failed to destroy network for sandbox \"1319cab9f03e6740b651453195b159fec6c3d3eca4e840c60fa25e2ceb11e0ec\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:00.169762 containerd[1567]: time="2025-08-13T01:26:00.168996229Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"1319cab9f03e6740b651453195b159fec6c3d3eca4e840c60fa25e2ceb11e0ec\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:00.170389 systemd[1]: run-netns-cni\x2ded9c1cbe\x2d8f03\x2d537f\x2deda4\x2d9a4004332a5e.mount: Deactivated successfully. Aug 13 01:26:00.171010 kubelet[2720]: E0813 01:26:00.170983 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1319cab9f03e6740b651453195b159fec6c3d3eca4e840c60fa25e2ceb11e0ec\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:00.171174 kubelet[2720]: E0813 01:26:00.171117 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1319cab9f03e6740b651453195b159fec6c3d3eca4e840c60fa25e2ceb11e0ec\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:00.171267 kubelet[2720]: E0813 01:26:00.171249 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1319cab9f03e6740b651453195b159fec6c3d3eca4e840c60fa25e2ceb11e0ec\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:00.171451 kubelet[2720]: E0813 01:26:00.171381 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"1319cab9f03e6740b651453195b159fec6c3d3eca4e840c60fa25e2ceb11e0ec\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:26:00.243323 kubelet[2720]: I0813 01:26:00.243283 2720 kubelet.go:2405] "Pod admission denied" podUID="a4a9c8dc-3824-4005-a44c-dd4a3fb6d80a" pod="tigera-operator/tigera-operator-747864d56d-lbqdr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:00.340794 kubelet[2720]: I0813 01:26:00.340671 2720 kubelet.go:2405] "Pod admission denied" podUID="4c57d108-3483-4074-96bd-3b637b198316" pod="tigera-operator/tigera-operator-747864d56d-h7825" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:00.543578 kubelet[2720]: I0813 01:26:00.543228 2720 kubelet.go:2405] "Pod admission denied" podUID="e074255e-08c4-40c0-bf3d-5ba8719edaa8" pod="tigera-operator/tigera-operator-747864d56d-p2qfd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:00.652115 kubelet[2720]: I0813 01:26:00.651751 2720 kubelet.go:2405] "Pod admission denied" podUID="63bb7f37-4bb8-426e-9190-1ed8106b7c81" pod="tigera-operator/tigera-operator-747864d56d-gfmn5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:00.750458 kubelet[2720]: I0813 01:26:00.748921 2720 kubelet.go:2405] "Pod admission denied" podUID="0d69849b-e3be-4f90-9a8b-2634562af2f6" pod="tigera-operator/tigera-operator-747864d56d-rbrj6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:00.844854 kubelet[2720]: I0813 01:26:00.844743 2720 kubelet.go:2405] "Pod admission denied" podUID="318b6875-18f8-444f-9b19-642c0b4210e2" pod="tigera-operator/tigera-operator-747864d56d-xq6vh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:00.894206 kubelet[2720]: I0813 01:26:00.893985 2720 kubelet.go:2405] "Pod admission denied" podUID="c517aaee-1966-49b2-b35d-6947286476d0" pod="tigera-operator/tigera-operator-747864d56d-dbg6v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:00.999523 kubelet[2720]: I0813 01:26:00.999484 2720 kubelet.go:2405] "Pod admission denied" podUID="0a80650b-031f-47f7-a296-51f8a515804d" pod="tigera-operator/tigera-operator-747864d56d-hs8m9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:01.199297 kubelet[2720]: I0813 01:26:01.198689 2720 kubelet.go:2405] "Pod admission denied" podUID="15ff3fb2-74b6-466f-b0c5-0401be00fbc9" pod="tigera-operator/tigera-operator-747864d56d-jcb59" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:01.299177 kubelet[2720]: I0813 01:26:01.298994 2720 kubelet.go:2405] "Pod admission denied" podUID="ef0c7e23-f337-445e-ab75-bd4d63230962" pod="tigera-operator/tigera-operator-747864d56d-5g4t8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:01.398067 kubelet[2720]: I0813 01:26:01.397957 2720 kubelet.go:2405] "Pod admission denied" podUID="67858c56-81d3-4756-958f-ebdb2c23b34e" pod="tigera-operator/tigera-operator-747864d56d-4zxwh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:01.497342 kubelet[2720]: I0813 01:26:01.497305 2720 kubelet.go:2405] "Pod admission denied" podUID="41b72aea-b59a-44a8-b0f7-ffba5a9c8dcf" pod="tigera-operator/tigera-operator-747864d56d-6wq4l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:01.596941 kubelet[2720]: I0813 01:26:01.596583 2720 kubelet.go:2405] "Pod admission denied" podUID="e547de59-059e-4be4-8b6e-48106bce4a1e" pod="tigera-operator/tigera-operator-747864d56d-zmkkq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:01.697264 kubelet[2720]: I0813 01:26:01.697217 2720 kubelet.go:2405] "Pod admission denied" podUID="7b6de8b0-5097-41a4-92de-a727c9b9a8f6" pod="tigera-operator/tigera-operator-747864d56d-8p4xm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:01.798736 kubelet[2720]: I0813 01:26:01.798593 2720 kubelet.go:2405] "Pod admission denied" podUID="a5b63cfa-225e-42da-b064-edab11291e66" pod="tigera-operator/tigera-operator-747864d56d-stvs6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:01.901226 kubelet[2720]: I0813 01:26:01.901107 2720 kubelet.go:2405] "Pod admission denied" podUID="794bc092-6264-4f2f-af75-d4f5b6a82794" pod="tigera-operator/tigera-operator-747864d56d-6f57l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:01.947936 kubelet[2720]: I0813 01:26:01.947700 2720 kubelet.go:2405] "Pod admission denied" podUID="3f591658-6785-47bb-b1cf-165382df45c6" pod="tigera-operator/tigera-operator-747864d56d-9crgr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:02.051274 kubelet[2720]: I0813 01:26:02.051023 2720 kubelet.go:2405] "Pod admission denied" podUID="03a30854-0298-469d-a7a1-28f1302f2a21" pod="tigera-operator/tigera-operator-747864d56d-v9wwr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:02.076515 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount1533876103.mount: Deactivated successfully. Aug 13 01:26:02.080774 containerd[1567]: time="2025-08-13T01:26:02.080695299Z" level=error msg="PullImage \"ghcr.io/flatcar/calico/node:v3.30.2\" failed" error="rpc error: code = Unknown desc = failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1533876103: write /var/lib/containerd/tmpmounts/containerd-mount1533876103/usr/bin/calico-node: no space left on device" Aug 13 01:26:02.081744 containerd[1567]: time="2025-08-13T01:26:02.080755519Z" level=info msg="stop pulling image ghcr.io/flatcar/calico/node:v3.30.2: active requests=0, bytes read=158500163" Aug 13 01:26:02.081791 kubelet[2720]: E0813 01:26:02.081479 2720 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1533876103: write /var/lib/containerd/tmpmounts/containerd-mount1533876103/usr/bin/calico-node: no space left on device" image="ghcr.io/flatcar/calico/node:v3.30.2" Aug 13 01:26:02.082704 kubelet[2720]: E0813 01:26:02.081869 2720 kuberuntime_image.go:42] "Failed to pull image" err="failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1533876103: write /var/lib/containerd/tmpmounts/containerd-mount1533876103/usr/bin/calico-node: no space left on device" image="ghcr.io/flatcar/calico/node:v3.30.2" Aug 13 01:26:02.082784 kubelet[2720]: E0813 01:26:02.082181 2720 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:calico-node,Image:ghcr.io/flatcar/calico/node:v3.30.2,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:DATASTORE_TYPE,Value:kubernetes,ValueFrom:nil,},EnvVar{Name:WAIT_FOR_DATASTORE,Value:true,ValueFrom:nil,},EnvVar{Name:CLUSTER_TYPE,Value:k8s,operator,bgp,ValueFrom:nil,},EnvVar{Name:CALICO_DISABLE_FILE_LOGGING,Value:false,ValueFrom:nil,},EnvVar{Name:FELIX_DEFAULTENDPOINTTOHOSTACTION,Value:ACCEPT,ValueFrom:nil,},EnvVar{Name:FELIX_HEALTHENABLED,Value:true,ValueFrom:nil,},EnvVar{Name:FELIX_HEALTHPORT,Value:9099,ValueFrom:nil,},EnvVar{Name:NODENAME,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:spec.nodeName,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:FELIX_TYPHAK8SNAMESPACE,Value:calico-system,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHAK8SSERVICENAME,Value:calico-typha,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHACAFILE,Value:/etc/pki/tls/certs/tigera-ca-bundle.crt,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHACERTFILE,Value:/node-certs/tls.crt,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHAKEYFILE,Value:/node-certs/tls.key,ValueFrom:nil,},EnvVar{Name:NO_DEFAULT_POOLS,Value:true,ValueFrom:nil,},EnvVar{Name:FELIX_FLOWLOGSGOLDMANESERVER,Value:goldmane.calico-system.svc:7443,ValueFrom:nil,},EnvVar{Name:FELIX_FLOWLOGSFLUSHINTERVAL,Value:15,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHACN,Value:typha-server,ValueFrom:nil,},EnvVar{Name:CALICO_MANAGE_CNI,Value:true,ValueFrom:nil,},EnvVar{Name:CALICO_NETWORKING_BACKEND,Value:bird,ValueFrom:nil,},EnvVar{Name:IP,Value:autodetect,ValueFrom:nil,},EnvVar{Name:IP_AUTODETECTION_METHOD,Value:first-found,ValueFrom:nil,},EnvVar{Name:IP6,Value:none,ValueFrom:nil,},EnvVar{Name:FELIX_IPV6SUPPORT,Value:false,ValueFrom:nil,},EnvVar{Name:KUBERNETES_SERVICE_HOST,Value:10.96.0.1,ValueFrom:nil,},EnvVar{Name:KUBERNETES_SERVICE_PORT,Value:443,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:cni-log-dir,ReadOnly:false,MountPath:/var/log/calico/cni,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:cni-net-dir,ReadOnly:false,MountPath:/host/etc/cni/net.d,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:lib-modules,ReadOnly:true,MountPath:/lib/modules,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:node-certs,ReadOnly:true,MountPath:/node-certs,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:policysync,ReadOnly:false,MountPath:/var/run/nodeagent,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:tigera-ca-bundle,ReadOnly:true,MountPath:/etc/pki/tls/certs,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:tigera-ca-bundle,ReadOnly:true,MountPath:/etc/pki/tls/cert.pem,SubPath:ca-bundle.crt,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:var-lib-calico,ReadOnly:false,MountPath:/var/lib/calico,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:var-run-calico,ReadOnly:false,MountPath:/var/run/calico,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:xtables-lock,ReadOnly:false,MountPath:/run/xtables.lock,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-jl8ww,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/liveness,Port:{0 9099 },Host:localhost,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:10,PeriodSeconds:60,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:&ExecAction{Command:[/bin/calico-node -bird-ready -felix-ready],},HTTPGet:nil,TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:10,PeriodSeconds:30,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:&Lifecycle{PostStart:nil,PreStop:&LifecycleHandler{Exec:&ExecAction{Command:[/bin/calico-node -shutdown],},HTTPGet:nil,TCPSocket:nil,Sleep:nil,},StopSignal:nil,},TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:*true,SELinuxOptions:nil,RunAsUser:*0,RunAsNonRoot:*false,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*true,RunAsGroup:*0,ProcMount:nil,WindowsOptions:nil,SeccompProfile:&SeccompProfile{Type:RuntimeDefault,LocalhostProfile:nil,},AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod calico-node-2p8hj_calico-system(4f334296-e8c0-427e-a090-d8fbd3582f26): ErrImagePull: failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1533876103: write /var/lib/containerd/tmpmounts/containerd-mount1533876103/usr/bin/calico-node: no space left on device" logger="UnhandledError" Aug 13 01:26:02.084245 kubelet[2720]: E0813 01:26:02.084210 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"calico-node\" with ErrImagePull: \"failed to pull and unpack image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1533876103: write /var/lib/containerd/tmpmounts/containerd-mount1533876103/usr/bin/calico-node: no space left on device\"" pod="calico-system/calico-node-2p8hj" podUID="4f334296-e8c0-427e-a090-d8fbd3582f26" Aug 13 01:26:02.147698 kubelet[2720]: I0813 01:26:02.147618 2720 kubelet.go:2405] "Pod admission denied" podUID="79def739-c79f-4dec-a91c-150d81179cfc" pod="tigera-operator/tigera-operator-747864d56d-vb22z" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:02.256149 kubelet[2720]: I0813 01:26:02.256112 2720 kubelet.go:2405] "Pod admission denied" podUID="e95db96a-49c8-432b-8553-6b3d4329a5e3" pod="tigera-operator/tigera-operator-747864d56d-sl4f4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:02.444876 kubelet[2720]: I0813 01:26:02.444833 2720 kubelet.go:2405] "Pod admission denied" podUID="eb09bef9-6f45-470c-8446-ceda7e8c32a0" pod="tigera-operator/tigera-operator-747864d56d-ntr6v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:02.541919 kubelet[2720]: I0813 01:26:02.541813 2720 kubelet.go:2405] "Pod admission denied" podUID="9d69c80c-22ec-4221-bcd2-ed7f4a7a532a" pod="tigera-operator/tigera-operator-747864d56d-dm5j5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:02.648519 kubelet[2720]: I0813 01:26:02.648410 2720 kubelet.go:2405] "Pod admission denied" podUID="27bf0fe6-e264-4fb5-8b22-4669b20e481a" pod="tigera-operator/tigera-operator-747864d56d-4mzjs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:02.741507 kubelet[2720]: I0813 01:26:02.741464 2720 kubelet.go:2405] "Pod admission denied" podUID="b27b1768-43e7-4f5e-aaf5-fa89fafb0733" pod="tigera-operator/tigera-operator-747864d56d-p84c5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:02.794668 kubelet[2720]: I0813 01:26:02.794469 2720 kubelet.go:2405] "Pod admission denied" podUID="09599f79-b676-4cc9-855e-306802d20fce" pod="tigera-operator/tigera-operator-747864d56d-v49kg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:02.892529 kubelet[2720]: I0813 01:26:02.892493 2720 kubelet.go:2405] "Pod admission denied" podUID="3858c044-d959-47b8-9642-3e2e46515ea4" pod="tigera-operator/tigera-operator-747864d56d-xw67d" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:03.092392 kubelet[2720]: I0813 01:26:03.092203 2720 kubelet.go:2405] "Pod admission denied" podUID="862da7ab-cff7-4a90-916c-1c0756782f90" pod="tigera-operator/tigera-operator-747864d56d-snpgk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:03.192449 kubelet[2720]: I0813 01:26:03.192408 2720 kubelet.go:2405] "Pod admission denied" podUID="a220eadc-6bed-4480-8447-b85bb71c3065" pod="tigera-operator/tigera-operator-747864d56d-w66sd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:03.295796 kubelet[2720]: I0813 01:26:03.295758 2720 kubelet.go:2405] "Pod admission denied" podUID="ab3b911c-b1b3-4cf9-9141-bafe548a01da" pod="tigera-operator/tigera-operator-747864d56d-mck67" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:03.394952 kubelet[2720]: I0813 01:26:03.394839 2720 kubelet.go:2405] "Pod admission denied" podUID="33f6af89-c78f-43e6-8428-5c2a214ec05f" pod="tigera-operator/tigera-operator-747864d56d-rjz7b" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:03.443444 kubelet[2720]: I0813 01:26:03.443397 2720 kubelet.go:2405] "Pod admission denied" podUID="87aa343b-e41b-4c7d-8e5c-59ce25cc6c47" pod="tigera-operator/tigera-operator-747864d56d-cs947" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:03.543947 kubelet[2720]: I0813 01:26:03.543898 2720 kubelet.go:2405] "Pod admission denied" podUID="33732e42-ed01-4978-bd50-dd5d8efd8b0b" pod="tigera-operator/tigera-operator-747864d56d-4f66f" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:03.642599 kubelet[2720]: I0813 01:26:03.642546 2720 kubelet.go:2405] "Pod admission denied" podUID="6a614d94-d13a-46f5-8a05-05dcb3e48daf" pod="tigera-operator/tigera-operator-747864d56d-vl9xz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:03.743508 kubelet[2720]: I0813 01:26:03.742881 2720 kubelet.go:2405] "Pod admission denied" podUID="0f3ff843-bf33-4ce1-b917-9f9f43c860c8" pod="tigera-operator/tigera-operator-747864d56d-9nnn8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:03.844102 kubelet[2720]: I0813 01:26:03.844071 2720 kubelet.go:2405] "Pod admission denied" podUID="14642a8b-2b37-408f-aedb-f8bcf095f425" pod="tigera-operator/tigera-operator-747864d56d-mm7sd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:03.943258 kubelet[2720]: I0813 01:26:03.943191 2720 kubelet.go:2405] "Pod admission denied" podUID="bb99c479-d766-4595-a030-25841d451151" pod="tigera-operator/tigera-operator-747864d56d-c24r9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:04.145857 kubelet[2720]: I0813 01:26:04.145708 2720 kubelet.go:2405] "Pod admission denied" podUID="0d7d558c-ea01-42e6-8569-741e8ac21dc5" pod="tigera-operator/tigera-operator-747864d56d-q2xr2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:04.246674 kubelet[2720]: I0813 01:26:04.246478 2720 kubelet.go:2405] "Pod admission denied" podUID="9000d87e-b5d9-42ab-9f89-0857fcb174ce" pod="tigera-operator/tigera-operator-747864d56d-nbgnc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:04.343915 kubelet[2720]: I0813 01:26:04.343878 2720 kubelet.go:2405] "Pod admission denied" podUID="ea2d6353-c14a-4a02-a508-2e0a9adf2bba" pod="tigera-operator/tigera-operator-747864d56d-z7rnx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:04.543841 kubelet[2720]: I0813 01:26:04.543791 2720 kubelet.go:2405] "Pod admission denied" podUID="03e7f2f3-08ef-4c02-9cd6-c67feef051de" pod="tigera-operator/tigera-operator-747864d56d-nqdzf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:04.642828 kubelet[2720]: I0813 01:26:04.642799 2720 kubelet.go:2405] "Pod admission denied" podUID="0c5b0222-5d2d-4ffa-82ea-b83d438a17e6" pod="tigera-operator/tigera-operator-747864d56d-l92q9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:04.691245 kubelet[2720]: I0813 01:26:04.691225 2720 kubelet.go:2405] "Pod admission denied" podUID="91192af9-8bba-446f-a28d-04a055d5cf81" pod="tigera-operator/tigera-operator-747864d56d-bv2zb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:04.802840 kubelet[2720]: I0813 01:26:04.802110 2720 kubelet.go:2405] "Pod admission denied" podUID="1de042bb-04a6-47f9-99da-189135218840" pod="tigera-operator/tigera-operator-747864d56d-dvccv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:04.893883 kubelet[2720]: I0813 01:26:04.893838 2720 kubelet.go:2405] "Pod admission denied" podUID="0902fcf4-9886-43fc-9c71-c61cc77aa8fb" pod="tigera-operator/tigera-operator-747864d56d-xrhfr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:04.943054 kubelet[2720]: I0813 01:26:04.943012 2720 kubelet.go:2405] "Pod admission denied" podUID="896b601c-2ae1-4ea7-b477-0a6e142acd9e" pod="tigera-operator/tigera-operator-747864d56d-d29pp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:05.042453 kubelet[2720]: I0813 01:26:05.042422 2720 kubelet.go:2405] "Pod admission denied" podUID="1918d66b-be59-4d37-bed8-8a630fdb6d6e" pod="tigera-operator/tigera-operator-747864d56d-gknl5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:05.143507 kubelet[2720]: I0813 01:26:05.143404 2720 kubelet.go:2405] "Pod admission denied" podUID="f9f51c08-8492-43fe-84df-d4f63fa2d9f0" pod="tigera-operator/tigera-operator-747864d56d-lg7j9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:05.243376 kubelet[2720]: I0813 01:26:05.243339 2720 kubelet.go:2405] "Pod admission denied" podUID="e9dded8d-767c-454b-9e13-7718ef4a8459" pod="tigera-operator/tigera-operator-747864d56d-zjg22" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:05.343135 kubelet[2720]: I0813 01:26:05.343082 2720 kubelet.go:2405] "Pod admission denied" podUID="77fb88fa-e2c8-4ea5-8530-1b2b55da37a9" pod="tigera-operator/tigera-operator-747864d56d-pfb67" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:05.445884 kubelet[2720]: I0813 01:26:05.445080 2720 kubelet.go:2405] "Pod admission denied" podUID="51b8339c-bbcd-4ce2-a5b6-14d2af7efbd0" pod="tigera-operator/tigera-operator-747864d56d-89bh8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:05.643841 kubelet[2720]: I0813 01:26:05.643790 2720 kubelet.go:2405] "Pod admission denied" podUID="d54379f4-911e-426f-a27a-93ae257f621e" pod="tigera-operator/tigera-operator-747864d56d-k77lp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:05.744552 kubelet[2720]: I0813 01:26:05.744506 2720 kubelet.go:2405] "Pod admission denied" podUID="96971b88-2ab7-41c4-b43e-6bab0385af10" pod="tigera-operator/tigera-operator-747864d56d-pcxcc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:05.844984 kubelet[2720]: I0813 01:26:05.844827 2720 kubelet.go:2405] "Pod admission denied" podUID="8940a4e8-0ded-4db1-8c97-c1497f7fc707" pod="tigera-operator/tigera-operator-747864d56d-z27lq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:06.045618 kubelet[2720]: I0813 01:26:06.045510 2720 kubelet.go:2405] "Pod admission denied" podUID="986bc6f1-e35b-4518-ba6f-2a71e928ea6a" pod="tigera-operator/tigera-operator-747864d56d-qbdkc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:06.143466 kubelet[2720]: I0813 01:26:06.143426 2720 kubelet.go:2405] "Pod admission denied" podUID="c8fc2ba3-c0bf-46a8-82f6-fcca6069f4d3" pod="tigera-operator/tigera-operator-747864d56d-sq4td" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:06.244416 kubelet[2720]: I0813 01:26:06.244362 2720 kubelet.go:2405] "Pod admission denied" podUID="3919eea3-65d7-41b3-bd95-9e791c8900cc" pod="tigera-operator/tigera-operator-747864d56d-btg9b" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:06.345602 kubelet[2720]: I0813 01:26:06.345497 2720 kubelet.go:2405] "Pod admission denied" podUID="cc0c2c08-df7b-474b-9ffc-3760d536796b" pod="tigera-operator/tigera-operator-747864d56d-bnnrc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:06.444035 kubelet[2720]: I0813 01:26:06.443744 2720 kubelet.go:2405] "Pod admission denied" podUID="9cc4fa19-b5a9-451e-94b5-4302b1dada47" pod="tigera-operator/tigera-operator-747864d56d-cx9l6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:06.646792 kubelet[2720]: I0813 01:26:06.646666 2720 kubelet.go:2405] "Pod admission denied" podUID="68a50b20-5858-447d-8ba0-d7b368f209c9" pod="tigera-operator/tigera-operator-747864d56d-plxqn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:06.745078 kubelet[2720]: I0813 01:26:06.745037 2720 kubelet.go:2405] "Pod admission denied" podUID="2cfa2faf-f1d5-478e-a2b0-08ba3e5037ae" pod="tigera-operator/tigera-operator-747864d56d-66ggh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:06.842503 kubelet[2720]: I0813 01:26:06.842466 2720 kubelet.go:2405] "Pod admission denied" podUID="dc8c19ba-c417-4a87-92e4-25e98b005568" pod="tigera-operator/tigera-operator-747864d56d-tkhk6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:06.945453 kubelet[2720]: I0813 01:26:06.945070 2720 kubelet.go:2405] "Pod admission denied" podUID="47f4d93c-2434-40d8-836d-2e53c6a35cc2" pod="tigera-operator/tigera-operator-747864d56d-vwplx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:06.992568 kubelet[2720]: I0813 01:26:06.992531 2720 kubelet.go:2405] "Pod admission denied" podUID="78d0a84d-ab59-4e7a-b81e-7816ed73faba" pod="tigera-operator/tigera-operator-747864d56d-fc792" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:07.093609 kubelet[2720]: I0813 01:26:07.093572 2720 kubelet.go:2405] "Pod admission denied" podUID="0b2f8d12-fcee-4de5-b961-bce9f2396c3c" pod="tigera-operator/tigera-operator-747864d56d-mpl62" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:07.194546 kubelet[2720]: I0813 01:26:07.194501 2720 kubelet.go:2405] "Pod admission denied" podUID="fb8fe2fd-d8bd-4756-9439-c5beafb24f75" pod="tigera-operator/tigera-operator-747864d56d-4bqf6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:07.295490 kubelet[2720]: I0813 01:26:07.295450 2720 kubelet.go:2405] "Pod admission denied" podUID="ebb919be-becf-484d-82b0-be1b2c80be4d" pod="tigera-operator/tigera-operator-747864d56d-7lf85" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:07.395168 kubelet[2720]: I0813 01:26:07.395119 2720 kubelet.go:2405] "Pod admission denied" podUID="adfa151d-6cb9-4c9c-a534-6de6866e94eb" pod="tigera-operator/tigera-operator-747864d56d-924j8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:07.496187 kubelet[2720]: I0813 01:26:07.496141 2720 kubelet.go:2405] "Pod admission denied" podUID="d751f5be-03f6-4864-96ca-3fd1d40fec45" pod="tigera-operator/tigera-operator-747864d56d-c6l97" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:07.522642 kubelet[2720]: I0813 01:26:07.522600 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:26:07.522642 kubelet[2720]: I0813 01:26:07.522645 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:26:07.524593 kubelet[2720]: I0813 01:26:07.524546 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:26:07.537015 kubelet[2720]: I0813 01:26:07.536986 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:26:07.537143 kubelet[2720]: I0813 01:26:07.537063 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-node-2p8hj","calico-system/csi-node-driver-8bzzs","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:26:07.537143 kubelet[2720]: E0813 01:26:07.537090 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:26:07.537143 kubelet[2720]: E0813 01:26:07.537099 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:07.537143 kubelet[2720]: E0813 01:26:07.537106 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:26:07.537143 kubelet[2720]: E0813 01:26:07.537112 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:26:07.537143 kubelet[2720]: E0813 01:26:07.537118 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:07.537143 kubelet[2720]: E0813 01:26:07.537126 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:26:07.537143 kubelet[2720]: E0813 01:26:07.537135 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:26:07.537143 kubelet[2720]: E0813 01:26:07.537143 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:26:07.537143 kubelet[2720]: E0813 01:26:07.537151 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:26:07.537427 kubelet[2720]: E0813 01:26:07.537159 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:26:07.537427 kubelet[2720]: I0813 01:26:07.537168 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:26:07.595215 kubelet[2720]: I0813 01:26:07.594331 2720 kubelet.go:2405] "Pod admission denied" podUID="f2c25f43-b84a-4e8e-8070-54a54f88633b" pod="tigera-operator/tigera-operator-747864d56d-pdw5m" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:07.643681 kubelet[2720]: I0813 01:26:07.643635 2720 kubelet.go:2405] "Pod admission denied" podUID="0d6bf473-81e5-4fb3-880c-e052c9420151" pod="tigera-operator/tigera-operator-747864d56d-bpkn5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:07.745608 kubelet[2720]: I0813 01:26:07.745564 2720 kubelet.go:2405] "Pod admission denied" podUID="44038863-c2ee-4cd3-8a80-eb820bf1d160" pod="tigera-operator/tigera-operator-747864d56d-7q4wl" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:07.844571 kubelet[2720]: I0813 01:26:07.844534 2720 kubelet.go:2405] "Pod admission denied" podUID="cda12822-7557-4164-8e12-2c086c5c5362" pod="tigera-operator/tigera-operator-747864d56d-7fwxr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:07.955818 kubelet[2720]: I0813 01:26:07.955194 2720 kubelet.go:2405] "Pod admission denied" podUID="b5efcea3-c6ba-499d-91d5-87c8497ff8c5" pod="tigera-operator/tigera-operator-747864d56d-67zjk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:08.044897 kubelet[2720]: I0813 01:26:08.044843 2720 kubelet.go:2405] "Pod admission denied" podUID="cccfb179-c4bc-497f-9cee-6ebce7cb2eb2" pod="tigera-operator/tigera-operator-747864d56d-mtnx5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:08.145163 kubelet[2720]: I0813 01:26:08.145126 2720 kubelet.go:2405] "Pod admission denied" podUID="d4764bf2-e83c-4c5c-9c52-fe5fb3866188" pod="tigera-operator/tigera-operator-747864d56d-qdfmq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:08.245046 kubelet[2720]: I0813 01:26:08.245012 2720 kubelet.go:2405] "Pod admission denied" podUID="d539b310-7af8-464d-8c63-cb89b497f6ca" pod="tigera-operator/tigera-operator-747864d56d-vzls8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:08.293777 kubelet[2720]: I0813 01:26:08.293698 2720 kubelet.go:2405] "Pod admission denied" podUID="3035d7d6-6967-4ceb-af5b-8afcc25a4bb3" pod="tigera-operator/tigera-operator-747864d56d-rkl2w" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:08.395150 kubelet[2720]: I0813 01:26:08.395055 2720 kubelet.go:2405] "Pod admission denied" podUID="e3fb1684-f434-49a4-9e83-124ea91758d5" pod="tigera-operator/tigera-operator-747864d56d-7nwcg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:08.494932 kubelet[2720]: I0813 01:26:08.494884 2720 kubelet.go:2405] "Pod admission denied" podUID="6ae20110-7551-4392-9607-1cad24b08088" pod="tigera-operator/tigera-operator-747864d56d-dxc2x" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:08.543236 kubelet[2720]: I0813 01:26:08.543134 2720 kubelet.go:2405] "Pod admission denied" podUID="83eaaec1-8445-447e-bfa5-80aa2d77ada7" pod="tigera-operator/tigera-operator-747864d56d-5qb2f" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:08.644427 kubelet[2720]: I0813 01:26:08.644367 2720 kubelet.go:2405] "Pod admission denied" podUID="b187dcac-ea06-46d4-ac8b-6401e0aa433f" pod="tigera-operator/tigera-operator-747864d56d-njl8m" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:08.845586 kubelet[2720]: I0813 01:26:08.845062 2720 kubelet.go:2405] "Pod admission denied" podUID="e526a0d0-ac6d-4eeb-aa9d-b624d20b92c4" pod="tigera-operator/tigera-operator-747864d56d-vmc6c" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:08.943308 kubelet[2720]: I0813 01:26:08.943278 2720 kubelet.go:2405] "Pod admission denied" podUID="844eaf37-8db0-4d8d-808e-1b3dc76e22b6" pod="tigera-operator/tigera-operator-747864d56d-rbwhh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:08.995730 kubelet[2720]: I0813 01:26:08.995431 2720 kubelet.go:2405] "Pod admission denied" podUID="3dcc2dda-5260-4de8-bf97-2c747afe2f87" pod="tigera-operator/tigera-operator-747864d56d-jhrrt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:09.073449 kubelet[2720]: E0813 01:26:09.073426 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:26:09.073906 containerd[1567]: time="2025-08-13T01:26:09.073843539Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:26:09.097520 kubelet[2720]: I0813 01:26:09.097316 2720 kubelet.go:2405] "Pod admission denied" podUID="ed3582fe-4357-458e-9984-77ff265cc131" pod="tigera-operator/tigera-operator-747864d56d-d84kh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:09.129910 containerd[1567]: time="2025-08-13T01:26:09.129859001Z" level=error msg="Failed to destroy network for sandbox \"112392741f52af36e6b1bb24af34ef889dbb2c27ef47f7a57c44366c09df3f4e\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:09.131557 containerd[1567]: time="2025-08-13T01:26:09.131492502Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"112392741f52af36e6b1bb24af34ef889dbb2c27ef47f7a57c44366c09df3f4e\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:09.132035 kubelet[2720]: E0813 01:26:09.132000 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"112392741f52af36e6b1bb24af34ef889dbb2c27ef47f7a57c44366c09df3f4e\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:09.132112 kubelet[2720]: E0813 01:26:09.132088 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"112392741f52af36e6b1bb24af34ef889dbb2c27ef47f7a57c44366c09df3f4e\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:26:09.132164 kubelet[2720]: E0813 01:26:09.132115 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"112392741f52af36e6b1bb24af34ef889dbb2c27ef47f7a57c44366c09df3f4e\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:26:09.132190 kubelet[2720]: E0813 01:26:09.132161 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"112392741f52af36e6b1bb24af34ef889dbb2c27ef47f7a57c44366c09df3f4e\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:26:09.132554 systemd[1]: run-netns-cni\x2df5e81093\x2d01e2\x2d9dd5\x2d94c1\x2d63ae5dbee525.mount: Deactivated successfully. Aug 13 01:26:09.296204 kubelet[2720]: I0813 01:26:09.295960 2720 kubelet.go:2405] "Pod admission denied" podUID="492a0469-b489-40bb-9380-0e67a4a776a3" pod="tigera-operator/tigera-operator-747864d56d-hjhjn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:09.393464 kubelet[2720]: I0813 01:26:09.393355 2720 kubelet.go:2405] "Pod admission denied" podUID="42a5e7a5-0ea7-4478-ac72-bfe445b054c2" pod="tigera-operator/tigera-operator-747864d56d-m4brd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:09.441504 kubelet[2720]: I0813 01:26:09.441181 2720 kubelet.go:2405] "Pod admission denied" podUID="2fa3fa14-3cd4-4b7d-9b61-5155d4605752" pod="tigera-operator/tigera-operator-747864d56d-cw2p5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:09.544304 kubelet[2720]: I0813 01:26:09.544264 2720 kubelet.go:2405] "Pod admission denied" podUID="133d8024-f12d-4800-a1c7-ade76da0b5c9" pod="tigera-operator/tigera-operator-747864d56d-lcrjt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:09.643604 kubelet[2720]: I0813 01:26:09.643502 2720 kubelet.go:2405] "Pod admission denied" podUID="83da5392-cdd7-46b6-8363-534f4347fd0f" pod="tigera-operator/tigera-operator-747864d56d-9d4dk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:09.742840 kubelet[2720]: I0813 01:26:09.742811 2720 kubelet.go:2405] "Pod admission denied" podUID="590b13c1-d612-4b82-a1e2-ca0d086a0f5b" pod="tigera-operator/tigera-operator-747864d56d-dcfws" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:09.841570 kubelet[2720]: I0813 01:26:09.841536 2720 kubelet.go:2405] "Pod admission denied" podUID="7dcbef66-7b56-444c-b1ff-48c52d170b32" pod="tigera-operator/tigera-operator-747864d56d-t8vbx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:09.944802 kubelet[2720]: I0813 01:26:09.944481 2720 kubelet.go:2405] "Pod admission denied" podUID="40dab74d-3e6a-41b2-a9c5-8832cef60cc0" pod="tigera-operator/tigera-operator-747864d56d-hg7nh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:10.044122 kubelet[2720]: I0813 01:26:10.044082 2720 kubelet.go:2405] "Pod admission denied" podUID="c984960c-97aa-41b0-bb1f-45d78caa36cf" pod="tigera-operator/tigera-operator-747864d56d-rcbs9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:10.074320 containerd[1567]: time="2025-08-13T01:26:10.074059255Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:26:10.114708 kubelet[2720]: I0813 01:26:10.113998 2720 kubelet.go:2405] "Pod admission denied" podUID="a0a05b51-67eb-42a5-889a-0916d79657fd" pod="tigera-operator/tigera-operator-747864d56d-fcrk5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:10.138462 containerd[1567]: time="2025-08-13T01:26:10.138376780Z" level=error msg="Failed to destroy network for sandbox \"920837ea1c441c1d62a5a4baac91690808347a54739402e74f11406ad9999268\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:10.140302 containerd[1567]: time="2025-08-13T01:26:10.139985501Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"920837ea1c441c1d62a5a4baac91690808347a54739402e74f11406ad9999268\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:10.140483 kubelet[2720]: E0813 01:26:10.140458 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"920837ea1c441c1d62a5a4baac91690808347a54739402e74f11406ad9999268\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:10.140615 kubelet[2720]: E0813 01:26:10.140585 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"920837ea1c441c1d62a5a4baac91690808347a54739402e74f11406ad9999268\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:26:10.140680 kubelet[2720]: E0813 01:26:10.140666 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"920837ea1c441c1d62a5a4baac91690808347a54739402e74f11406ad9999268\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:26:10.140797 kubelet[2720]: E0813 01:26:10.140777 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"920837ea1c441c1d62a5a4baac91690808347a54739402e74f11406ad9999268\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:26:10.143807 systemd[1]: run-netns-cni\x2d69665dab\x2d1cbf\x2d836a\x2d081a\x2dc9a91be59322.mount: Deactivated successfully. Aug 13 01:26:10.201062 kubelet[2720]: I0813 01:26:10.200935 2720 kubelet.go:2405] "Pod admission denied" podUID="7f43fb4a-345b-4639-8707-36fda1d4a0c5" pod="tigera-operator/tigera-operator-747864d56d-lqjwg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:10.294282 kubelet[2720]: I0813 01:26:10.293806 2720 kubelet.go:2405] "Pod admission denied" podUID="e4938387-c246-4438-9bb0-c2c880b57699" pod="tigera-operator/tigera-operator-747864d56d-wk8t9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:10.395528 kubelet[2720]: I0813 01:26:10.395494 2720 kubelet.go:2405] "Pod admission denied" podUID="0768d73a-7554-40de-94bc-7b962f6fe02f" pod="tigera-operator/tigera-operator-747864d56d-4bw8m" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:10.598329 kubelet[2720]: I0813 01:26:10.598275 2720 kubelet.go:2405] "Pod admission denied" podUID="f080efae-b228-4fb7-b5c9-e28e3787211d" pod="tigera-operator/tigera-operator-747864d56d-bphbs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:10.700735 kubelet[2720]: I0813 01:26:10.699924 2720 kubelet.go:2405] "Pod admission denied" podUID="8009bb46-d7c2-49d1-aee8-5b724550341d" pod="tigera-operator/tigera-operator-747864d56d-bnp84" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:10.797646 kubelet[2720]: I0813 01:26:10.797585 2720 kubelet.go:2405] "Pod admission denied" podUID="2a87efd4-8194-462c-a420-9ddf9b442d85" pod="tigera-operator/tigera-operator-747864d56d-8nx8h" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:10.895234 kubelet[2720]: I0813 01:26:10.895098 2720 kubelet.go:2405] "Pod admission denied" podUID="9621511e-ca44-4935-8bc0-065f2a7dff51" pod="tigera-operator/tigera-operator-747864d56d-tbkv7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:10.996366 kubelet[2720]: I0813 01:26:10.996316 2720 kubelet.go:2405] "Pod admission denied" podUID="da9b4f21-c520-4dc0-9596-8dd4882a6619" pod="tigera-operator/tigera-operator-747864d56d-fjkbd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:11.093804 kubelet[2720]: I0813 01:26:11.093749 2720 kubelet.go:2405] "Pod admission denied" podUID="ed560dfa-a28b-4567-a9f5-5112bdd1ad0f" pod="tigera-operator/tigera-operator-747864d56d-hpfkg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:11.194564 kubelet[2720]: I0813 01:26:11.194314 2720 kubelet.go:2405] "Pod admission denied" podUID="465c08cf-c24a-4170-a447-385dbd769f59" pod="tigera-operator/tigera-operator-747864d56d-vfkc2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:11.394242 kubelet[2720]: I0813 01:26:11.394195 2720 kubelet.go:2405] "Pod admission denied" podUID="c3b481e9-93ee-4cb2-a2a0-b96d29d02756" pod="tigera-operator/tigera-operator-747864d56d-x92xq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:11.493488 kubelet[2720]: I0813 01:26:11.493448 2720 kubelet.go:2405] "Pod admission denied" podUID="587f6aae-986f-461c-9897-3730922a85fd" pod="tigera-operator/tigera-operator-747864d56d-8pclg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:11.595363 kubelet[2720]: I0813 01:26:11.595285 2720 kubelet.go:2405] "Pod admission denied" podUID="1cbe14bd-0044-4176-be07-c82494be230b" pod="tigera-operator/tigera-operator-747864d56d-qts5l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:11.695480 kubelet[2720]: I0813 01:26:11.695426 2720 kubelet.go:2405] "Pod admission denied" podUID="1397bbc3-66cd-47d1-ade3-c5dd18c973b4" pod="tigera-operator/tigera-operator-747864d56d-svqk8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:11.794465 kubelet[2720]: I0813 01:26:11.794366 2720 kubelet.go:2405] "Pod admission denied" podUID="1f81973b-78fb-4ff3-bc8d-2a57ebba06b9" pod="tigera-operator/tigera-operator-747864d56d-rgxtc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:11.893728 kubelet[2720]: I0813 01:26:11.893688 2720 kubelet.go:2405] "Pod admission denied" podUID="ca68e1be-f1e9-46a6-81ab-48ff631a99ec" pod="tigera-operator/tigera-operator-747864d56d-cg8cj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:11.995030 kubelet[2720]: I0813 01:26:11.994988 2720 kubelet.go:2405] "Pod admission denied" podUID="05005193-d0eb-4cd4-9d9c-2c182bcbc7ec" pod="tigera-operator/tigera-operator-747864d56d-4z72q" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:12.080367 containerd[1567]: time="2025-08-13T01:26:12.079892946Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:26:12.105648 kubelet[2720]: I0813 01:26:12.105242 2720 kubelet.go:2405] "Pod admission denied" podUID="64846ca7-c03e-49a9-b2c5-2dacb7171c1b" pod="tigera-operator/tigera-operator-747864d56d-vsdbr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:12.156807 containerd[1567]: time="2025-08-13T01:26:12.156755789Z" level=error msg="Failed to destroy network for sandbox \"7e67fe288698e46c56107a8bd68292fc5126f2e39a3387db4f26c0e08af16cf8\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:12.159032 systemd[1]: run-netns-cni\x2d50e83c2e\x2db450\x2dd218\x2d422b\x2df120a615d8ca.mount: Deactivated successfully. Aug 13 01:26:12.160233 containerd[1567]: time="2025-08-13T01:26:12.160196039Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"7e67fe288698e46c56107a8bd68292fc5126f2e39a3387db4f26c0e08af16cf8\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:12.160677 kubelet[2720]: E0813 01:26:12.160630 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7e67fe288698e46c56107a8bd68292fc5126f2e39a3387db4f26c0e08af16cf8\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:12.160785 kubelet[2720]: E0813 01:26:12.160696 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7e67fe288698e46c56107a8bd68292fc5126f2e39a3387db4f26c0e08af16cf8\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:12.160785 kubelet[2720]: E0813 01:26:12.160734 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7e67fe288698e46c56107a8bd68292fc5126f2e39a3387db4f26c0e08af16cf8\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:12.160880 kubelet[2720]: E0813 01:26:12.160809 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"7e67fe288698e46c56107a8bd68292fc5126f2e39a3387db4f26c0e08af16cf8\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:26:12.195930 kubelet[2720]: I0813 01:26:12.195877 2720 kubelet.go:2405] "Pod admission denied" podUID="3bf12dd7-09d8-47b8-9512-9a201b5849ad" pod="tigera-operator/tigera-operator-747864d56d-7tcw6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:12.293994 kubelet[2720]: I0813 01:26:12.293962 2720 kubelet.go:2405] "Pod admission denied" podUID="e5270213-cb8e-4bf1-9ae8-7131104b2dc1" pod="tigera-operator/tigera-operator-747864d56d-fvl76" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:12.393508 kubelet[2720]: I0813 01:26:12.393379 2720 kubelet.go:2405] "Pod admission denied" podUID="756b0141-a86f-445b-a148-24b5cfafdacc" pod="tigera-operator/tigera-operator-747864d56d-drvw9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:12.493523 kubelet[2720]: I0813 01:26:12.493489 2720 kubelet.go:2405] "Pod admission denied" podUID="7eaef360-dc8e-4153-a403-4e8ee9df6394" pod="tigera-operator/tigera-operator-747864d56d-7smrr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:12.595092 kubelet[2720]: I0813 01:26:12.595063 2720 kubelet.go:2405] "Pod admission denied" podUID="34feefc5-c1ec-46f5-86fd-f82bfc6cd70f" pod="tigera-operator/tigera-operator-747864d56d-9pbc7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:12.694337 kubelet[2720]: I0813 01:26:12.694165 2720 kubelet.go:2405] "Pod admission denied" podUID="8645b4ed-cb22-4c8a-91a3-616dc95eb7f5" pod="tigera-operator/tigera-operator-747864d56d-jnk7p" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:12.794173 kubelet[2720]: I0813 01:26:12.794137 2720 kubelet.go:2405] "Pod admission denied" podUID="f8742081-8338-47f3-893a-d1e4fcd3b60d" pod="tigera-operator/tigera-operator-747864d56d-d945r" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:12.892689 kubelet[2720]: I0813 01:26:12.892644 2720 kubelet.go:2405] "Pod admission denied" podUID="53866b2e-f49e-432f-9840-6eb865956153" pod="tigera-operator/tigera-operator-747864d56d-fkf4s" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:12.997087 kubelet[2720]: I0813 01:26:12.997038 2720 kubelet.go:2405] "Pod admission denied" podUID="0db2c773-fab3-4374-b0d5-096fd1223c19" pod="tigera-operator/tigera-operator-747864d56d-z2dv7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:13.095322 kubelet[2720]: I0813 01:26:13.095273 2720 kubelet.go:2405] "Pod admission denied" podUID="062d1ea4-c1c7-499e-8a3c-f497777ebf10" pod="tigera-operator/tigera-operator-747864d56d-w9s97" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:13.196153 kubelet[2720]: I0813 01:26:13.196119 2720 kubelet.go:2405] "Pod admission denied" podUID="ad0b1235-5150-496e-9d63-77af02e830d3" pod="tigera-operator/tigera-operator-747864d56d-vmfhg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:13.397793 kubelet[2720]: I0813 01:26:13.397282 2720 kubelet.go:2405] "Pod admission denied" podUID="6c43f5f4-70fd-4683-a3a9-794e6867f547" pod="tigera-operator/tigera-operator-747864d56d-45wjg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:13.494135 kubelet[2720]: I0813 01:26:13.494101 2720 kubelet.go:2405] "Pod admission denied" podUID="6a8ed769-d935-4586-a8d3-5b1a48ac16f1" pod="tigera-operator/tigera-operator-747864d56d-fv99v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:13.594742 kubelet[2720]: I0813 01:26:13.594676 2720 kubelet.go:2405] "Pod admission denied" podUID="0acbbea1-53d8-4799-a0fd-c51ef65e643c" pod="tigera-operator/tigera-operator-747864d56d-gcxrs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:13.793834 kubelet[2720]: I0813 01:26:13.793808 2720 kubelet.go:2405] "Pod admission denied" podUID="fb826e90-adc3-4cae-9531-5d3f5c1fea1b" pod="tigera-operator/tigera-operator-747864d56d-nqh55" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:13.893320 kubelet[2720]: I0813 01:26:13.893295 2720 kubelet.go:2405] "Pod admission denied" podUID="7c229e58-fcd6-46ca-9908-9820d20ff8c2" pod="tigera-operator/tigera-operator-747864d56d-9wrsv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:13.999851 kubelet[2720]: I0813 01:26:13.999769 2720 kubelet.go:2405] "Pod admission denied" podUID="74691e2e-fefb-4bf4-8d71-596b9bea91f2" pod="tigera-operator/tigera-operator-747864d56d-zhkwj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:14.076332 kubelet[2720]: E0813 01:26:14.076056 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:26:14.076332 kubelet[2720]: E0813 01:26:14.076256 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"calico-node\" with ImagePullBackOff: \"Back-off pulling image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": ErrImagePull: failed to pull and unpack image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1533876103: write /var/lib/containerd/tmpmounts/containerd-mount1533876103/usr/bin/calico-node: no space left on device\"" pod="calico-system/calico-node-2p8hj" podUID="4f334296-e8c0-427e-a090-d8fbd3582f26" Aug 13 01:26:14.076752 containerd[1567]: time="2025-08-13T01:26:14.076638488Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:26:14.112585 kubelet[2720]: I0813 01:26:14.112032 2720 kubelet.go:2405] "Pod admission denied" podUID="930bfe9d-246b-47bc-980b-56ae785fdfab" pod="tigera-operator/tigera-operator-747864d56d-rklks" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:14.147368 kubelet[2720]: I0813 01:26:14.147311 2720 kubelet.go:2405] "Pod admission denied" podUID="27efc2e5-29c1-47da-8d88-a7879447859a" pod="tigera-operator/tigera-operator-747864d56d-7lmch" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:14.149613 containerd[1567]: time="2025-08-13T01:26:14.149081658Z" level=error msg="Failed to destroy network for sandbox \"739b9d5c12bb9d0076ffd4cd2484f854a29288109220713b581847fa6e8a4659\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:14.151952 containerd[1567]: time="2025-08-13T01:26:14.151874399Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"739b9d5c12bb9d0076ffd4cd2484f854a29288109220713b581847fa6e8a4659\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:14.152822 kubelet[2720]: E0813 01:26:14.152136 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"739b9d5c12bb9d0076ffd4cd2484f854a29288109220713b581847fa6e8a4659\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:14.152822 kubelet[2720]: E0813 01:26:14.152204 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"739b9d5c12bb9d0076ffd4cd2484f854a29288109220713b581847fa6e8a4659\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:14.152822 kubelet[2720]: E0813 01:26:14.152268 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"739b9d5c12bb9d0076ffd4cd2484f854a29288109220713b581847fa6e8a4659\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:14.152822 kubelet[2720]: E0813 01:26:14.152308 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"739b9d5c12bb9d0076ffd4cd2484f854a29288109220713b581847fa6e8a4659\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:26:14.153634 systemd[1]: run-netns-cni\x2d4cc945d2\x2ded24\x2d91ba\x2d1c0d\x2d2e6414265706.mount: Deactivated successfully. Aug 13 01:26:14.244563 kubelet[2720]: I0813 01:26:14.244522 2720 kubelet.go:2405] "Pod admission denied" podUID="96172d64-e4ec-4a56-9052-4fbaf33456f7" pod="tigera-operator/tigera-operator-747864d56d-jx8vf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:14.447241 kubelet[2720]: I0813 01:26:14.446802 2720 kubelet.go:2405] "Pod admission denied" podUID="d7195440-d13c-45e6-97cb-e8d775e849de" pod="tigera-operator/tigera-operator-747864d56d-z6rdz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:14.543815 kubelet[2720]: I0813 01:26:14.543761 2720 kubelet.go:2405] "Pod admission denied" podUID="76ac868a-7712-4f33-84c6-a6493c91fe75" pod="tigera-operator/tigera-operator-747864d56d-zxfln" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:14.646040 kubelet[2720]: I0813 01:26:14.645993 2720 kubelet.go:2405] "Pod admission denied" podUID="ae5b6fcd-7555-4ccc-b850-508dbfa4ed5c" pod="tigera-operator/tigera-operator-747864d56d-67qwk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:14.847903 kubelet[2720]: I0813 01:26:14.847855 2720 kubelet.go:2405] "Pod admission denied" podUID="d457af90-b46a-41b8-8acc-e29ba959df0d" pod="tigera-operator/tigera-operator-747864d56d-mvhjv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:14.948091 kubelet[2720]: I0813 01:26:14.947903 2720 kubelet.go:2405] "Pod admission denied" podUID="a6d16242-bfaf-472e-a14f-5e21e66e8baa" pod="tigera-operator/tigera-operator-747864d56d-k6dmw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:15.044051 kubelet[2720]: I0813 01:26:15.044020 2720 kubelet.go:2405] "Pod admission denied" podUID="7d7a41bc-d794-4964-8b71-48de0f57e12b" pod="tigera-operator/tigera-operator-747864d56d-cxns4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:15.144683 kubelet[2720]: I0813 01:26:15.144560 2720 kubelet.go:2405] "Pod admission denied" podUID="092b1136-5c7d-456b-87d7-1f2e6ee6adb6" pod="tigera-operator/tigera-operator-747864d56d-b9dbd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:15.245060 kubelet[2720]: I0813 01:26:15.245023 2720 kubelet.go:2405] "Pod admission denied" podUID="5029d4aa-db10-4604-aba8-4c13ffb14a6d" pod="tigera-operator/tigera-operator-747864d56d-zhqw9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:15.344251 kubelet[2720]: I0813 01:26:15.344209 2720 kubelet.go:2405] "Pod admission denied" podUID="dde87693-2a88-44c7-b844-cf0ef15f2567" pod="tigera-operator/tigera-operator-747864d56d-hqj5g" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:15.445245 kubelet[2720]: I0813 01:26:15.445131 2720 kubelet.go:2405] "Pod admission denied" podUID="f1809b3c-fa68-43cc-a8f0-7d90ce5baced" pod="tigera-operator/tigera-operator-747864d56d-lgkjn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:15.544573 kubelet[2720]: I0813 01:26:15.544540 2720 kubelet.go:2405] "Pod admission denied" podUID="00d5be6a-4f83-4b0c-86ee-00ea90feeff1" pod="tigera-operator/tigera-operator-747864d56d-rl2vs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:15.644156 kubelet[2720]: I0813 01:26:15.644118 2720 kubelet.go:2405] "Pod admission denied" podUID="82816bd2-0d32-48d6-9078-845b878591ba" pod="tigera-operator/tigera-operator-747864d56d-zm8sg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:15.744091 kubelet[2720]: I0813 01:26:15.744056 2720 kubelet.go:2405] "Pod admission denied" podUID="16c066a2-881f-4cc2-99e2-5f91adff0a95" pod="tigera-operator/tigera-operator-747864d56d-rpnh2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:15.852967 kubelet[2720]: I0813 01:26:15.851074 2720 kubelet.go:2405] "Pod admission denied" podUID="a40278fd-fb3b-4cd8-a991-a7b1387f133a" pod="tigera-operator/tigera-operator-747864d56d-2jw6t" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:15.944539 kubelet[2720]: I0813 01:26:15.944485 2720 kubelet.go:2405] "Pod admission denied" podUID="e23c31d6-7adb-4862-99a8-f3032cec6e5d" pod="tigera-operator/tigera-operator-747864d56d-bztsc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:16.015527 kubelet[2720]: I0813 01:26:16.015416 2720 kubelet.go:2405] "Pod admission denied" podUID="6676328e-a45c-43b2-af28-391708026d39" pod="tigera-operator/tigera-operator-747864d56d-qs6k4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:16.094654 kubelet[2720]: I0813 01:26:16.094620 2720 kubelet.go:2405] "Pod admission denied" podUID="b642839f-5751-4089-a254-05f1666c1684" pod="tigera-operator/tigera-operator-747864d56d-9k8hn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:16.195995 kubelet[2720]: I0813 01:26:16.195948 2720 kubelet.go:2405] "Pod admission denied" podUID="947eac85-5ac1-430f-8d8c-f003ea97cd6a" pod="tigera-operator/tigera-operator-747864d56d-5lkq9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:16.295183 kubelet[2720]: I0813 01:26:16.294567 2720 kubelet.go:2405] "Pod admission denied" podUID="27e399ec-8b96-432a-88b1-387ba11147a4" pod="tigera-operator/tigera-operator-747864d56d-7tw5j" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:16.394459 kubelet[2720]: I0813 01:26:16.393830 2720 kubelet.go:2405] "Pod admission denied" podUID="ccd0289b-8490-4948-8674-ebc58585f9ee" pod="tigera-operator/tigera-operator-747864d56d-b9gc7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:16.495204 kubelet[2720]: I0813 01:26:16.495156 2720 kubelet.go:2405] "Pod admission denied" podUID="f530a194-b928-4907-b22e-e6cd4ea329f2" pod="tigera-operator/tigera-operator-747864d56d-cxhls" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:16.695139 kubelet[2720]: I0813 01:26:16.695041 2720 kubelet.go:2405] "Pod admission denied" podUID="3c668af3-147d-4918-bc7b-526a05393aac" pod="tigera-operator/tigera-operator-747864d56d-clp7h" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:16.799738 kubelet[2720]: I0813 01:26:16.799488 2720 kubelet.go:2405] "Pod admission denied" podUID="bf3b1a2d-ae96-42e6-8efe-ea4d215bccac" pod="tigera-operator/tigera-operator-747864d56d-94pts" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:16.893665 kubelet[2720]: I0813 01:26:16.893631 2720 kubelet.go:2405] "Pod admission denied" podUID="80484138-a0c3-4e41-8fb9-e8505ab0066e" pod="tigera-operator/tigera-operator-747864d56d-bhfl7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:16.996309 kubelet[2720]: I0813 01:26:16.996256 2720 kubelet.go:2405] "Pod admission denied" podUID="0525fd68-92f4-4c4d-a188-211d691d3a34" pod="tigera-operator/tigera-operator-747864d56d-zjj5k" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:17.043398 kubelet[2720]: I0813 01:26:17.043353 2720 kubelet.go:2405] "Pod admission denied" podUID="68a2dccf-6027-4064-9232-edddf48f5473" pod="tigera-operator/tigera-operator-747864d56d-lpnkd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:17.146590 kubelet[2720]: I0813 01:26:17.146532 2720 kubelet.go:2405] "Pod admission denied" podUID="0ca59524-1740-4d69-900f-855912e11975" pod="tigera-operator/tigera-operator-747864d56d-d8x7l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:17.245534 kubelet[2720]: I0813 01:26:17.245469 2720 kubelet.go:2405] "Pod admission denied" podUID="4ee76712-e69d-4993-ae26-6c77f4f713fc" pod="tigera-operator/tigera-operator-747864d56d-rkk6b" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:17.349666 kubelet[2720]: I0813 01:26:17.349530 2720 kubelet.go:2405] "Pod admission denied" podUID="fc2c385f-953c-465a-a3ee-4cbdea55572a" pod="tigera-operator/tigera-operator-747864d56d-dhk67" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:17.447355 kubelet[2720]: I0813 01:26:17.447307 2720 kubelet.go:2405] "Pod admission denied" podUID="068c7f02-d9e3-4009-bc82-cff9b73bb876" pod="tigera-operator/tigera-operator-747864d56d-hkzrr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:17.493649 kubelet[2720]: I0813 01:26:17.493605 2720 kubelet.go:2405] "Pod admission denied" podUID="04f3d5f7-6118-4cd8-87b5-3636e91ba52d" pod="tigera-operator/tigera-operator-747864d56d-grplc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:17.560831 kubelet[2720]: I0813 01:26:17.560800 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:26:17.560831 kubelet[2720]: I0813 01:26:17.560834 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:26:17.563483 kubelet[2720]: I0813 01:26:17.563419 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:26:17.572663 kubelet[2720]: I0813 01:26:17.572646 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:26:17.572751 kubelet[2720]: I0813 01:26:17.572732 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","calico-system/calico-node-2p8hj","calico-system/csi-node-driver-8bzzs","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:26:17.572847 kubelet[2720]: E0813 01:26:17.572760 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:26:17.572847 kubelet[2720]: E0813 01:26:17.572770 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:26:17.572847 kubelet[2720]: E0813 01:26:17.572775 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:17.572847 kubelet[2720]: E0813 01:26:17.572781 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:26:17.572847 kubelet[2720]: E0813 01:26:17.572787 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:17.572847 kubelet[2720]: E0813 01:26:17.572796 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:26:17.572847 kubelet[2720]: E0813 01:26:17.572803 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:26:17.572847 kubelet[2720]: E0813 01:26:17.572811 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:26:17.572847 kubelet[2720]: E0813 01:26:17.572818 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:26:17.572847 kubelet[2720]: E0813 01:26:17.572825 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:26:17.572847 kubelet[2720]: I0813 01:26:17.572834 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:26:17.597496 kubelet[2720]: I0813 01:26:17.597464 2720 kubelet.go:2405] "Pod admission denied" podUID="fa584c0f-bc7d-436b-a39b-ecea06b4298c" pod="tigera-operator/tigera-operator-747864d56d-8qhwq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:17.696618 kubelet[2720]: I0813 01:26:17.696494 2720 kubelet.go:2405] "Pod admission denied" podUID="b1fbdd69-fb0e-41a9-a710-4b697d2c120a" pod="tigera-operator/tigera-operator-747864d56d-btt24" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:17.795481 kubelet[2720]: I0813 01:26:17.795434 2720 kubelet.go:2405] "Pod admission denied" podUID="b0afb0cd-dffe-4397-a61a-92d1034f223a" pod="tigera-operator/tigera-operator-747864d56d-jb527" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:17.896531 kubelet[2720]: I0813 01:26:17.896473 2720 kubelet.go:2405] "Pod admission denied" podUID="73561002-121e-4081-9010-4fe358883fc9" pod="tigera-operator/tigera-operator-747864d56d-fkr2t" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:17.996052 kubelet[2720]: I0813 01:26:17.996007 2720 kubelet.go:2405] "Pod admission denied" podUID="b8f866a1-eabf-40df-a2f5-f5d4f8ff5465" pod="tigera-operator/tigera-operator-747864d56d-v76tn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:18.094294 kubelet[2720]: I0813 01:26:18.094260 2720 kubelet.go:2405] "Pod admission denied" podUID="cc1536ac-9039-4e80-8abe-3b2c4afb7357" pod="tigera-operator/tigera-operator-747864d56d-vnp9t" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:18.143205 kubelet[2720]: I0813 01:26:18.143168 2720 kubelet.go:2405] "Pod admission denied" podUID="38865b30-dc0b-496e-a01f-3bf4b3a6ca24" pod="tigera-operator/tigera-operator-747864d56d-dbhpg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:18.247109 kubelet[2720]: I0813 01:26:18.246995 2720 kubelet.go:2405] "Pod admission denied" podUID="ed7c8631-a076-488a-a17b-72275c47ce7d" pod="tigera-operator/tigera-operator-747864d56d-4w6pm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:18.348735 kubelet[2720]: I0813 01:26:18.348682 2720 kubelet.go:2405] "Pod admission denied" podUID="8658c44f-98e2-4eb6-ab27-cdab594ad6e6" pod="tigera-operator/tigera-operator-747864d56d-tdsjh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:18.394747 kubelet[2720]: I0813 01:26:18.394699 2720 kubelet.go:2405] "Pod admission denied" podUID="d6851c83-f5dc-4631-8389-12c82630e3e9" pod="tigera-operator/tigera-operator-747864d56d-gf9dm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:18.497703 kubelet[2720]: I0813 01:26:18.497384 2720 kubelet.go:2405] "Pod admission denied" podUID="e6bef43f-c187-4dd4-853e-7f9908440f8e" pod="tigera-operator/tigera-operator-747864d56d-s8rkr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:18.595416 kubelet[2720]: I0813 01:26:18.595376 2720 kubelet.go:2405] "Pod admission denied" podUID="bd815dff-c54a-45c9-9321-21f3c9a79d41" pod="tigera-operator/tigera-operator-747864d56d-8wvm7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:18.646422 kubelet[2720]: I0813 01:26:18.646385 2720 kubelet.go:2405] "Pod admission denied" podUID="e6ba90e8-d1c9-475f-8862-c05cafb4b864" pod="tigera-operator/tigera-operator-747864d56d-gwcwg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:18.745427 kubelet[2720]: I0813 01:26:18.745386 2720 kubelet.go:2405] "Pod admission denied" podUID="3ea83e4e-4a64-4fa2-89cf-e16bf6875a8c" pod="tigera-operator/tigera-operator-747864d56d-fj6gn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:18.846315 kubelet[2720]: I0813 01:26:18.846191 2720 kubelet.go:2405] "Pod admission denied" podUID="fa53c964-cd5d-4695-99b6-69d73d2fce62" pod="tigera-operator/tigera-operator-747864d56d-w9vbw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:18.947518 kubelet[2720]: I0813 01:26:18.947442 2720 kubelet.go:2405] "Pod admission denied" podUID="6904ec34-2c70-4778-b42c-6d5132cb78fa" pod="tigera-operator/tigera-operator-747864d56d-tws42" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:19.046859 kubelet[2720]: I0813 01:26:19.046791 2720 kubelet.go:2405] "Pod admission denied" podUID="9232cc10-8608-46a2-a179-11238a9aeb3c" pod="tigera-operator/tigera-operator-747864d56d-tw57f" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:19.147550 kubelet[2720]: I0813 01:26:19.146893 2720 kubelet.go:2405] "Pod admission denied" podUID="f8b1f9fb-48e4-46d6-b746-1745650a4611" pod="tigera-operator/tigera-operator-747864d56d-5frrs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:19.246064 kubelet[2720]: I0813 01:26:19.246026 2720 kubelet.go:2405] "Pod admission denied" podUID="28d7c481-86cc-432c-bda9-1aa1fd98d4d3" pod="tigera-operator/tigera-operator-747864d56d-vhf96" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:19.347522 kubelet[2720]: I0813 01:26:19.347473 2720 kubelet.go:2405] "Pod admission denied" podUID="3df9972d-6031-48a6-abb2-4bb35f99f592" pod="tigera-operator/tigera-operator-747864d56d-f6jk4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:19.445462 kubelet[2720]: I0813 01:26:19.445337 2720 kubelet.go:2405] "Pod admission denied" podUID="cd3a64f5-1260-413d-b909-2e4d12171537" pod="tigera-operator/tigera-operator-747864d56d-zlxfm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:19.547484 kubelet[2720]: I0813 01:26:19.547433 2720 kubelet.go:2405] "Pod admission denied" podUID="59b422a2-5b75-46b2-bc27-078ba4bab2a9" pod="tigera-operator/tigera-operator-747864d56d-lhf72" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:19.647961 kubelet[2720]: I0813 01:26:19.647896 2720 kubelet.go:2405] "Pod admission denied" podUID="b49ebb8e-1906-45b0-92b5-3c7e81be8eaa" pod="tigera-operator/tigera-operator-747864d56d-4dqr7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:19.695766 kubelet[2720]: I0813 01:26:19.695515 2720 kubelet.go:2405] "Pod admission denied" podUID="e32b3731-e2fd-45c1-8a69-c0fc21bbd67e" pod="tigera-operator/tigera-operator-747864d56d-m9ch9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:19.796974 kubelet[2720]: I0813 01:26:19.796919 2720 kubelet.go:2405] "Pod admission denied" podUID="d5a69eea-75b6-4a12-bc65-a188c45984b9" pod="tigera-operator/tigera-operator-747864d56d-7vd4z" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:19.998890 kubelet[2720]: I0813 01:26:19.998840 2720 kubelet.go:2405] "Pod admission denied" podUID="36c2d1fb-8e96-4781-bf31-f8fc7f296518" pod="tigera-operator/tigera-operator-747864d56d-fh2pr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:20.098882 kubelet[2720]: I0813 01:26:20.098830 2720 kubelet.go:2405] "Pod admission denied" podUID="f68b89b5-1226-4235-a964-64f72e4bf819" pod="tigera-operator/tigera-operator-747864d56d-4x7ch" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:20.199235 kubelet[2720]: I0813 01:26:20.199186 2720 kubelet.go:2405] "Pod admission denied" podUID="c2461ae9-1f8d-4f36-8d0d-103d3fb8fa0c" pod="tigera-operator/tigera-operator-747864d56d-l7kkk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:20.404743 kubelet[2720]: I0813 01:26:20.404603 2720 kubelet.go:2405] "Pod admission denied" podUID="804d8d3d-811e-47b1-a916-82a52fe480bf" pod="tigera-operator/tigera-operator-747864d56d-b4288" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:20.499187 kubelet[2720]: I0813 01:26:20.499135 2720 kubelet.go:2405] "Pod admission denied" podUID="19dab1e0-09d4-44c3-ac01-26704ddcb61d" pod="tigera-operator/tigera-operator-747864d56d-7r4vc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:20.595911 kubelet[2720]: I0813 01:26:20.595853 2720 kubelet.go:2405] "Pod admission denied" podUID="69ae9521-95f8-4ce6-bd15-13bea2c260e7" pod="tigera-operator/tigera-operator-747864d56d-rhw5k" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:20.696738 kubelet[2720]: I0813 01:26:20.696316 2720 kubelet.go:2405] "Pod admission denied" podUID="86c983e8-1e34-4d9d-999c-14992eaf6868" pod="tigera-operator/tigera-operator-747864d56d-89lkl" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:20.794149 kubelet[2720]: I0813 01:26:20.794111 2720 kubelet.go:2405] "Pod admission denied" podUID="d7304fdb-b80d-4e1e-b2a1-2c0062b0f0e3" pod="tigera-operator/tigera-operator-747864d56d-5kwf2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:21.005303 kubelet[2720]: I0813 01:26:21.004738 2720 kubelet.go:2405] "Pod admission denied" podUID="ffca8518-959a-4995-baa4-3cff75b162d4" pod="tigera-operator/tigera-operator-747864d56d-ccxp9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:21.103040 kubelet[2720]: I0813 01:26:21.102997 2720 kubelet.go:2405] "Pod admission denied" podUID="7beb8463-dc3d-4149-a25a-c75767060db5" pod="tigera-operator/tigera-operator-747864d56d-nmbcj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:21.196615 kubelet[2720]: I0813 01:26:21.196237 2720 kubelet.go:2405] "Pod admission denied" podUID="8859d791-6900-44ae-8562-95034a586381" pod="tigera-operator/tigera-operator-747864d56d-gnccg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:21.398678 kubelet[2720]: I0813 01:26:21.398551 2720 kubelet.go:2405] "Pod admission denied" podUID="a5174450-d82c-46a7-a1c2-5a89de44247b" pod="tigera-operator/tigera-operator-747864d56d-5f2pj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:21.498706 kubelet[2720]: I0813 01:26:21.498648 2720 kubelet.go:2405] "Pod admission denied" podUID="6ff038db-ae87-45d1-af86-bcd66397334b" pod="tigera-operator/tigera-operator-747864d56d-q2qtx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:21.596477 kubelet[2720]: I0813 01:26:21.596428 2720 kubelet.go:2405] "Pod admission denied" podUID="af636c7b-a055-4e7f-97fc-9a10a456413c" pod="tigera-operator/tigera-operator-747864d56d-jgk89" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:21.795054 kubelet[2720]: I0813 01:26:21.795021 2720 kubelet.go:2405] "Pod admission denied" podUID="70becc62-7811-46a7-b0ab-3137573b3b4b" pod="tigera-operator/tigera-operator-747864d56d-45fqp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:21.894919 kubelet[2720]: I0813 01:26:21.894874 2720 kubelet.go:2405] "Pod admission denied" podUID="32b60943-21ef-4796-bc1b-43c9e2d3a62f" pod="tigera-operator/tigera-operator-747864d56d-wnbfz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:21.996197 kubelet[2720]: I0813 01:26:21.996116 2720 kubelet.go:2405] "Pod admission denied" podUID="de9bf031-9faf-4ad3-b24d-6e224c63753f" pod="tigera-operator/tigera-operator-747864d56d-kgn5p" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:22.075800 kubelet[2720]: E0813 01:26:22.075509 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:26:22.075891 containerd[1567]: time="2025-08-13T01:26:22.075831445Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:26:22.113060 kubelet[2720]: I0813 01:26:22.112626 2720 kubelet.go:2405] "Pod admission denied" podUID="fd403f3b-29c7-400c-a1e5-082d2e383a37" pod="tigera-operator/tigera-operator-747864d56d-v9jzp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:22.153950 kubelet[2720]: I0813 01:26:22.153891 2720 kubelet.go:2405] "Pod admission denied" podUID="ad4ad2ac-f787-41e7-86f9-b1503bc8670b" pod="tigera-operator/tigera-operator-747864d56d-h95dm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:22.168528 containerd[1567]: time="2025-08-13T01:26:22.168485137Z" level=error msg="Failed to destroy network for sandbox \"4f6fb7119b7753ba4e8718aad025df715306c0dc3feaae50a32f26b368656de7\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:22.170490 containerd[1567]: time="2025-08-13T01:26:22.169796728Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"4f6fb7119b7753ba4e8718aad025df715306c0dc3feaae50a32f26b368656de7\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:22.172393 systemd[1]: run-netns-cni\x2d31a6c5fc\x2d80e3\x2d9a92\x2db18d\x2d9a8dfee85a80.mount: Deactivated successfully. Aug 13 01:26:22.173351 kubelet[2720]: E0813 01:26:22.172857 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"4f6fb7119b7753ba4e8718aad025df715306c0dc3feaae50a32f26b368656de7\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:22.173351 kubelet[2720]: E0813 01:26:22.172922 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"4f6fb7119b7753ba4e8718aad025df715306c0dc3feaae50a32f26b368656de7\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:26:22.173351 kubelet[2720]: E0813 01:26:22.173039 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"4f6fb7119b7753ba4e8718aad025df715306c0dc3feaae50a32f26b368656de7\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:26:22.173351 kubelet[2720]: E0813 01:26:22.173114 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"4f6fb7119b7753ba4e8718aad025df715306c0dc3feaae50a32f26b368656de7\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:26:22.258681 kubelet[2720]: I0813 01:26:22.258525 2720 kubelet.go:2405] "Pod admission denied" podUID="c52ccf36-f3e9-4646-841a-001e96986728" pod="tigera-operator/tigera-operator-747864d56d-m7bd2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:22.348000 kubelet[2720]: I0813 01:26:22.347864 2720 kubelet.go:2405] "Pod admission denied" podUID="7d9a4dff-ee87-4c9c-b273-28b71ebb110e" pod="tigera-operator/tigera-operator-747864d56d-zw7gd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:22.444825 kubelet[2720]: I0813 01:26:22.444774 2720 kubelet.go:2405] "Pod admission denied" podUID="e7b16fea-2811-4083-b570-04f22603b62b" pod="tigera-operator/tigera-operator-747864d56d-d7w6v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:22.648814 kubelet[2720]: I0813 01:26:22.648143 2720 kubelet.go:2405] "Pod admission denied" podUID="1f8db783-6397-41fc-8d21-2204289a941c" pod="tigera-operator/tigera-operator-747864d56d-d8psn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:22.746800 kubelet[2720]: I0813 01:26:22.746751 2720 kubelet.go:2405] "Pod admission denied" podUID="e63ee67e-e0a0-4cc1-9f7a-38e33d2f0640" pod="tigera-operator/tigera-operator-747864d56d-q6mhp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:22.795996 kubelet[2720]: I0813 01:26:22.795957 2720 kubelet.go:2405] "Pod admission denied" podUID="8c777c9e-7fad-468e-9daf-96fade145a33" pod="tigera-operator/tigera-operator-747864d56d-22q28" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:22.895630 kubelet[2720]: I0813 01:26:22.895581 2720 kubelet.go:2405] "Pod admission denied" podUID="f82cda88-5d76-4035-a042-55d40419e15f" pod="tigera-operator/tigera-operator-747864d56d-ngq8b" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:22.996310 kubelet[2720]: I0813 01:26:22.996268 2720 kubelet.go:2405] "Pod admission denied" podUID="5e340376-6890-407c-841d-d35d899610d5" pod="tigera-operator/tigera-operator-747864d56d-m8vh5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:23.096585 kubelet[2720]: I0813 01:26:23.096542 2720 kubelet.go:2405] "Pod admission denied" podUID="5a47635b-219e-4033-9526-b720cd3265d7" pod="tigera-operator/tigera-operator-747864d56d-tlz8g" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:23.196538 kubelet[2720]: I0813 01:26:23.196487 2720 kubelet.go:2405] "Pod admission denied" podUID="14686e7e-b65a-482f-a0ce-6c9161c23677" pod="tigera-operator/tigera-operator-747864d56d-gqdvr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:23.296133 kubelet[2720]: I0813 01:26:23.295793 2720 kubelet.go:2405] "Pod admission denied" podUID="e084e27d-be97-4f63-9382-a6e8928ea686" pod="tigera-operator/tigera-operator-747864d56d-bkzlv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:23.397100 kubelet[2720]: I0813 01:26:23.397046 2720 kubelet.go:2405] "Pod admission denied" podUID="934a7ecc-e08a-4f9e-8e75-e3d59ae4ad77" pod="tigera-operator/tigera-operator-747864d56d-5dh8w" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:23.495964 kubelet[2720]: I0813 01:26:23.495917 2720 kubelet.go:2405] "Pod admission denied" podUID="29e36e6b-4445-43f3-8e1f-d182e18b7058" pod="tigera-operator/tigera-operator-747864d56d-x5vr7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:23.597455 kubelet[2720]: I0813 01:26:23.597089 2720 kubelet.go:2405] "Pod admission denied" podUID="d4dbbfc0-6466-49a7-bbfd-c63800689025" pod="tigera-operator/tigera-operator-747864d56d-8bkpj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:23.695479 kubelet[2720]: I0813 01:26:23.695440 2720 kubelet.go:2405] "Pod admission denied" podUID="c26419d0-dd39-4f36-9de9-c1e6120ddab1" pod="tigera-operator/tigera-operator-747864d56d-h72kg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:23.897530 kubelet[2720]: I0813 01:26:23.897277 2720 kubelet.go:2405] "Pod admission denied" podUID="2f5df7e0-204f-44ba-b3ab-676390d22c9a" pod="tigera-operator/tigera-operator-747864d56d-l6tzm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:23.999501 kubelet[2720]: I0813 01:26:23.999458 2720 kubelet.go:2405] "Pod admission denied" podUID="d4e27ea3-6b2f-4cae-93be-9d1088669ff9" pod="tigera-operator/tigera-operator-747864d56d-8mj8n" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:24.074742 containerd[1567]: time="2025-08-13T01:26:24.074172945Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:26:24.106358 kubelet[2720]: I0813 01:26:24.106292 2720 kubelet.go:2405] "Pod admission denied" podUID="5b18a9f4-d503-4240-af72-0948b7f96a83" pod="tigera-operator/tigera-operator-747864d56d-76t9g" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:24.146064 containerd[1567]: time="2025-08-13T01:26:24.146011338Z" level=error msg="Failed to destroy network for sandbox \"3de56d6ccaabfd00d2f24dd3d01ffb7f82ededdafff2316b5687af277b051e47\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:24.148707 systemd[1]: run-netns-cni\x2dfb545fca\x2d1b9c\x2daff3\x2d488c\x2dc54f4baef7c1.mount: Deactivated successfully. Aug 13 01:26:24.149568 containerd[1567]: time="2025-08-13T01:26:24.148881070Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"3de56d6ccaabfd00d2f24dd3d01ffb7f82ededdafff2316b5687af277b051e47\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:24.150262 kubelet[2720]: E0813 01:26:24.149465 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3de56d6ccaabfd00d2f24dd3d01ffb7f82ededdafff2316b5687af277b051e47\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:24.150262 kubelet[2720]: E0813 01:26:24.150192 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3de56d6ccaabfd00d2f24dd3d01ffb7f82ededdafff2316b5687af277b051e47\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:26:24.150262 kubelet[2720]: E0813 01:26:24.150222 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3de56d6ccaabfd00d2f24dd3d01ffb7f82ededdafff2316b5687af277b051e47\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:26:24.150448 kubelet[2720]: E0813 01:26:24.150377 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"3de56d6ccaabfd00d2f24dd3d01ffb7f82ededdafff2316b5687af277b051e47\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:26:24.199610 kubelet[2720]: I0813 01:26:24.199558 2720 kubelet.go:2405] "Pod admission denied" podUID="8724c1ec-36b2-41e9-82d1-69b1e458877f" pod="tigera-operator/tigera-operator-747864d56d-9r52w" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:24.299191 kubelet[2720]: I0813 01:26:24.299143 2720 kubelet.go:2405] "Pod admission denied" podUID="885ad016-35fc-458e-bc45-be5b8cea186b" pod="tigera-operator/tigera-operator-747864d56d-d58fx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:24.498732 kubelet[2720]: I0813 01:26:24.498661 2720 kubelet.go:2405] "Pod admission denied" podUID="a230c35e-288a-4365-8eb0-aec16e1cb7e1" pod="tigera-operator/tigera-operator-747864d56d-m7mmj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:24.599529 kubelet[2720]: I0813 01:26:24.599470 2720 kubelet.go:2405] "Pod admission denied" podUID="465849d0-71e0-432a-843d-a0ee7c417351" pod="tigera-operator/tigera-operator-747864d56d-2zx9s" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:24.647786 kubelet[2720]: I0813 01:26:24.647749 2720 kubelet.go:2405] "Pod admission denied" podUID="9490f2c8-c396-42f8-ac4e-b57d2773b9a2" pod="tigera-operator/tigera-operator-747864d56d-njq9g" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:24.747649 kubelet[2720]: I0813 01:26:24.747609 2720 kubelet.go:2405] "Pod admission denied" podUID="e70ba3c5-f492-4152-9fec-6b51274e8d70" pod="tigera-operator/tigera-operator-747864d56d-jdnbg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:24.949645 kubelet[2720]: I0813 01:26:24.949245 2720 kubelet.go:2405] "Pod admission denied" podUID="7f6a5ec2-e991-4200-9e35-eba69dd3c941" pod="tigera-operator/tigera-operator-747864d56d-khzbc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:25.046173 kubelet[2720]: I0813 01:26:25.046132 2720 kubelet.go:2405] "Pod admission denied" podUID="22359d4e-56ea-422f-abde-1d06dc8f774f" pod="tigera-operator/tigera-operator-747864d56d-v7546" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:25.073858 containerd[1567]: time="2025-08-13T01:26:25.073823145Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:26:25.125814 containerd[1567]: time="2025-08-13T01:26:25.125519438Z" level=error msg="Failed to destroy network for sandbox \"ce5f14fc3cc8f726949a420b19552ecdf323d61bad98c21ba63d69ddc3e5bd5e\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:25.128332 containerd[1567]: time="2025-08-13T01:26:25.128270140Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"ce5f14fc3cc8f726949a420b19552ecdf323d61bad98c21ba63d69ddc3e5bd5e\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:25.128969 kubelet[2720]: E0813 01:26:25.128563 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ce5f14fc3cc8f726949a420b19552ecdf323d61bad98c21ba63d69ddc3e5bd5e\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:25.128969 kubelet[2720]: E0813 01:26:25.128627 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ce5f14fc3cc8f726949a420b19552ecdf323d61bad98c21ba63d69ddc3e5bd5e\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:25.128969 kubelet[2720]: E0813 01:26:25.128648 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ce5f14fc3cc8f726949a420b19552ecdf323d61bad98c21ba63d69ddc3e5bd5e\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:25.128969 kubelet[2720]: E0813 01:26:25.128692 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"ce5f14fc3cc8f726949a420b19552ecdf323d61bad98c21ba63d69ddc3e5bd5e\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:26:25.128849 systemd[1]: run-netns-cni\x2ddcede9e1\x2dcee8\x2d6e60\x2d034f\x2dffe59e4e0572.mount: Deactivated successfully. Aug 13 01:26:25.153094 kubelet[2720]: I0813 01:26:25.152704 2720 kubelet.go:2405] "Pod admission denied" podUID="367910fe-af84-4bce-9017-2d77a2c05c5f" pod="tigera-operator/tigera-operator-747864d56d-kf5zt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:25.247147 kubelet[2720]: I0813 01:26:25.247106 2720 kubelet.go:2405] "Pod admission denied" podUID="0fe5275d-cddf-4e7c-80fc-f40173120206" pod="tigera-operator/tigera-operator-747864d56d-cs2sg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:25.298873 kubelet[2720]: I0813 01:26:25.298826 2720 kubelet.go:2405] "Pod admission denied" podUID="781ede12-a80e-4841-bb9e-56ee0545729c" pod="tigera-operator/tigera-operator-747864d56d-pl25r" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:25.396636 kubelet[2720]: I0813 01:26:25.396588 2720 kubelet.go:2405] "Pod admission denied" podUID="e9f73e35-d710-4703-95d7-3bf3892cb6d2" pod="tigera-operator/tigera-operator-747864d56d-kmgpr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:25.496882 kubelet[2720]: I0813 01:26:25.496834 2720 kubelet.go:2405] "Pod admission denied" podUID="f2e4f9b5-35b1-42af-9fb5-8551a6feb145" pod="tigera-operator/tigera-operator-747864d56d-hhdkg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:25.597353 kubelet[2720]: I0813 01:26:25.597010 2720 kubelet.go:2405] "Pod admission denied" podUID="c970e9a8-3979-4ec6-b38b-d7bdbeb9aac0" pod="tigera-operator/tigera-operator-747864d56d-hkcb5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:25.800364 kubelet[2720]: I0813 01:26:25.800306 2720 kubelet.go:2405] "Pod admission denied" podUID="f5259a3f-ae7d-44b0-8075-1804e28d8a6f" pod="tigera-operator/tigera-operator-747864d56d-fw92b" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:25.898829 kubelet[2720]: I0813 01:26:25.898726 2720 kubelet.go:2405] "Pod admission denied" podUID="daed8059-0c49-4886-ae10-15dd12b23874" pod="tigera-operator/tigera-operator-747864d56d-rtq4h" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:26.004274 kubelet[2720]: I0813 01:26:26.004217 2720 kubelet.go:2405] "Pod admission denied" podUID="b96f0a4c-d299-43c7-b3ba-5576cf85ff02" pod="tigera-operator/tigera-operator-747864d56d-tl8wr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:26.098021 kubelet[2720]: I0813 01:26:26.097977 2720 kubelet.go:2405] "Pod admission denied" podUID="b303658a-1cd1-4eaa-a02d-dac29880c0ac" pod="tigera-operator/tigera-operator-747864d56d-tnctc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:26.197969 kubelet[2720]: I0813 01:26:26.197444 2720 kubelet.go:2405] "Pod admission denied" podUID="6b999992-0b1f-4914-894f-8a5b75c75641" pod="tigera-operator/tigera-operator-747864d56d-24r64" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:26.296573 kubelet[2720]: I0813 01:26:26.296526 2720 kubelet.go:2405] "Pod admission denied" podUID="fae90cdc-29ac-4a6e-a777-33d064c6a0ca" pod="tigera-operator/tigera-operator-747864d56d-9xsfk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:26.397787 kubelet[2720]: I0813 01:26:26.397747 2720 kubelet.go:2405] "Pod admission denied" podUID="164fa773-951e-417c-ad30-130b5ceab9ff" pod="tigera-operator/tigera-operator-747864d56d-qpskw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:26.496170 kubelet[2720]: I0813 01:26:26.496128 2720 kubelet.go:2405] "Pod admission denied" podUID="82fbb537-a556-44eb-9af6-c55ce4998814" pod="tigera-operator/tigera-operator-747864d56d-bvnln" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:26.602225 kubelet[2720]: I0813 01:26:26.601380 2720 kubelet.go:2405] "Pod admission denied" podUID="cb8c1e61-f93b-4183-84d8-a1568cf8fa0c" pod="tigera-operator/tigera-operator-747864d56d-f5qhg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:26.696859 kubelet[2720]: I0813 01:26:26.696807 2720 kubelet.go:2405] "Pod admission denied" podUID="9bf4f124-76bd-45ad-9e7d-b952cd421792" pod="tigera-operator/tigera-operator-747864d56d-5jqwr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:26.746502 kubelet[2720]: I0813 01:26:26.746394 2720 kubelet.go:2405] "Pod admission denied" podUID="9ebea3df-9fce-45e9-8a3d-6162a36f578b" pod="tigera-operator/tigera-operator-747864d56d-2n7s6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:26.845751 kubelet[2720]: I0813 01:26:26.845693 2720 kubelet.go:2405] "Pod admission denied" podUID="cc76a865-eb3d-43de-8505-e969b6774246" pod="tigera-operator/tigera-operator-747864d56d-nx7f2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:27.046941 kubelet[2720]: I0813 01:26:27.046823 2720 kubelet.go:2405] "Pod admission denied" podUID="a9558167-d64f-472a-8628-a24860d44e8b" pod="tigera-operator/tigera-operator-747864d56d-jx7sc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:27.074560 containerd[1567]: time="2025-08-13T01:26:27.074526662Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/node:v3.30.2\"" Aug 13 01:26:27.146156 kubelet[2720]: I0813 01:26:27.146112 2720 kubelet.go:2405] "Pod admission denied" podUID="abf9c2d8-2609-4dd2-b883-6ccd86bf0aa3" pod="tigera-operator/tigera-operator-747864d56d-cqc4z" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:27.249733 kubelet[2720]: I0813 01:26:27.248010 2720 kubelet.go:2405] "Pod admission denied" podUID="25ef0420-c271-4936-9110-d9cdb8e35dcb" pod="tigera-operator/tigera-operator-747864d56d-rp9cv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:27.346348 kubelet[2720]: I0813 01:26:27.346129 2720 kubelet.go:2405] "Pod admission denied" podUID="ab638ec8-19a5-415e-b75a-d7c2e3876841" pod="tigera-operator/tigera-operator-747864d56d-9qm54" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:27.394666 kubelet[2720]: I0813 01:26:27.394625 2720 kubelet.go:2405] "Pod admission denied" podUID="956f1aa1-07ad-4744-a49a-72d5089bd337" pod="tigera-operator/tigera-operator-747864d56d-hzpfx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:27.495565 kubelet[2720]: I0813 01:26:27.495484 2720 kubelet.go:2405] "Pod admission denied" podUID="963f9113-00f6-4bbf-b3be-4717cc16b33f" pod="tigera-operator/tigera-operator-747864d56d-pjzsj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:27.590736 kubelet[2720]: I0813 01:26:27.590656 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:26:27.590736 kubelet[2720]: I0813 01:26:27.590689 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:26:27.594110 kubelet[2720]: I0813 01:26:27.594023 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:26:27.604469 kubelet[2720]: I0813 01:26:27.604408 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:26:27.604631 kubelet[2720]: I0813 01:26:27.604615 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/csi-node-driver-8bzzs","calico-system/calico-node-2p8hj","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:26:27.604855 kubelet[2720]: E0813 01:26:27.604739 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:26:27.604855 kubelet[2720]: E0813 01:26:27.604753 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:27.604855 kubelet[2720]: E0813 01:26:27.604760 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:26:27.604855 kubelet[2720]: E0813 01:26:27.604767 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:27.604855 kubelet[2720]: E0813 01:26:27.604774 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:26:27.604855 kubelet[2720]: E0813 01:26:27.604784 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:26:27.604855 kubelet[2720]: E0813 01:26:27.604792 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:26:27.604855 kubelet[2720]: E0813 01:26:27.604800 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:26:27.604855 kubelet[2720]: E0813 01:26:27.604809 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:26:27.604855 kubelet[2720]: E0813 01:26:27.604817 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:26:27.604855 kubelet[2720]: I0813 01:26:27.604826 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:26:27.605203 kubelet[2720]: I0813 01:26:27.605165 2720 kubelet.go:2405] "Pod admission denied" podUID="409407fb-f2d5-4e24-b7a6-6a62f02c413a" pod="tigera-operator/tigera-operator-747864d56d-trvcc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:27.657073 kubelet[2720]: I0813 01:26:27.657016 2720 kubelet.go:2405] "Pod admission denied" podUID="6b0c84f0-9c6f-4763-b8fb-f9d86040b3a3" pod="tigera-operator/tigera-operator-747864d56d-r7x5t" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:27.751736 kubelet[2720]: I0813 01:26:27.751546 2720 kubelet.go:2405] "Pod admission denied" podUID="a5a58a4a-3db3-49e1-acbe-7327503e6ed8" pod="tigera-operator/tigera-operator-747864d56d-vhpj6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:27.860544 kubelet[2720]: I0813 01:26:27.860173 2720 kubelet.go:2405] "Pod admission denied" podUID="56646ab3-c0a6-4892-b244-32bcb088a7c9" pod="tigera-operator/tigera-operator-747864d56d-lzt6q" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:27.950963 kubelet[2720]: I0813 01:26:27.950846 2720 kubelet.go:2405] "Pod admission denied" podUID="826f7710-0876-4a32-8d59-1b68c6358545" pod="tigera-operator/tigera-operator-747864d56d-rsxnx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:28.051070 kubelet[2720]: I0813 01:26:28.051021 2720 kubelet.go:2405] "Pod admission denied" podUID="49f168a7-2848-493b-a022-dd35ee2026cc" pod="tigera-operator/tigera-operator-747864d56d-kzmxf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:28.187307 kubelet[2720]: I0813 01:26:28.184868 2720 kubelet.go:2405] "Pod admission denied" podUID="a8939bb9-141c-4bbc-bc01-c2ace2e273ab" pod="tigera-operator/tigera-operator-747864d56d-hg2lh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:28.251074 kubelet[2720]: I0813 01:26:28.250999 2720 kubelet.go:2405] "Pod admission denied" podUID="2e6eaae9-9462-4df7-b82d-35451e42023c" pod="tigera-operator/tigera-operator-747864d56d-cjr9c" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:28.359505 kubelet[2720]: I0813 01:26:28.358487 2720 kubelet.go:2405] "Pod admission denied" podUID="c2ea9193-ee9e-4c1e-9fe6-6c4bda638789" pod="tigera-operator/tigera-operator-747864d56d-kcmnw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:28.558875 kubelet[2720]: I0813 01:26:28.558811 2720 kubelet.go:2405] "Pod admission denied" podUID="a4a1bc8a-1137-448f-a980-0c7d1ba76258" pod="tigera-operator/tigera-operator-747864d56d-hhf9t" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:28.663986 kubelet[2720]: I0813 01:26:28.663947 2720 kubelet.go:2405] "Pod admission denied" podUID="3e7b8d47-b0b0-4fa7-b393-38c133f66841" pod="tigera-operator/tigera-operator-747864d56d-j6hd2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:28.750849 kubelet[2720]: I0813 01:26:28.750815 2720 kubelet.go:2405] "Pod admission denied" podUID="dd48a859-5598-4b98-b120-d1e309710f87" pod="tigera-operator/tigera-operator-747864d56d-n8ltg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:28.855621 kubelet[2720]: I0813 01:26:28.854707 2720 kubelet.go:2405] "Pod admission denied" podUID="329498e1-d1f6-49fb-a81a-9b3b8f68c4e2" pod="tigera-operator/tigera-operator-747864d56d-8mz8j" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:28.964548 kubelet[2720]: I0813 01:26:28.964494 2720 kubelet.go:2405] "Pod admission denied" podUID="53da9f03-0190-424b-9f1b-51807ddf42ca" pod="tigera-operator/tigera-operator-747864d56d-jj9bx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:29.053420 kubelet[2720]: I0813 01:26:29.053380 2720 kubelet.go:2405] "Pod admission denied" podUID="3f52deb7-f1bd-4013-af6e-e5d4484901de" pod="tigera-operator/tigera-operator-747864d56d-wzgrt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:29.074068 kubelet[2720]: E0813 01:26:29.073339 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:26:29.074602 containerd[1567]: time="2025-08-13T01:26:29.074574858Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:26:29.153079 kubelet[2720]: I0813 01:26:29.152819 2720 kubelet.go:2405] "Pod admission denied" podUID="5f001516-5b9c-443a-94f2-dd17f411c2b7" pod="tigera-operator/tigera-operator-747864d56d-qnjts" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:29.171303 containerd[1567]: time="2025-08-13T01:26:29.171232614Z" level=error msg="Failed to destroy network for sandbox \"f75d68cea236635772866ab4b62dd05840572c422ca5d465f0eaa4139d05a8f9\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:29.173692 systemd[1]: run-netns-cni\x2db07c1807\x2d021b\x2d33e2\x2dfc70\x2d283ada90cbaf.mount: Deactivated successfully. Aug 13 01:26:29.176407 containerd[1567]: time="2025-08-13T01:26:29.175586875Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"f75d68cea236635772866ab4b62dd05840572c422ca5d465f0eaa4139d05a8f9\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:29.176760 kubelet[2720]: E0813 01:26:29.176683 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f75d68cea236635772866ab4b62dd05840572c422ca5d465f0eaa4139d05a8f9\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:29.176760 kubelet[2720]: E0813 01:26:29.176754 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f75d68cea236635772866ab4b62dd05840572c422ca5d465f0eaa4139d05a8f9\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:29.176877 kubelet[2720]: E0813 01:26:29.176776 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f75d68cea236635772866ab4b62dd05840572c422ca5d465f0eaa4139d05a8f9\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:29.176877 kubelet[2720]: E0813 01:26:29.176836 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"f75d68cea236635772866ab4b62dd05840572c422ca5d465f0eaa4139d05a8f9\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:26:29.316417 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount1682018348.mount: Deactivated successfully. Aug 13 01:26:29.319042 containerd[1567]: time="2025-08-13T01:26:29.319005713Z" level=error msg="PullImage \"ghcr.io/flatcar/calico/node:v3.30.2\" failed" error="rpc error: code = Unknown desc = failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1682018348: write /var/lib/containerd/tmpmounts/containerd-mount1682018348/usr/bin/calico-node: no space left on device" Aug 13 01:26:29.319300 containerd[1567]: time="2025-08-13T01:26:29.319190813Z" level=info msg="stop pulling image ghcr.io/flatcar/calico/node:v3.30.2: active requests=0, bytes read=158500163" Aug 13 01:26:29.319552 kubelet[2720]: E0813 01:26:29.319491 2720 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1682018348: write /var/lib/containerd/tmpmounts/containerd-mount1682018348/usr/bin/calico-node: no space left on device" image="ghcr.io/flatcar/calico/node:v3.30.2" Aug 13 01:26:29.319644 kubelet[2720]: E0813 01:26:29.319556 2720 kuberuntime_image.go:42] "Failed to pull image" err="failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1682018348: write /var/lib/containerd/tmpmounts/containerd-mount1682018348/usr/bin/calico-node: no space left on device" image="ghcr.io/flatcar/calico/node:v3.30.2" Aug 13 01:26:29.319926 kubelet[2720]: E0813 01:26:29.319858 2720 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:calico-node,Image:ghcr.io/flatcar/calico/node:v3.30.2,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:DATASTORE_TYPE,Value:kubernetes,ValueFrom:nil,},EnvVar{Name:WAIT_FOR_DATASTORE,Value:true,ValueFrom:nil,},EnvVar{Name:CLUSTER_TYPE,Value:k8s,operator,bgp,ValueFrom:nil,},EnvVar{Name:CALICO_DISABLE_FILE_LOGGING,Value:false,ValueFrom:nil,},EnvVar{Name:FELIX_DEFAULTENDPOINTTOHOSTACTION,Value:ACCEPT,ValueFrom:nil,},EnvVar{Name:FELIX_HEALTHENABLED,Value:true,ValueFrom:nil,},EnvVar{Name:FELIX_HEALTHPORT,Value:9099,ValueFrom:nil,},EnvVar{Name:NODENAME,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:spec.nodeName,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:FELIX_TYPHAK8SNAMESPACE,Value:calico-system,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHAK8SSERVICENAME,Value:calico-typha,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHACAFILE,Value:/etc/pki/tls/certs/tigera-ca-bundle.crt,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHACERTFILE,Value:/node-certs/tls.crt,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHAKEYFILE,Value:/node-certs/tls.key,ValueFrom:nil,},EnvVar{Name:NO_DEFAULT_POOLS,Value:true,ValueFrom:nil,},EnvVar{Name:FELIX_FLOWLOGSGOLDMANESERVER,Value:goldmane.calico-system.svc:7443,ValueFrom:nil,},EnvVar{Name:FELIX_FLOWLOGSFLUSHINTERVAL,Value:15,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHACN,Value:typha-server,ValueFrom:nil,},EnvVar{Name:CALICO_MANAGE_CNI,Value:true,ValueFrom:nil,},EnvVar{Name:CALICO_NETWORKING_BACKEND,Value:bird,ValueFrom:nil,},EnvVar{Name:IP,Value:autodetect,ValueFrom:nil,},EnvVar{Name:IP_AUTODETECTION_METHOD,Value:first-found,ValueFrom:nil,},EnvVar{Name:IP6,Value:none,ValueFrom:nil,},EnvVar{Name:FELIX_IPV6SUPPORT,Value:false,ValueFrom:nil,},EnvVar{Name:KUBERNETES_SERVICE_HOST,Value:10.96.0.1,ValueFrom:nil,},EnvVar{Name:KUBERNETES_SERVICE_PORT,Value:443,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:cni-log-dir,ReadOnly:false,MountPath:/var/log/calico/cni,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:cni-net-dir,ReadOnly:false,MountPath:/host/etc/cni/net.d,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:lib-modules,ReadOnly:true,MountPath:/lib/modules,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:node-certs,ReadOnly:true,MountPath:/node-certs,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:policysync,ReadOnly:false,MountPath:/var/run/nodeagent,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:tigera-ca-bundle,ReadOnly:true,MountPath:/etc/pki/tls/certs,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:tigera-ca-bundle,ReadOnly:true,MountPath:/etc/pki/tls/cert.pem,SubPath:ca-bundle.crt,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:var-lib-calico,ReadOnly:false,MountPath:/var/lib/calico,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:var-run-calico,ReadOnly:false,MountPath:/var/run/calico,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:xtables-lock,ReadOnly:false,MountPath:/run/xtables.lock,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-jl8ww,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/liveness,Port:{0 9099 },Host:localhost,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:10,PeriodSeconds:60,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:&ExecAction{Command:[/bin/calico-node -bird-ready -felix-ready],},HTTPGet:nil,TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:10,PeriodSeconds:30,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:&Lifecycle{PostStart:nil,PreStop:&LifecycleHandler{Exec:&ExecAction{Command:[/bin/calico-node -shutdown],},HTTPGet:nil,TCPSocket:nil,Sleep:nil,},StopSignal:nil,},TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:*true,SELinuxOptions:nil,RunAsUser:*0,RunAsNonRoot:*false,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*true,RunAsGroup:*0,ProcMount:nil,WindowsOptions:nil,SeccompProfile:&SeccompProfile{Type:RuntimeDefault,LocalhostProfile:nil,},AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod calico-node-2p8hj_calico-system(4f334296-e8c0-427e-a090-d8fbd3582f26): ErrImagePull: failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1682018348: write /var/lib/containerd/tmpmounts/containerd-mount1682018348/usr/bin/calico-node: no space left on device" logger="UnhandledError" Aug 13 01:26:29.322000 kubelet[2720]: E0813 01:26:29.321869 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"calico-node\" with ErrImagePull: \"failed to pull and unpack image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1682018348: write /var/lib/containerd/tmpmounts/containerd-mount1682018348/usr/bin/calico-node: no space left on device\"" pod="calico-system/calico-node-2p8hj" podUID="4f334296-e8c0-427e-a090-d8fbd3582f26" Aug 13 01:26:29.349205 kubelet[2720]: I0813 01:26:29.349154 2720 kubelet.go:2405] "Pod admission denied" podUID="d934f78f-430b-421b-95ab-c84b1c9806f4" pod="tigera-operator/tigera-operator-747864d56d-tp29q" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:29.447054 kubelet[2720]: I0813 01:26:29.446932 2720 kubelet.go:2405] "Pod admission denied" podUID="28d77ac6-fa33-4f1f-82c5-2d1f9e00e2c6" pod="tigera-operator/tigera-operator-747864d56d-kqw9t" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:29.555752 kubelet[2720]: I0813 01:26:29.554699 2720 kubelet.go:2405] "Pod admission denied" podUID="f3cf9e13-7524-49ed-8b03-c0567d746508" pod="tigera-operator/tigera-operator-747864d56d-5lmlz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:29.648067 kubelet[2720]: I0813 01:26:29.648019 2720 kubelet.go:2405] "Pod admission denied" podUID="00f0498b-0fae-4fe2-bba3-b6e204e3872c" pod="tigera-operator/tigera-operator-747864d56d-8swtm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:29.701956 kubelet[2720]: I0813 01:26:29.700972 2720 kubelet.go:2405] "Pod admission denied" podUID="8b08afe1-4883-4e6c-8257-914d3f6c1f19" pod="tigera-operator/tigera-operator-747864d56d-sfc65" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:29.796628 kubelet[2720]: I0813 01:26:29.796579 2720 kubelet.go:2405] "Pod admission denied" podUID="7dc01a40-3e97-4559-ae0d-565149624203" pod="tigera-operator/tigera-operator-747864d56d-cls7z" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:29.898238 kubelet[2720]: I0813 01:26:29.898189 2720 kubelet.go:2405] "Pod admission denied" podUID="ca3c5393-5ead-4b27-a4e8-77a145ed1f68" pod="tigera-operator/tigera-operator-747864d56d-n86zg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:29.996693 kubelet[2720]: I0813 01:26:29.996658 2720 kubelet.go:2405] "Pod admission denied" podUID="75c8f3f8-b092-4839-92cb-f445033523c0" pod="tigera-operator/tigera-operator-747864d56d-x57kw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:30.099895 kubelet[2720]: I0813 01:26:30.099843 2720 kubelet.go:2405] "Pod admission denied" podUID="bb81c5c0-9d59-438c-b006-812553ca22e2" pod="tigera-operator/tigera-operator-747864d56d-6n7sb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:30.198877 kubelet[2720]: I0813 01:26:30.198822 2720 kubelet.go:2405] "Pod admission denied" podUID="f83132a7-ec95-4198-a8ae-47913311f4df" pod="tigera-operator/tigera-operator-747864d56d-zdf6d" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:30.298761 kubelet[2720]: I0813 01:26:30.297992 2720 kubelet.go:2405] "Pod admission denied" podUID="d3fbbebe-2d9e-4dd2-985a-0cc872a162d6" pod="tigera-operator/tigera-operator-747864d56d-nd8vq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:30.436741 kubelet[2720]: I0813 01:26:30.435122 2720 kubelet.go:2405] "Pod admission denied" podUID="87a1a1e3-9b57-40e8-be63-d97fa60a308a" pod="tigera-operator/tigera-operator-747864d56d-fm7t7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:30.598216 kubelet[2720]: I0813 01:26:30.598080 2720 kubelet.go:2405] "Pod admission denied" podUID="54da62c2-14a7-4970-83c8-4cff7cd6c237" pod="tigera-operator/tigera-operator-747864d56d-cqkdj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:30.699555 kubelet[2720]: I0813 01:26:30.699498 2720 kubelet.go:2405] "Pod admission denied" podUID="6a883043-bf44-4a3f-80ed-99295f87cf46" pod="tigera-operator/tigera-operator-747864d56d-55dpq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:30.797419 kubelet[2720]: I0813 01:26:30.797389 2720 kubelet.go:2405] "Pod admission denied" podUID="be1c094d-cccf-4bf8-97b4-73a4a1d6e894" pod="tigera-operator/tigera-operator-747864d56d-gnkp9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:30.899342 kubelet[2720]: I0813 01:26:30.898271 2720 kubelet.go:2405] "Pod admission denied" podUID="fcf85e3e-93bc-4e34-8329-a91218f74581" pod="tigera-operator/tigera-operator-747864d56d-dczmb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:30.996018 kubelet[2720]: I0813 01:26:30.995985 2720 kubelet.go:2405] "Pod admission denied" podUID="70e8e4a1-99bf-4a38-a998-614d0fcac726" pod="tigera-operator/tigera-operator-747864d56d-cp7ws" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:31.202155 kubelet[2720]: I0813 01:26:31.201900 2720 kubelet.go:2405] "Pod admission denied" podUID="e7181d90-e814-4847-98b8-e4136d19c733" pod="tigera-operator/tigera-operator-747864d56d-hxbb2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:31.299168 kubelet[2720]: I0813 01:26:31.299107 2720 kubelet.go:2405] "Pod admission denied" podUID="8c40b96b-1a6a-4b2d-8aea-9a24d6230f80" pod="tigera-operator/tigera-operator-747864d56d-mblrt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:31.398604 kubelet[2720]: I0813 01:26:31.398568 2720 kubelet.go:2405] "Pod admission denied" podUID="52002861-d24c-417f-b569-5495dda4d445" pod="tigera-operator/tigera-operator-747864d56d-2w4z7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:31.497757 kubelet[2720]: I0813 01:26:31.497690 2720 kubelet.go:2405] "Pod admission denied" podUID="594921b7-7f96-4145-9ec7-5059b4585288" pod="tigera-operator/tigera-operator-747864d56d-5pjqt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:31.600930 kubelet[2720]: I0813 01:26:31.600871 2720 kubelet.go:2405] "Pod admission denied" podUID="0da70508-5a02-4653-aa18-f3b7d335501e" pod="tigera-operator/tigera-operator-747864d56d-4nxcv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:31.697365 kubelet[2720]: I0813 01:26:31.697322 2720 kubelet.go:2405] "Pod admission denied" podUID="cbf2b982-5aa7-4c0b-8250-36b5c14374e0" pod="tigera-operator/tigera-operator-747864d56d-4skkp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:31.796871 kubelet[2720]: I0813 01:26:31.796303 2720 kubelet.go:2405] "Pod admission denied" podUID="dd09f5e4-0982-469c-be33-c861130bdcc4" pod="tigera-operator/tigera-operator-747864d56d-gg29l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:31.897931 kubelet[2720]: I0813 01:26:31.897866 2720 kubelet.go:2405] "Pod admission denied" podUID="17f59521-8493-4ae5-ab50-1779d49d0a8a" pod="tigera-operator/tigera-operator-747864d56d-wxf4q" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:31.998672 kubelet[2720]: I0813 01:26:31.998625 2720 kubelet.go:2405] "Pod admission denied" podUID="5842a6e1-3c9a-43e9-9657-d63e499dbee5" pod="tigera-operator/tigera-operator-747864d56d-9sptx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:32.096937 kubelet[2720]: I0813 01:26:32.096811 2720 kubelet.go:2405] "Pod admission denied" podUID="1f276b64-c5ed-4c43-b9ca-1ad3f9c5a957" pod="tigera-operator/tigera-operator-747864d56d-ttbl5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:32.147197 kubelet[2720]: I0813 01:26:32.147149 2720 kubelet.go:2405] "Pod admission denied" podUID="066d83b1-0e17-44dc-9fc9-e95a75dc21f9" pod="tigera-operator/tigera-operator-747864d56d-s5s25" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:32.248685 kubelet[2720]: I0813 01:26:32.248634 2720 kubelet.go:2405] "Pod admission denied" podUID="8818ddab-591f-4c9c-b56e-c7f25e24ad7e" pod="tigera-operator/tigera-operator-747864d56d-gjfrq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:32.348824 kubelet[2720]: I0813 01:26:32.347737 2720 kubelet.go:2405] "Pod admission denied" podUID="0288ddb4-fcf4-4bb0-83c4-9a4322f395e8" pod="tigera-operator/tigera-operator-747864d56d-fdnpd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:32.448902 kubelet[2720]: I0813 01:26:32.448848 2720 kubelet.go:2405] "Pod admission denied" podUID="fd2737ec-211e-444e-afbb-571ac93a8923" pod="tigera-operator/tigera-operator-747864d56d-vsq65" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:32.650012 kubelet[2720]: I0813 01:26:32.649451 2720 kubelet.go:2405] "Pod admission denied" podUID="9c0ed844-917d-47e6-98c8-ba1830478b5b" pod="tigera-operator/tigera-operator-747864d56d-2knnz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:32.746180 kubelet[2720]: I0813 01:26:32.746136 2720 kubelet.go:2405] "Pod admission denied" podUID="c012a3bc-245a-4233-a6fa-a2ecd27a4b44" pod="tigera-operator/tigera-operator-747864d56d-zmrcs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:32.847538 kubelet[2720]: I0813 01:26:32.847458 2720 kubelet.go:2405] "Pod admission denied" podUID="8e1ecfb0-53e0-427d-9df0-aee1b47a0399" pod="tigera-operator/tigera-operator-747864d56d-72q9k" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:32.949271 kubelet[2720]: I0813 01:26:32.948599 2720 kubelet.go:2405] "Pod admission denied" podUID="c7a2a0f8-a9c5-44aa-b30b-184552dfc225" pod="tigera-operator/tigera-operator-747864d56d-r8zpf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:32.999188 kubelet[2720]: I0813 01:26:32.999137 2720 kubelet.go:2405] "Pod admission denied" podUID="b27588c9-f91f-493b-ab5b-a8db82d2647c" pod="tigera-operator/tigera-operator-747864d56d-jn75f" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:33.098205 kubelet[2720]: I0813 01:26:33.098151 2720 kubelet.go:2405] "Pod admission denied" podUID="125b1547-2209-4b60-81f8-321b3113223b" pod="tigera-operator/tigera-operator-747864d56d-pcf7j" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:33.198502 kubelet[2720]: I0813 01:26:33.198440 2720 kubelet.go:2405] "Pod admission denied" podUID="8c2c563b-1589-4486-9bb3-199451fd538c" pod="tigera-operator/tigera-operator-747864d56d-qqtmk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:33.296786 kubelet[2720]: I0813 01:26:33.296746 2720 kubelet.go:2405] "Pod admission denied" podUID="11afbd0d-7ca7-4c59-bf7f-316bb7c37434" pod="tigera-operator/tigera-operator-747864d56d-hpdtq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:33.396708 kubelet[2720]: I0813 01:26:33.396663 2720 kubelet.go:2405] "Pod admission denied" podUID="1ed9373f-557d-4d72-8ea1-5913d46e1a34" pod="tigera-operator/tigera-operator-747864d56d-xn8pj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:33.447609 kubelet[2720]: I0813 01:26:33.447559 2720 kubelet.go:2405] "Pod admission denied" podUID="b5aa3c6d-e089-4253-93b2-ce519ce50cdf" pod="tigera-operator/tigera-operator-747864d56d-rsf7l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:33.546671 kubelet[2720]: I0813 01:26:33.546620 2720 kubelet.go:2405] "Pod admission denied" podUID="73f39889-520d-4d11-b690-0accf32a43d4" pod="tigera-operator/tigera-operator-747864d56d-prvgv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:33.657065 kubelet[2720]: I0813 01:26:33.656399 2720 kubelet.go:2405] "Pod admission denied" podUID="18da5062-6df8-4796-a3f4-ce01862106ca" pod="tigera-operator/tigera-operator-747864d56d-52tpw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:33.746295 kubelet[2720]: I0813 01:26:33.746252 2720 kubelet.go:2405] "Pod admission denied" podUID="9afb2122-b957-4021-8b32-cd8f75b33d58" pod="tigera-operator/tigera-operator-747864d56d-872fd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:33.847877 kubelet[2720]: I0813 01:26:33.847822 2720 kubelet.go:2405] "Pod admission denied" podUID="2166acfc-21e3-4767-85cc-01e6d5be940f" pod="tigera-operator/tigera-operator-747864d56d-gk94s" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:33.947575 kubelet[2720]: I0813 01:26:33.947455 2720 kubelet.go:2405] "Pod admission denied" podUID="92c4e84d-0921-4bf0-be41-2f947d24e015" pod="tigera-operator/tigera-operator-747864d56d-xrk9f" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:34.148167 kubelet[2720]: I0813 01:26:34.148124 2720 kubelet.go:2405] "Pod admission denied" podUID="fe4c24b2-5d67-41ef-94cb-8abc029978ac" pod="tigera-operator/tigera-operator-747864d56d-42m9n" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:34.247921 kubelet[2720]: I0813 01:26:34.247880 2720 kubelet.go:2405] "Pod admission denied" podUID="0e64ca2f-5b9d-416d-913d-fddcb6382980" pod="tigera-operator/tigera-operator-747864d56d-znt5h" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:34.296917 kubelet[2720]: I0813 01:26:34.296869 2720 kubelet.go:2405] "Pod admission denied" podUID="a4eaea92-776b-4f95-8314-817ea8315d55" pod="tigera-operator/tigera-operator-747864d56d-lhl78" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:34.397698 kubelet[2720]: I0813 01:26:34.397648 2720 kubelet.go:2405] "Pod admission denied" podUID="85a6ec1f-885f-4594-94be-0b2c3dc11de1" pod="tigera-operator/tigera-operator-747864d56d-tn7n5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:34.497555 kubelet[2720]: I0813 01:26:34.497511 2720 kubelet.go:2405] "Pod admission denied" podUID="6d510e7f-e0ea-4d63-bad4-d3113f3ecde7" pod="tigera-operator/tigera-operator-747864d56d-f98gb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:34.601201 kubelet[2720]: I0813 01:26:34.600599 2720 kubelet.go:2405] "Pod admission denied" podUID="7c3fd95c-d9be-4979-8288-6ca9f08c4890" pod="tigera-operator/tigera-operator-747864d56d-75nwm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:34.697684 kubelet[2720]: I0813 01:26:34.697642 2720 kubelet.go:2405] "Pod admission denied" podUID="5901d644-7ce7-48bf-9db7-22c4d0d25b9a" pod="tigera-operator/tigera-operator-747864d56d-f4xrd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:34.796162 kubelet[2720]: I0813 01:26:34.796120 2720 kubelet.go:2405] "Pod admission denied" podUID="a0e20939-8de4-4083-81f0-c6482ec5c670" pod="tigera-operator/tigera-operator-747864d56d-l7znn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:34.897582 kubelet[2720]: I0813 01:26:34.897466 2720 kubelet.go:2405] "Pod admission denied" podUID="b94249e0-b5e0-4643-91d7-11ffc0a6f1aa" pod="tigera-operator/tigera-operator-747864d56d-jz49j" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:34.998578 kubelet[2720]: I0813 01:26:34.998526 2720 kubelet.go:2405] "Pod admission denied" podUID="a63828b7-57f0-442e-aa2c-8a854b6d389c" pod="tigera-operator/tigera-operator-747864d56d-q52tn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:35.197797 kubelet[2720]: I0813 01:26:35.197677 2720 kubelet.go:2405] "Pod admission denied" podUID="86eef137-310e-4f42-88db-377be00f6a0e" pod="tigera-operator/tigera-operator-747864d56d-bh7lm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:35.297821 kubelet[2720]: I0813 01:26:35.297774 2720 kubelet.go:2405] "Pod admission denied" podUID="27d8d2e0-c59c-4906-a14c-d49ce86d0de8" pod="tigera-operator/tigera-operator-747864d56d-p4pbj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:35.346090 kubelet[2720]: I0813 01:26:35.346049 2720 kubelet.go:2405] "Pod admission denied" podUID="959d4875-cc1f-4372-8552-5291ccb4f9fa" pod="tigera-operator/tigera-operator-747864d56d-k5s28" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:35.449622 kubelet[2720]: I0813 01:26:35.449449 2720 kubelet.go:2405] "Pod admission denied" podUID="2894ac03-d009-4697-9820-f07ce9183633" pod="tigera-operator/tigera-operator-747864d56d-htgn7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:35.649907 kubelet[2720]: I0813 01:26:35.649866 2720 kubelet.go:2405] "Pod admission denied" podUID="0027b4b3-a727-45c0-a576-613a30a4d191" pod="tigera-operator/tigera-operator-747864d56d-kfwsf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:35.750517 kubelet[2720]: I0813 01:26:35.750467 2720 kubelet.go:2405] "Pod admission denied" podUID="d59363e5-6cc7-45dc-9636-948c3c8aaa30" pod="tigera-operator/tigera-operator-747864d56d-vw62l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:35.850546 kubelet[2720]: I0813 01:26:35.850500 2720 kubelet.go:2405] "Pod admission denied" podUID="c9d8c51c-564d-45f7-ac97-f4febe70732a" pod="tigera-operator/tigera-operator-747864d56d-65rwx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:35.947193 kubelet[2720]: I0813 01:26:35.947149 2720 kubelet.go:2405] "Pod admission denied" podUID="bc303668-12b6-415c-a8de-6790425d2a46" pod="tigera-operator/tigera-operator-747864d56d-c89zw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:36.048915 kubelet[2720]: I0813 01:26:36.048478 2720 kubelet.go:2405] "Pod admission denied" podUID="4659b611-6cc0-483f-bc08-ece7adb6b871" pod="tigera-operator/tigera-operator-747864d56d-qbsmc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:36.074361 containerd[1567]: time="2025-08-13T01:26:36.074075521Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:26:36.137941 containerd[1567]: time="2025-08-13T01:26:36.137880007Z" level=error msg="Failed to destroy network for sandbox \"b67e0b59661da3e526e172461bd5f8487d1214e5ab24f8b2ad44c39ed24a2217\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:36.140673 systemd[1]: run-netns-cni\x2d70d04716\x2d6bce\x2d7810\x2d19c4\x2d05228cf7d2b9.mount: Deactivated successfully. Aug 13 01:26:36.141925 containerd[1567]: time="2025-08-13T01:26:36.141884578Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"b67e0b59661da3e526e172461bd5f8487d1214e5ab24f8b2ad44c39ed24a2217\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:36.146055 kubelet[2720]: E0813 01:26:36.145885 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b67e0b59661da3e526e172461bd5f8487d1214e5ab24f8b2ad44c39ed24a2217\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:36.146055 kubelet[2720]: E0813 01:26:36.145940 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b67e0b59661da3e526e172461bd5f8487d1214e5ab24f8b2ad44c39ed24a2217\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:36.146055 kubelet[2720]: E0813 01:26:36.145963 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b67e0b59661da3e526e172461bd5f8487d1214e5ab24f8b2ad44c39ed24a2217\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:36.146055 kubelet[2720]: E0813 01:26:36.146009 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"b67e0b59661da3e526e172461bd5f8487d1214e5ab24f8b2ad44c39ed24a2217\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:26:36.156121 kubelet[2720]: I0813 01:26:36.156086 2720 kubelet.go:2405] "Pod admission denied" podUID="1a59f135-f362-40c2-9bf9-9083f0dcefd0" pod="tigera-operator/tigera-operator-747864d56d-bqp29" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:36.198333 kubelet[2720]: I0813 01:26:36.198276 2720 kubelet.go:2405] "Pod admission denied" podUID="a6c93b3d-6891-47b5-a49c-bb7ab6804cf3" pod="tigera-operator/tigera-operator-747864d56d-dfvsm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:36.298142 kubelet[2720]: I0813 01:26:36.298095 2720 kubelet.go:2405] "Pod admission denied" podUID="ba4e06c9-edaf-4ddd-aa56-61bc5550534c" pod="tigera-operator/tigera-operator-747864d56d-6vl4n" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:36.398413 kubelet[2720]: I0813 01:26:36.398287 2720 kubelet.go:2405] "Pod admission denied" podUID="6c2f2e75-05be-4e32-a722-ab3df98b09a1" pod="tigera-operator/tigera-operator-747864d56d-r7z28" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:36.448947 kubelet[2720]: I0813 01:26:36.448897 2720 kubelet.go:2405] "Pod admission denied" podUID="7c5f0387-a80f-494e-b79f-b26b1d0c2dbf" pod="tigera-operator/tigera-operator-747864d56d-97fdh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:36.546563 kubelet[2720]: I0813 01:26:36.546509 2720 kubelet.go:2405] "Pod admission denied" podUID="6388757b-5997-4eb0-bff3-8092616bac0b" pod="tigera-operator/tigera-operator-747864d56d-g8mvx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:36.749931 kubelet[2720]: I0813 01:26:36.749881 2720 kubelet.go:2405] "Pod admission denied" podUID="2caa5e03-90d2-4481-80be-93ec2fd3a379" pod="tigera-operator/tigera-operator-747864d56d-5kxt6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:36.848932 kubelet[2720]: I0813 01:26:36.848885 2720 kubelet.go:2405] "Pod admission denied" podUID="5c7df1f9-341a-4536-861c-faaa4fc9d137" pod="tigera-operator/tigera-operator-747864d56d-rqp95" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:36.948662 kubelet[2720]: I0813 01:26:36.948607 2720 kubelet.go:2405] "Pod admission denied" podUID="ea922df6-9012-41ff-875d-2e8bf420621b" pod="tigera-operator/tigera-operator-747864d56d-2qrl9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:37.047369 kubelet[2720]: I0813 01:26:37.047263 2720 kubelet.go:2405] "Pod admission denied" podUID="ed3e495b-f9f2-4829-8acf-1d18bbf9175a" pod="tigera-operator/tigera-operator-747864d56d-jjqmz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:37.073685 kubelet[2720]: E0813 01:26:37.073519 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:26:37.074846 containerd[1567]: time="2025-08-13T01:26:37.074780492Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:26:37.111733 kubelet[2720]: I0813 01:26:37.111399 2720 kubelet.go:2405] "Pod admission denied" podUID="44b9a513-de25-4a71-a08a-727497ef08c8" pod="tigera-operator/tigera-operator-747864d56d-dfnpb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:37.143623 containerd[1567]: time="2025-08-13T01:26:37.143581024Z" level=error msg="Failed to destroy network for sandbox \"8b636edd7d98149e0b149bfca04c698f235eea41ed530223e25872eb9fb0b5b3\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:37.148840 containerd[1567]: time="2025-08-13T01:26:37.148798427Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"8b636edd7d98149e0b149bfca04c698f235eea41ed530223e25872eb9fb0b5b3\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:37.149510 systemd[1]: run-netns-cni\x2d76d80e73\x2de56b\x2d3333\x2d474a\x2de9a6d2aa348e.mount: Deactivated successfully. Aug 13 01:26:37.150886 kubelet[2720]: E0813 01:26:37.149627 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8b636edd7d98149e0b149bfca04c698f235eea41ed530223e25872eb9fb0b5b3\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:37.150886 kubelet[2720]: E0813 01:26:37.149693 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8b636edd7d98149e0b149bfca04c698f235eea41ed530223e25872eb9fb0b5b3\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:26:37.150886 kubelet[2720]: E0813 01:26:37.149826 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8b636edd7d98149e0b149bfca04c698f235eea41ed530223e25872eb9fb0b5b3\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:26:37.150886 kubelet[2720]: E0813 01:26:37.149923 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"8b636edd7d98149e0b149bfca04c698f235eea41ed530223e25872eb9fb0b5b3\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:26:37.199825 kubelet[2720]: I0813 01:26:37.199769 2720 kubelet.go:2405] "Pod admission denied" podUID="cd5b88c5-c257-43cd-b614-b574ef35c352" pod="tigera-operator/tigera-operator-747864d56d-zgtw9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:37.300900 kubelet[2720]: I0813 01:26:37.300227 2720 kubelet.go:2405] "Pod admission denied" podUID="810ea3d5-2a60-4643-9e30-c751af76fb63" pod="tigera-operator/tigera-operator-747864d56d-mfbl9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:37.400964 kubelet[2720]: I0813 01:26:37.400913 2720 kubelet.go:2405] "Pod admission denied" podUID="f0bba856-7076-4922-9074-44d376e62949" pod="tigera-operator/tigera-operator-747864d56d-sk5l7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:37.499105 kubelet[2720]: I0813 01:26:37.499052 2720 kubelet.go:2405] "Pod admission denied" podUID="0bc3efef-8473-47a7-a905-fb94b71810e6" pod="tigera-operator/tigera-operator-747864d56d-c9ncr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:37.597889 kubelet[2720]: I0813 01:26:37.597785 2720 kubelet.go:2405] "Pod admission denied" podUID="95d8f65a-45e0-4df8-a37b-233677a7ead9" pod="tigera-operator/tigera-operator-747864d56d-jbs6s" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:37.633082 kubelet[2720]: I0813 01:26:37.632045 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:26:37.633485 kubelet[2720]: I0813 01:26:37.633322 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:26:37.638663 kubelet[2720]: I0813 01:26:37.638637 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:26:37.651661 kubelet[2720]: I0813 01:26:37.651640 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:26:37.651761 kubelet[2720]: I0813 01:26:37.651740 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/csi-node-driver-8bzzs","calico-system/calico-node-2p8hj","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:26:37.651862 kubelet[2720]: E0813 01:26:37.651770 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:26:37.651862 kubelet[2720]: E0813 01:26:37.651781 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:37.651862 kubelet[2720]: E0813 01:26:37.651787 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:26:37.651862 kubelet[2720]: E0813 01:26:37.651794 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:37.651862 kubelet[2720]: E0813 01:26:37.651800 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:26:37.651862 kubelet[2720]: E0813 01:26:37.651810 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:26:37.651862 kubelet[2720]: E0813 01:26:37.651818 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:26:37.651862 kubelet[2720]: E0813 01:26:37.651826 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:26:37.651862 kubelet[2720]: E0813 01:26:37.651834 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:26:37.651862 kubelet[2720]: E0813 01:26:37.651842 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:26:37.651862 kubelet[2720]: I0813 01:26:37.651851 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:26:37.698078 kubelet[2720]: I0813 01:26:37.698041 2720 kubelet.go:2405] "Pod admission denied" podUID="ecd9914b-b1e2-45d3-a514-9c22e12dd8c2" pod="tigera-operator/tigera-operator-747864d56d-gqgnf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:37.749572 kubelet[2720]: I0813 01:26:37.749518 2720 kubelet.go:2405] "Pod admission denied" podUID="6d93fcb8-6011-41f5-b747-1a0eb2a8753f" pod="tigera-operator/tigera-operator-747864d56d-g99z6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:37.849392 kubelet[2720]: I0813 01:26:37.848885 2720 kubelet.go:2405] "Pod admission denied" podUID="d49c4db8-79c3-49cd-9b24-0cbd6b5ab016" pod="tigera-operator/tigera-operator-747864d56d-6t9cw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:37.950060 kubelet[2720]: I0813 01:26:37.950010 2720 kubelet.go:2405] "Pod admission denied" podUID="d41b7fcf-9d6b-461e-bc82-97e5801b8a06" pod="tigera-operator/tigera-operator-747864d56d-zp8bf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:38.047901 kubelet[2720]: I0813 01:26:38.047855 2720 kubelet.go:2405] "Pod admission denied" podUID="53568989-8f4a-4400-8747-bb6e40243d73" pod="tigera-operator/tigera-operator-747864d56d-wvgj5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:38.073985 containerd[1567]: time="2025-08-13T01:26:38.073952070Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:26:38.129749 containerd[1567]: time="2025-08-13T01:26:38.129476483Z" level=error msg="Failed to destroy network for sandbox \"c686b4edf483ab9cb19485ae23f2c125d97d16b9ab84022414b0f83920e86a89\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:38.132468 systemd[1]: run-netns-cni\x2d0ae0c79f\x2dcfa1\x2d2301\x2da81f\x2dbc69c742c161.mount: Deactivated successfully. Aug 13 01:26:38.134404 containerd[1567]: time="2025-08-13T01:26:38.134072167Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c686b4edf483ab9cb19485ae23f2c125d97d16b9ab84022414b0f83920e86a89\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:38.134689 kubelet[2720]: E0813 01:26:38.134566 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c686b4edf483ab9cb19485ae23f2c125d97d16b9ab84022414b0f83920e86a89\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:38.134764 kubelet[2720]: E0813 01:26:38.134688 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c686b4edf483ab9cb19485ae23f2c125d97d16b9ab84022414b0f83920e86a89\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:26:38.134764 kubelet[2720]: E0813 01:26:38.134755 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c686b4edf483ab9cb19485ae23f2c125d97d16b9ab84022414b0f83920e86a89\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:26:38.134910 kubelet[2720]: E0813 01:26:38.134841 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c686b4edf483ab9cb19485ae23f2c125d97d16b9ab84022414b0f83920e86a89\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:26:38.252942 kubelet[2720]: I0813 01:26:38.252894 2720 kubelet.go:2405] "Pod admission denied" podUID="9ac4a9f2-e88a-4908-a877-6a19ccdaa4d1" pod="tigera-operator/tigera-operator-747864d56d-4xdvz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:38.348805 kubelet[2720]: I0813 01:26:38.348765 2720 kubelet.go:2405] "Pod admission denied" podUID="b3730d99-a2fb-4379-bf1e-0e6f6f3d31eb" pod="tigera-operator/tigera-operator-747864d56d-4z96s" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:38.401166 kubelet[2720]: I0813 01:26:38.401031 2720 kubelet.go:2405] "Pod admission denied" podUID="143053ae-6b63-472f-b9e6-20ecc32994f3" pod="tigera-operator/tigera-operator-747864d56d-x5bwm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:38.500665 kubelet[2720]: I0813 01:26:38.500603 2720 kubelet.go:2405] "Pod admission denied" podUID="8bcf84e2-ca4e-4a0b-a321-0be78d0e3090" pod="tigera-operator/tigera-operator-747864d56d-f2frt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:38.702755 kubelet[2720]: I0813 01:26:38.702093 2720 kubelet.go:2405] "Pod admission denied" podUID="6bcf81b1-3474-4d84-a1eb-0fdac6f2019c" pod="tigera-operator/tigera-operator-747864d56d-l6gp6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:38.808748 kubelet[2720]: I0813 01:26:38.808689 2720 kubelet.go:2405] "Pod admission denied" podUID="5e81076e-3bb0-4e48-8d58-333fb608a6fe" pod="tigera-operator/tigera-operator-747864d56d-ks8tt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:38.904655 kubelet[2720]: I0813 01:26:38.904145 2720 kubelet.go:2405] "Pod admission denied" podUID="79dec3a3-a65c-4548-8f01-4fe102f4ee01" pod="tigera-operator/tigera-operator-747864d56d-4rfdw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:38.999354 kubelet[2720]: I0813 01:26:38.999298 2720 kubelet.go:2405] "Pod admission denied" podUID="fe00ec07-2832-4870-8c44-1dfa5f637866" pod="tigera-operator/tigera-operator-747864d56d-gl2lb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:39.073475 kubelet[2720]: E0813 01:26:39.073445 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:26:39.098365 kubelet[2720]: I0813 01:26:39.098016 2720 kubelet.go:2405] "Pod admission denied" podUID="6ff23004-d863-4644-91f7-e82f83b1e241" pod="tigera-operator/tigera-operator-747864d56d-kqdhr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:39.200744 kubelet[2720]: I0813 01:26:39.200684 2720 kubelet.go:2405] "Pod admission denied" podUID="d1a6281b-6d0d-4f75-b8e2-c39702a0e51d" pod="tigera-operator/tigera-operator-747864d56d-q7kkv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:39.248246 kubelet[2720]: I0813 01:26:39.248188 2720 kubelet.go:2405] "Pod admission denied" podUID="897138ad-c9d1-455a-958c-dbea0c07e5c2" pod="tigera-operator/tigera-operator-747864d56d-wccm8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:39.352304 kubelet[2720]: I0813 01:26:39.352172 2720 kubelet.go:2405] "Pod admission denied" podUID="6ce2d027-4d76-4733-b078-7b0a5df622b3" pod="tigera-operator/tigera-operator-747864d56d-6pkl5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:39.553299 kubelet[2720]: I0813 01:26:39.553246 2720 kubelet.go:2405] "Pod admission denied" podUID="6d67d3a0-11ad-4e40-ae04-38b6bcc161cc" pod="tigera-operator/tigera-operator-747864d56d-xktlt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:39.647506 kubelet[2720]: I0813 01:26:39.647383 2720 kubelet.go:2405] "Pod admission denied" podUID="60318d76-b9e1-4685-a3aa-eb4629c2b26d" pod="tigera-operator/tigera-operator-747864d56d-njnxf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:39.698022 kubelet[2720]: I0813 01:26:39.697971 2720 kubelet.go:2405] "Pod admission denied" podUID="4b96106f-4e32-4e1f-80cd-927473497638" pod="tigera-operator/tigera-operator-747864d56d-6k8l8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:39.797582 kubelet[2720]: I0813 01:26:39.797524 2720 kubelet.go:2405] "Pod admission denied" podUID="3f704219-18cc-4047-982b-f293ae6c7b96" pod="tigera-operator/tigera-operator-747864d56d-6wwmd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:39.901638 kubelet[2720]: I0813 01:26:39.901148 2720 kubelet.go:2405] "Pod admission denied" podUID="eb218f87-357b-4fc3-a03d-14875f3ed9fa" pod="tigera-operator/tigera-operator-747864d56d-bzltr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:40.000702 kubelet[2720]: I0813 01:26:40.000643 2720 kubelet.go:2405] "Pod admission denied" podUID="b04987df-dcb2-49ba-93d5-13d6543e7d85" pod="tigera-operator/tigera-operator-747864d56d-nds99" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:40.074629 kubelet[2720]: E0813 01:26:40.074030 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:26:40.075541 containerd[1567]: time="2025-08-13T01:26:40.075173698Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:26:40.136296 kubelet[2720]: I0813 01:26:40.136237 2720 kubelet.go:2405] "Pod admission denied" podUID="9c9f31e1-f573-476c-8533-f7870300e1cf" pod="tigera-operator/tigera-operator-747864d56d-mdspw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:40.172881 containerd[1567]: time="2025-08-13T01:26:40.170075391Z" level=error msg="Failed to destroy network for sandbox \"47c6f157324a4f01c65fb572a705ef06f32b7707d5fbc961cea048773d6f23fd\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:40.174418 containerd[1567]: time="2025-08-13T01:26:40.174370965Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"47c6f157324a4f01c65fb572a705ef06f32b7707d5fbc961cea048773d6f23fd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:40.174852 kubelet[2720]: E0813 01:26:40.174680 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"47c6f157324a4f01c65fb572a705ef06f32b7707d5fbc961cea048773d6f23fd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:40.175665 kubelet[2720]: E0813 01:26:40.175414 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"47c6f157324a4f01c65fb572a705ef06f32b7707d5fbc961cea048773d6f23fd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:40.175665 kubelet[2720]: E0813 01:26:40.175443 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"47c6f157324a4f01c65fb572a705ef06f32b7707d5fbc961cea048773d6f23fd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:40.175665 kubelet[2720]: E0813 01:26:40.175485 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"47c6f157324a4f01c65fb572a705ef06f32b7707d5fbc961cea048773d6f23fd\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:26:40.175529 systemd[1]: run-netns-cni\x2de4f488dc\x2d2a64\x2dc402\x2dfed7\x2d3d4cf552f5fa.mount: Deactivated successfully. Aug 13 01:26:40.196811 kubelet[2720]: I0813 01:26:40.196773 2720 kubelet.go:2405] "Pod admission denied" podUID="42fb1c99-4974-4add-b7f9-8485f0c2a9cd" pod="tigera-operator/tigera-operator-747864d56d-rbdtv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:40.301766 kubelet[2720]: I0813 01:26:40.301700 2720 kubelet.go:2405] "Pod admission denied" podUID="92093005-a0b2-4311-b825-e6712f60c584" pod="tigera-operator/tigera-operator-747864d56d-stqz4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:40.410901 kubelet[2720]: I0813 01:26:40.410294 2720 kubelet.go:2405] "Pod admission denied" podUID="28da32c0-6d3a-42e7-889c-60f68f95a80b" pod="tigera-operator/tigera-operator-747864d56d-4k2hn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:40.500987 kubelet[2720]: I0813 01:26:40.500940 2720 kubelet.go:2405] "Pod admission denied" podUID="ece22406-954f-46bd-989a-0245aaacfe8c" pod="tigera-operator/tigera-operator-747864d56d-hx4hv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:40.601542 kubelet[2720]: I0813 01:26:40.601491 2720 kubelet.go:2405] "Pod admission denied" podUID="8f19566b-9c96-4593-8124-50bf99780e99" pod="tigera-operator/tigera-operator-747864d56d-m9wq9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:40.812611 kubelet[2720]: I0813 01:26:40.812499 2720 kubelet.go:2405] "Pod admission denied" podUID="e1743b3d-30a0-49f7-a7fd-9a9cdcabc5eb" pod="tigera-operator/tigera-operator-747864d56d-h66fm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:40.900401 kubelet[2720]: I0813 01:26:40.900190 2720 kubelet.go:2405] "Pod admission denied" podUID="6158f048-b6eb-494a-9668-d8e3ae905514" pod="tigera-operator/tigera-operator-747864d56d-58tnn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:41.001030 kubelet[2720]: I0813 01:26:41.000978 2720 kubelet.go:2405] "Pod admission denied" podUID="68ccebf5-6fc8-4c55-ae7a-a6b2953443b7" pod="tigera-operator/tigera-operator-747864d56d-brjct" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:41.115843 kubelet[2720]: I0813 01:26:41.114439 2720 kubelet.go:2405] "Pod admission denied" podUID="a2741032-b6e6-4708-aa1a-819cba9a7383" pod="tigera-operator/tigera-operator-747864d56d-mq7kc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:41.201671 kubelet[2720]: I0813 01:26:41.201614 2720 kubelet.go:2405] "Pod admission denied" podUID="8ef3cca8-b239-49f3-a71c-2f249120fa60" pod="tigera-operator/tigera-operator-747864d56d-8q6cl" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:41.298777 kubelet[2720]: I0813 01:26:41.298728 2720 kubelet.go:2405] "Pod admission denied" podUID="9443c355-5eff-48b9-bee9-f9d8d11899db" pod="tigera-operator/tigera-operator-747864d56d-b6rph" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:41.411481 kubelet[2720]: I0813 01:26:41.410828 2720 kubelet.go:2405] "Pod admission denied" podUID="8be33eb1-1efe-4ab0-a012-4cf54895eb4f" pod="tigera-operator/tigera-operator-747864d56d-zv592" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:41.500545 kubelet[2720]: I0813 01:26:41.500498 2720 kubelet.go:2405] "Pod admission denied" podUID="5917aa2a-a06c-4bd5-85ba-688a3c5b5e47" pod="tigera-operator/tigera-operator-747864d56d-9wk6v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:41.599489 kubelet[2720]: I0813 01:26:41.599431 2720 kubelet.go:2405] "Pod admission denied" podUID="35988ebf-37c6-4609-84ac-fac0613d8572" pod="tigera-operator/tigera-operator-747864d56d-vrdwg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:41.706801 kubelet[2720]: I0813 01:26:41.706370 2720 kubelet.go:2405] "Pod admission denied" podUID="ff060bcb-787b-4c5e-a03f-b0791627a46d" pod="tigera-operator/tigera-operator-747864d56d-8kh2j" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:41.749299 kubelet[2720]: I0813 01:26:41.749239 2720 kubelet.go:2405] "Pod admission denied" podUID="9048f4d3-1cc0-4e07-835d-4e33e991f05d" pod="tigera-operator/tigera-operator-747864d56d-hzpt7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:41.851168 kubelet[2720]: I0813 01:26:41.851113 2720 kubelet.go:2405] "Pod admission denied" podUID="09e4e4fc-eb5f-4ca9-88fd-02bd709f519c" pod="tigera-operator/tigera-operator-747864d56d-lgvrj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:41.962388 kubelet[2720]: I0813 01:26:41.961009 2720 kubelet.go:2405] "Pod admission denied" podUID="16256135-f5dc-4007-b1a6-e69744f97d7a" pod="tigera-operator/tigera-operator-747864d56d-w4vbb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:42.051292 kubelet[2720]: I0813 01:26:42.051239 2720 kubelet.go:2405] "Pod admission denied" podUID="cda6d7bf-efe7-4cf8-9bd8-4d98da87b8a4" pod="tigera-operator/tigera-operator-747864d56d-gwg9g" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:42.077338 kubelet[2720]: E0813 01:26:42.077280 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"calico-node\" with ImagePullBackOff: \"Back-off pulling image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": ErrImagePull: failed to pull and unpack image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1682018348: write /var/lib/containerd/tmpmounts/containerd-mount1682018348/usr/bin/calico-node: no space left on device\"" pod="calico-system/calico-node-2p8hj" podUID="4f334296-e8c0-427e-a090-d8fbd3582f26" Aug 13 01:26:42.160383 kubelet[2720]: I0813 01:26:42.160136 2720 kubelet.go:2405] "Pod admission denied" podUID="d314cdc0-0f22-4689-80cb-2038da363269" pod="tigera-operator/tigera-operator-747864d56d-wncr9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:42.249599 kubelet[2720]: I0813 01:26:42.249084 2720 kubelet.go:2405] "Pod admission denied" podUID="b0f60cc2-8139-4ee0-a1ef-71d72ba6dcf6" pod="tigera-operator/tigera-operator-747864d56d-4xss2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:42.350891 kubelet[2720]: I0813 01:26:42.350838 2720 kubelet.go:2405] "Pod admission denied" podUID="1e11e06a-1ba1-4792-84e6-27b30fdec491" pod="tigera-operator/tigera-operator-747864d56d-wdwgk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:42.449045 kubelet[2720]: I0813 01:26:42.448999 2720 kubelet.go:2405] "Pod admission denied" podUID="5ac39b9f-8858-4ac4-afe2-33f983b7b8ed" pod="tigera-operator/tigera-operator-747864d56d-q8fgc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:42.549845 kubelet[2720]: I0813 01:26:42.549738 2720 kubelet.go:2405] "Pod admission denied" podUID="996f795e-f230-46a9-984e-1474bbd389dc" pod="tigera-operator/tigera-operator-747864d56d-2nxhj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:42.656738 kubelet[2720]: I0813 01:26:42.656018 2720 kubelet.go:2405] "Pod admission denied" podUID="8007e654-6b9e-4e9d-b20f-3c4595d9565b" pod="tigera-operator/tigera-operator-747864d56d-6r75k" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:42.851995 kubelet[2720]: I0813 01:26:42.850976 2720 kubelet.go:2405] "Pod admission denied" podUID="f3c54241-02c6-470d-a662-717fc246f476" pod="tigera-operator/tigera-operator-747864d56d-84nkp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:42.952335 kubelet[2720]: I0813 01:26:42.952284 2720 kubelet.go:2405] "Pod admission denied" podUID="87713439-f5f1-432b-a421-3c34209bfde8" pod="tigera-operator/tigera-operator-747864d56d-4jhbt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:42.999517 kubelet[2720]: I0813 01:26:42.999481 2720 kubelet.go:2405] "Pod admission denied" podUID="9fc724c2-46c7-45a3-b17a-2b8b27fd0c06" pod="tigera-operator/tigera-operator-747864d56d-wkvzf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:43.100522 kubelet[2720]: I0813 01:26:43.100488 2720 kubelet.go:2405] "Pod admission denied" podUID="36173c68-ea26-4ad1-99a7-34d03416618b" pod="tigera-operator/tigera-operator-747864d56d-4fk2l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:43.204970 kubelet[2720]: I0813 01:26:43.203700 2720 kubelet.go:2405] "Pod admission denied" podUID="9e7e8be4-6349-4e91-bce4-dc8792d91473" pod="tigera-operator/tigera-operator-747864d56d-sjt7h" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:43.253154 kubelet[2720]: I0813 01:26:43.253106 2720 kubelet.go:2405] "Pod admission denied" podUID="eb350aac-bfc0-4e4c-9e00-280d4c34bab0" pod="tigera-operator/tigera-operator-747864d56d-xxcv9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:43.350585 kubelet[2720]: I0813 01:26:43.350541 2720 kubelet.go:2405] "Pod admission denied" podUID="d590544c-c024-4212-86a5-08ed8d5bf454" pod="tigera-operator/tigera-operator-747864d56d-v86vg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:43.462817 kubelet[2720]: I0813 01:26:43.462608 2720 kubelet.go:2405] "Pod admission denied" podUID="fb32cb59-ba61-43c6-b7b4-86fd0a54d640" pod="tigera-operator/tigera-operator-747864d56d-dtppj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:43.550871 kubelet[2720]: I0813 01:26:43.550818 2720 kubelet.go:2405] "Pod admission denied" podUID="abe4eb41-86d8-4026-b288-5bcc4280deb2" pod="tigera-operator/tigera-operator-747864d56d-m46j5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:43.650866 kubelet[2720]: I0813 01:26:43.650812 2720 kubelet.go:2405] "Pod admission denied" podUID="717b8ae1-65ee-4912-8eac-42a7088185a6" pod="tigera-operator/tigera-operator-747864d56d-54z42" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:43.714072 kubelet[2720]: I0813 01:26:43.713220 2720 kubelet.go:2405] "Pod admission denied" podUID="c8cdee74-c056-4244-b2f7-a30feebe72a9" pod="tigera-operator/tigera-operator-747864d56d-nmrhf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:43.802059 kubelet[2720]: I0813 01:26:43.801999 2720 kubelet.go:2405] "Pod admission denied" podUID="7613df08-a52f-48fd-b230-2b1b6dc6a900" pod="tigera-operator/tigera-operator-747864d56d-p4zxc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:43.900627 kubelet[2720]: I0813 01:26:43.900537 2720 kubelet.go:2405] "Pod admission denied" podUID="77ba2bfb-49ec-4b8b-9c14-86874a60de07" pod="tigera-operator/tigera-operator-747864d56d-msdnm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:43.958736 kubelet[2720]: I0813 01:26:43.956741 2720 kubelet.go:2405] "Pod admission denied" podUID="fb637edb-e3ac-4096-a925-11a7428ed91e" pod="tigera-operator/tigera-operator-747864d56d-qrxgs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:44.049491 kubelet[2720]: I0813 01:26:44.049358 2720 kubelet.go:2405] "Pod admission denied" podUID="5aedb764-bf1b-4eb4-8a8f-092f98ac81f2" pod="tigera-operator/tigera-operator-747864d56d-4qz4v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:44.253243 kubelet[2720]: I0813 01:26:44.253195 2720 kubelet.go:2405] "Pod admission denied" podUID="99e045cd-ffb7-4ac6-8219-d2acb71262b9" pod="tigera-operator/tigera-operator-747864d56d-f2ksq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:44.352776 kubelet[2720]: I0813 01:26:44.351498 2720 kubelet.go:2405] "Pod admission denied" podUID="40c0c286-9180-427c-bcef-94276db01692" pod="tigera-operator/tigera-operator-747864d56d-x2hjd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:44.450547 kubelet[2720]: I0813 01:26:44.450496 2720 kubelet.go:2405] "Pod admission denied" podUID="4922c810-5dee-4eea-9c5e-20816995ab26" pod="tigera-operator/tigera-operator-747864d56d-h6pzj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:44.655810 kubelet[2720]: I0813 01:26:44.655561 2720 kubelet.go:2405] "Pod admission denied" podUID="77e1f8bc-8668-4726-a16d-55b6d997decc" pod="tigera-operator/tigera-operator-747864d56d-zv76d" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:44.751398 kubelet[2720]: I0813 01:26:44.751349 2720 kubelet.go:2405] "Pod admission denied" podUID="6a65a696-a73d-4061-a08b-8f5ef4525d09" pod="tigera-operator/tigera-operator-747864d56d-q2v99" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:44.849398 kubelet[2720]: I0813 01:26:44.849352 2720 kubelet.go:2405] "Pod admission denied" podUID="b76ae60c-d6a7-4c50-af7b-b67336964e5c" pod="tigera-operator/tigera-operator-747864d56d-f9pbz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:44.955345 kubelet[2720]: I0813 01:26:44.954760 2720 kubelet.go:2405] "Pod admission denied" podUID="553a82e3-8fb4-4493-ac89-3f1665180365" pod="tigera-operator/tigera-operator-747864d56d-pbb2s" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:45.000530 kubelet[2720]: I0813 01:26:45.000487 2720 kubelet.go:2405] "Pod admission denied" podUID="a8076a93-8e7a-493b-8195-4a70f043b36f" pod="tigera-operator/tigera-operator-747864d56d-wxr44" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:45.073304 kubelet[2720]: E0813 01:26:45.073279 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:26:45.104074 kubelet[2720]: I0813 01:26:45.104042 2720 kubelet.go:2405] "Pod admission denied" podUID="329aeb72-7262-43ff-89a0-fd588ca8785b" pod="tigera-operator/tigera-operator-747864d56d-5bp7h" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:45.199276 kubelet[2720]: I0813 01:26:45.199227 2720 kubelet.go:2405] "Pod admission denied" podUID="0c441594-9a7c-4b62-9f5f-747b75592d95" pod="tigera-operator/tigera-operator-747864d56d-zw8vc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:45.300166 kubelet[2720]: I0813 01:26:45.300124 2720 kubelet.go:2405] "Pod admission denied" podUID="2e7f5562-1a8b-48b1-83fb-15191d6e6401" pod="tigera-operator/tigera-operator-747864d56d-8stdc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:45.499238 kubelet[2720]: I0813 01:26:45.499197 2720 kubelet.go:2405] "Pod admission denied" podUID="92b8f8d8-7ae6-46ca-8ef0-ff673159f5ba" pod="tigera-operator/tigera-operator-747864d56d-m9wp5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:45.599476 kubelet[2720]: I0813 01:26:45.599352 2720 kubelet.go:2405] "Pod admission denied" podUID="62473d87-6bee-4be1-83b7-6ae37ac6a9bc" pod="tigera-operator/tigera-operator-747864d56d-nsnlz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:45.651919 kubelet[2720]: I0813 01:26:45.651871 2720 kubelet.go:2405] "Pod admission denied" podUID="d34c8e6b-db55-49af-8c56-cc93ddb255b9" pod="tigera-operator/tigera-operator-747864d56d-zbt7f" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:45.752274 kubelet[2720]: I0813 01:26:45.752217 2720 kubelet.go:2405] "Pod admission denied" podUID="c4fc3bc5-97a5-419c-a740-7eccb708f809" pod="tigera-operator/tigera-operator-747864d56d-jvhwf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:45.851923 kubelet[2720]: I0813 01:26:45.851452 2720 kubelet.go:2405] "Pod admission denied" podUID="0650c3e9-f728-41d1-84bc-abad9e8cc06f" pod="tigera-operator/tigera-operator-747864d56d-597lp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:45.898858 kubelet[2720]: I0813 01:26:45.898734 2720 kubelet.go:2405] "Pod admission denied" podUID="6d627f8f-110e-4bc4-b398-1b0ca1d95bdf" pod="tigera-operator/tigera-operator-747864d56d-w5sdv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:46.010081 kubelet[2720]: I0813 01:26:46.010034 2720 kubelet.go:2405] "Pod admission denied" podUID="1940e17d-37c1-494d-919f-0b3ab9092b84" pod="tigera-operator/tigera-operator-747864d56d-zwswq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:46.105759 kubelet[2720]: I0813 01:26:46.105616 2720 kubelet.go:2405] "Pod admission denied" podUID="33b1dd6e-3239-46eb-bed5-1a8156ea60bd" pod="tigera-operator/tigera-operator-747864d56d-clmpl" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:46.201793 kubelet[2720]: I0813 01:26:46.201741 2720 kubelet.go:2405] "Pod admission denied" podUID="7be24886-ae03-427a-a34e-bf3b429c34b4" pod="tigera-operator/tigera-operator-747864d56d-v5nqn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:46.307837 kubelet[2720]: I0813 01:26:46.307787 2720 kubelet.go:2405] "Pod admission denied" podUID="ac5000cb-2601-47d6-813a-5fea66a8b2af" pod="tigera-operator/tigera-operator-747864d56d-dbjvs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:46.402251 kubelet[2720]: I0813 01:26:46.401704 2720 kubelet.go:2405] "Pod admission denied" podUID="0b713d71-76f0-40ef-85c8-b9ff28f8ee55" pod="tigera-operator/tigera-operator-747864d56d-94x52" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:46.499273 kubelet[2720]: I0813 01:26:46.499229 2720 kubelet.go:2405] "Pod admission denied" podUID="aa51c758-4d9f-460f-9054-d23b15096ac8" pod="tigera-operator/tigera-operator-747864d56d-6gpdn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:46.602448 kubelet[2720]: I0813 01:26:46.602395 2720 kubelet.go:2405] "Pod admission denied" podUID="4a794dbd-2c20-481a-b78b-cdbd4a7c4a39" pod="tigera-operator/tigera-operator-747864d56d-sglfc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:46.700339 kubelet[2720]: I0813 01:26:46.700223 2720 kubelet.go:2405] "Pod admission denied" podUID="2704f8ae-9644-4e1c-b817-950cd6672c69" pod="tigera-operator/tigera-operator-747864d56d-nxjvv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:46.800008 kubelet[2720]: I0813 01:26:46.799966 2720 kubelet.go:2405] "Pod admission denied" podUID="593079c8-813d-48f9-b81a-76501167e01d" pod="tigera-operator/tigera-operator-747864d56d-lfhdt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:47.002482 kubelet[2720]: I0813 01:26:47.002448 2720 kubelet.go:2405] "Pod admission denied" podUID="80deeccf-77bf-497c-8565-74217e3175bd" pod="tigera-operator/tigera-operator-747864d56d-w9nv9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:47.073954 containerd[1567]: time="2025-08-13T01:26:47.073575215Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:26:47.121045 kubelet[2720]: I0813 01:26:47.120532 2720 kubelet.go:2405] "Pod admission denied" podUID="4d3eb17c-1398-48e8-9628-8ef629be10cf" pod="tigera-operator/tigera-operator-747864d56d-6wwmt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:47.186147 kubelet[2720]: I0813 01:26:47.186099 2720 kubelet.go:2405] "Pod admission denied" podUID="a53ce4e1-1ae6-44cb-8c76-a03378963c7b" pod="tigera-operator/tigera-operator-747864d56d-rkc8v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:47.187053 containerd[1567]: time="2025-08-13T01:26:47.187004560Z" level=error msg="Failed to destroy network for sandbox \"d4070bc471a583c5aee64d8ab345f4d9f9e6bd73560302507fd851dbb126d365\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:47.188419 containerd[1567]: time="2025-08-13T01:26:47.188349902Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"d4070bc471a583c5aee64d8ab345f4d9f9e6bd73560302507fd851dbb126d365\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:47.190309 systemd[1]: run-netns-cni\x2d256cd10b\x2def28\x2d32c9\x2d9fe8\x2dfdb6c48e6cc3.mount: Deactivated successfully. Aug 13 01:26:47.191352 kubelet[2720]: E0813 01:26:47.191328 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d4070bc471a583c5aee64d8ab345f4d9f9e6bd73560302507fd851dbb126d365\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:47.191473 kubelet[2720]: E0813 01:26:47.191454 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d4070bc471a583c5aee64d8ab345f4d9f9e6bd73560302507fd851dbb126d365\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:47.191554 kubelet[2720]: E0813 01:26:47.191537 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d4070bc471a583c5aee64d8ab345f4d9f9e6bd73560302507fd851dbb126d365\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:47.191682 kubelet[2720]: E0813 01:26:47.191646 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"d4070bc471a583c5aee64d8ab345f4d9f9e6bd73560302507fd851dbb126d365\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:26:47.303360 kubelet[2720]: I0813 01:26:47.303233 2720 kubelet.go:2405] "Pod admission denied" podUID="19a46d2d-7ac0-4aea-8424-41663fc7bd96" pod="tigera-operator/tigera-operator-747864d56d-bh8nt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:47.400280 kubelet[2720]: I0813 01:26:47.400239 2720 kubelet.go:2405] "Pod admission denied" podUID="f2ddfce9-4361-4a66-8670-15d36afdfb98" pod="tigera-operator/tigera-operator-747864d56d-jn8b5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:47.501530 kubelet[2720]: I0813 01:26:47.501470 2720 kubelet.go:2405] "Pod admission denied" podUID="c19faebd-0e57-4213-8818-914d7b1ea12b" pod="tigera-operator/tigera-operator-747864d56d-29n48" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:47.551697 kubelet[2720]: I0813 01:26:47.551641 2720 kubelet.go:2405] "Pod admission denied" podUID="cb054c2b-fcfe-4053-b903-71b84986b745" pod="tigera-operator/tigera-operator-747864d56d-hbjdr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:47.650800 kubelet[2720]: I0813 01:26:47.649984 2720 kubelet.go:2405] "Pod admission denied" podUID="34bcbda5-ef91-47fa-af5d-585caac48a6b" pod="tigera-operator/tigera-operator-747864d56d-xd6k8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:47.667861 kubelet[2720]: I0813 01:26:47.667534 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:26:47.667861 kubelet[2720]: I0813 01:26:47.667568 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:26:47.670680 kubelet[2720]: I0813 01:26:47.670642 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:26:47.680299 kubelet[2720]: I0813 01:26:47.680268 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:26:47.680369 kubelet[2720]: I0813 01:26:47.680340 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/csi-node-driver-8bzzs","calico-system/calico-node-2p8hj","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:26:47.680448 kubelet[2720]: E0813 01:26:47.680369 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:26:47.680448 kubelet[2720]: E0813 01:26:47.680378 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:47.680448 kubelet[2720]: E0813 01:26:47.680384 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:26:47.680448 kubelet[2720]: E0813 01:26:47.680392 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:47.680448 kubelet[2720]: E0813 01:26:47.680397 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:26:47.680448 kubelet[2720]: E0813 01:26:47.680407 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:26:47.680448 kubelet[2720]: E0813 01:26:47.680415 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:26:47.680448 kubelet[2720]: E0813 01:26:47.680422 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:26:47.680448 kubelet[2720]: E0813 01:26:47.680429 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:26:47.680448 kubelet[2720]: E0813 01:26:47.680436 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:26:47.680448 kubelet[2720]: I0813 01:26:47.680445 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:26:47.763199 kubelet[2720]: I0813 01:26:47.762597 2720 kubelet.go:2405] "Pod admission denied" podUID="f59dbada-c727-4d48-8bf4-5418d0609e36" pod="tigera-operator/tigera-operator-747864d56d-lr89v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:47.851685 kubelet[2720]: I0813 01:26:47.851630 2720 kubelet.go:2405] "Pod admission denied" podUID="393b524f-8980-44ea-a550-d03cd63a32db" pod="tigera-operator/tigera-operator-747864d56d-ql274" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:48.052764 kubelet[2720]: I0813 01:26:48.052704 2720 kubelet.go:2405] "Pod admission denied" podUID="abc50fd9-f5bb-4202-842a-a547cc5cbdb0" pod="tigera-operator/tigera-operator-747864d56d-9bhmh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:48.084734 kubelet[2720]: E0813 01:26:48.084148 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:26:48.086646 kubelet[2720]: E0813 01:26:48.084371 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:26:48.086744 containerd[1567]: time="2025-08-13T01:26:48.085610446Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:26:48.156490 containerd[1567]: time="2025-08-13T01:26:48.156433777Z" level=error msg="Failed to destroy network for sandbox \"388bd2459da0289998454cd82da609db2368739ef617f1fcaa3eb2d7713203bc\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:48.158427 kubelet[2720]: I0813 01:26:48.158394 2720 kubelet.go:2405] "Pod admission denied" podUID="f56bc5f5-0a84-4941-a194-5db80afe7985" pod="tigera-operator/tigera-operator-747864d56d-dlp6q" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:48.160924 containerd[1567]: time="2025-08-13T01:26:48.160888852Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"388bd2459da0289998454cd82da609db2368739ef617f1fcaa3eb2d7713203bc\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:48.162392 kubelet[2720]: E0813 01:26:48.161266 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"388bd2459da0289998454cd82da609db2368739ef617f1fcaa3eb2d7713203bc\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:48.162392 kubelet[2720]: E0813 01:26:48.161483 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"388bd2459da0289998454cd82da609db2368739ef617f1fcaa3eb2d7713203bc\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:26:48.162392 kubelet[2720]: E0813 01:26:48.161504 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"388bd2459da0289998454cd82da609db2368739ef617f1fcaa3eb2d7713203bc\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:26:48.162392 kubelet[2720]: E0813 01:26:48.161568 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"388bd2459da0289998454cd82da609db2368739ef617f1fcaa3eb2d7713203bc\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:26:48.161922 systemd[1]: run-netns-cni\x2de984cd28\x2dc272\x2d5646\x2d07b5\x2d480d26cf2a9b.mount: Deactivated successfully. Aug 13 01:26:48.250920 kubelet[2720]: I0813 01:26:48.250869 2720 kubelet.go:2405] "Pod admission denied" podUID="476ec10a-8df2-460f-af4a-392bdea64ee8" pod="tigera-operator/tigera-operator-747864d56d-6v7dx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:48.352368 kubelet[2720]: I0813 01:26:48.351706 2720 kubelet.go:2405] "Pod admission denied" podUID="20fcc1d2-05ff-4515-9635-b76a0f8a4b39" pod="tigera-operator/tigera-operator-747864d56d-bbl8r" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:48.451211 kubelet[2720]: I0813 01:26:48.451162 2720 kubelet.go:2405] "Pod admission denied" podUID="4dcb6b09-3fa4-440e-a208-8f22243cf5d8" pod="tigera-operator/tigera-operator-747864d56d-nzdxb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:48.562743 kubelet[2720]: I0813 01:26:48.562679 2720 kubelet.go:2405] "Pod admission denied" podUID="7ff6aa2e-d6fe-4838-ae21-49f9c4011877" pod="tigera-operator/tigera-operator-747864d56d-5bx2q" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:48.650587 kubelet[2720]: I0813 01:26:48.650486 2720 kubelet.go:2405] "Pod admission denied" podUID="1044ff1b-f2f6-4717-bf7a-879163a500fb" pod="tigera-operator/tigera-operator-747864d56d-n4kpt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:48.852211 kubelet[2720]: I0813 01:26:48.852156 2720 kubelet.go:2405] "Pod admission denied" podUID="30d13bdd-b03b-4e70-ac4a-936e4222dd1b" pod="tigera-operator/tigera-operator-747864d56d-6xwxb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:48.952995 kubelet[2720]: I0813 01:26:48.952875 2720 kubelet.go:2405] "Pod admission denied" podUID="61bbca73-6943-4065-934f-452d23d07c36" pod="tigera-operator/tigera-operator-747864d56d-lbbjv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:49.051470 kubelet[2720]: I0813 01:26:49.051417 2720 kubelet.go:2405] "Pod admission denied" podUID="59ec9f0e-7be0-40dc-8bf4-d2f28715926f" pod="tigera-operator/tigera-operator-747864d56d-rx7lp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:49.159739 kubelet[2720]: I0813 01:26:49.157833 2720 kubelet.go:2405] "Pod admission denied" podUID="8e1fd747-f54a-4402-8761-2332fdd411de" pod="tigera-operator/tigera-operator-747864d56d-lp9h8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:49.198962 kubelet[2720]: I0813 01:26:49.198920 2720 kubelet.go:2405] "Pod admission denied" podUID="4ff119c6-b083-46bc-b52f-8efa401c82a5" pod="tigera-operator/tigera-operator-747864d56d-p496t" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:49.302311 kubelet[2720]: I0813 01:26:49.302266 2720 kubelet.go:2405] "Pod admission denied" podUID="7ae0f91c-4193-4b5e-9e22-9dd3c1a985c6" pod="tigera-operator/tigera-operator-747864d56d-smwvx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:49.401283 kubelet[2720]: I0813 01:26:49.401234 2720 kubelet.go:2405] "Pod admission denied" podUID="f7335c52-95fc-4f4c-a032-6e2bebdbf03a" pod="tigera-operator/tigera-operator-747864d56d-9fv9l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:49.450531 kubelet[2720]: I0813 01:26:49.450485 2720 kubelet.go:2405] "Pod admission denied" podUID="038cd2b6-b2e0-4130-8bc2-6c6d449290e8" pod="tigera-operator/tigera-operator-747864d56d-92p2w" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:49.561970 kubelet[2720]: I0813 01:26:49.560784 2720 kubelet.go:2405] "Pod admission denied" podUID="3ab50e03-4cdd-4eeb-bee7-cabb7a740d68" pod="tigera-operator/tigera-operator-747864d56d-85nmj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:49.754694 kubelet[2720]: I0813 01:26:49.754643 2720 kubelet.go:2405] "Pod admission denied" podUID="d0f3c17e-acda-4935-8431-d678b6b3ce90" pod="tigera-operator/tigera-operator-747864d56d-r9245" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:49.851739 kubelet[2720]: I0813 01:26:49.851359 2720 kubelet.go:2405] "Pod admission denied" podUID="1ffbcb66-b3fd-4447-be22-c9fcc7503f77" pod="tigera-operator/tigera-operator-747864d56d-c54kw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:49.952323 kubelet[2720]: I0813 01:26:49.952274 2720 kubelet.go:2405] "Pod admission denied" podUID="39dfac9b-2c03-4671-a106-710bb7399125" pod="tigera-operator/tigera-operator-747864d56d-clfl6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:50.050927 kubelet[2720]: I0813 01:26:50.050875 2720 kubelet.go:2405] "Pod admission denied" podUID="d0a0185e-077e-49ef-97c4-b468112d5aac" pod="tigera-operator/tigera-operator-747864d56d-5f78r" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:50.073479 containerd[1567]: time="2025-08-13T01:26:50.073420754Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:26:50.134038 kubelet[2720]: I0813 01:26:50.133267 2720 kubelet.go:2405] "Pod admission denied" podUID="b4f5bdfb-02a8-4cac-bfe1-a4aa79a1a312" pod="tigera-operator/tigera-operator-747864d56d-tr7s2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:50.142143 containerd[1567]: time="2025-08-13T01:26:50.142090037Z" level=error msg="Failed to destroy network for sandbox \"1c47659ef77ad9a0c54bd326af77664a5a80d1270c613a460819d7f8bd6ae8ef\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:50.146122 containerd[1567]: time="2025-08-13T01:26:50.146061240Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"1c47659ef77ad9a0c54bd326af77664a5a80d1270c613a460819d7f8bd6ae8ef\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:50.146536 kubelet[2720]: E0813 01:26:50.146501 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1c47659ef77ad9a0c54bd326af77664a5a80d1270c613a460819d7f8bd6ae8ef\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:50.146604 kubelet[2720]: E0813 01:26:50.146550 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1c47659ef77ad9a0c54bd326af77664a5a80d1270c613a460819d7f8bd6ae8ef\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:26:50.146604 kubelet[2720]: E0813 01:26:50.146568 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1c47659ef77ad9a0c54bd326af77664a5a80d1270c613a460819d7f8bd6ae8ef\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:26:50.146654 kubelet[2720]: E0813 01:26:50.146612 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"1c47659ef77ad9a0c54bd326af77664a5a80d1270c613a460819d7f8bd6ae8ef\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:26:50.147356 systemd[1]: run-netns-cni\x2dc28a4577\x2d0077\x2d47b6\x2d0d24\x2dc46e1cc71c18.mount: Deactivated successfully. Aug 13 01:26:50.249703 kubelet[2720]: I0813 01:26:50.249655 2720 kubelet.go:2405] "Pod admission denied" podUID="f3ae575a-3cd6-4099-a4fc-27b4f2572a8f" pod="tigera-operator/tigera-operator-747864d56d-l5mkv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:50.299832 kubelet[2720]: I0813 01:26:50.299793 2720 kubelet.go:2405] "Pod admission denied" podUID="660512da-7d00-451d-9aaf-233bdea365a7" pod="tigera-operator/tigera-operator-747864d56d-vm6fv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:50.405752 kubelet[2720]: I0813 01:26:50.405238 2720 kubelet.go:2405] "Pod admission denied" podUID="9322fb25-14c2-47ea-9f2a-571f4c3459c0" pod="tigera-operator/tigera-operator-747864d56d-2p4sl" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:50.600322 kubelet[2720]: I0813 01:26:50.600272 2720 kubelet.go:2405] "Pod admission denied" podUID="6e642cf5-c0cb-4aa9-9208-c683abdbdfea" pod="tigera-operator/tigera-operator-747864d56d-p666t" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:50.700379 kubelet[2720]: I0813 01:26:50.700250 2720 kubelet.go:2405] "Pod admission denied" podUID="c92cc851-9267-4479-8208-afc6ff219417" pod="tigera-operator/tigera-operator-747864d56d-zp9ll" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:50.801697 kubelet[2720]: I0813 01:26:50.801246 2720 kubelet.go:2405] "Pod admission denied" podUID="15eec3bd-618d-48f8-bbaa-a1162df8a336" pod="tigera-operator/tigera-operator-747864d56d-twvfq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:50.901146 kubelet[2720]: I0813 01:26:50.901090 2720 kubelet.go:2405] "Pod admission denied" podUID="3de7cc05-d6db-4240-80f0-972e0e1fdbf2" pod="tigera-operator/tigera-operator-747864d56d-pwv6v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:51.011861 kubelet[2720]: I0813 01:26:51.010752 2720 kubelet.go:2405] "Pod admission denied" podUID="79f90696-55e0-485d-a602-f9879bf125e1" pod="tigera-operator/tigera-operator-747864d56d-jcl8g" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:51.102127 kubelet[2720]: I0813 01:26:51.102076 2720 kubelet.go:2405] "Pod admission denied" podUID="78999bee-3e71-4d0a-a588-a1f3f6124ae9" pod="tigera-operator/tigera-operator-747864d56d-5qb9n" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:51.202393 kubelet[2720]: I0813 01:26:51.202339 2720 kubelet.go:2405] "Pod admission denied" podUID="7a103bbc-8c07-4442-ab7c-21ba1e4f1788" pod="tigera-operator/tigera-operator-747864d56d-9nnjf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:51.302343 kubelet[2720]: I0813 01:26:51.301819 2720 kubelet.go:2405] "Pod admission denied" podUID="beef0146-7eda-491c-bf46-213bc45e6679" pod="tigera-operator/tigera-operator-747864d56d-6f6jv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:51.356328 kubelet[2720]: I0813 01:26:51.356273 2720 kubelet.go:2405] "Pod admission denied" podUID="e63f2cfe-4d8e-4be6-b7dd-9128b8818281" pod="tigera-operator/tigera-operator-747864d56d-k7kcp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:51.457734 kubelet[2720]: I0813 01:26:51.457496 2720 kubelet.go:2405] "Pod admission denied" podUID="2f30587a-3609-4edb-9b35-aa9f5768e50f" pod="tigera-operator/tigera-operator-747864d56d-m54p8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:51.653856 kubelet[2720]: I0813 01:26:51.652880 2720 kubelet.go:2405] "Pod admission denied" podUID="c93c4387-c631-4aeb-bb43-23095730241f" pod="tigera-operator/tigera-operator-747864d56d-bl25w" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:51.752375 kubelet[2720]: I0813 01:26:51.752323 2720 kubelet.go:2405] "Pod admission denied" podUID="4a2a28f9-0d42-4bf4-a9ce-7cd45d7b48c5" pod="tigera-operator/tigera-operator-747864d56d-g6zw8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:51.849920 kubelet[2720]: I0813 01:26:51.849865 2720 kubelet.go:2405] "Pod admission denied" podUID="0c1e2053-6eb3-4e9f-b5bf-21e6c7470a61" pod="tigera-operator/tigera-operator-747864d56d-fcpm8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:52.051068 kubelet[2720]: I0813 01:26:52.051016 2720 kubelet.go:2405] "Pod admission denied" podUID="2176bec8-784c-4f0b-9c9f-504099a7eb47" pod="tigera-operator/tigera-operator-747864d56d-p8hzb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:52.079741 kubelet[2720]: E0813 01:26:52.079550 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:26:52.164344 kubelet[2720]: I0813 01:26:52.164307 2720 kubelet.go:2405] "Pod admission denied" podUID="b13c34ae-1b4c-4fcc-ab91-fa2f865c7924" pod="tigera-operator/tigera-operator-747864d56d-mhq7c" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:52.265283 kubelet[2720]: I0813 01:26:52.265227 2720 kubelet.go:2405] "Pod admission denied" podUID="149532df-0a23-4df9-acea-8f324e6367c9" pod="tigera-operator/tigera-operator-747864d56d-6b2w2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:52.452637 kubelet[2720]: I0813 01:26:52.452512 2720 kubelet.go:2405] "Pod admission denied" podUID="95986609-1618-4530-a48b-58117d61ffbb" pod="tigera-operator/tigera-operator-747864d56d-gdbbx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:52.552782 kubelet[2720]: I0813 01:26:52.552733 2720 kubelet.go:2405] "Pod admission denied" podUID="c3247bc7-2b38-4ff6-bc8b-2353eeeca51c" pod="tigera-operator/tigera-operator-747864d56d-s2lwd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:52.601888 kubelet[2720]: I0813 01:26:52.601840 2720 kubelet.go:2405] "Pod admission denied" podUID="78de37c3-d9a8-448d-bf2d-3512f7d359ee" pod="tigera-operator/tigera-operator-747864d56d-6k6ps" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:52.709924 kubelet[2720]: I0813 01:26:52.709043 2720 kubelet.go:2405] "Pod admission denied" podUID="79352ddc-114e-48fd-9746-055c109757ab" pod="tigera-operator/tigera-operator-747864d56d-wsp72" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:52.801910 kubelet[2720]: I0813 01:26:52.801857 2720 kubelet.go:2405] "Pod admission denied" podUID="6d9bbfba-b07b-4d8e-9683-9324f9215f46" pod="tigera-operator/tigera-operator-747864d56d-hwdcg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:52.899695 kubelet[2720]: I0813 01:26:52.899640 2720 kubelet.go:2405] "Pod admission denied" podUID="edf6d5cd-c72c-4e17-a953-dba626a6686a" pod="tigera-operator/tigera-operator-747864d56d-dqqw9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:52.999082 kubelet[2720]: I0813 01:26:52.999037 2720 kubelet.go:2405] "Pod admission denied" podUID="d77ca09b-c5cd-4b90-9160-eea2aa5fd73a" pod="tigera-operator/tigera-operator-747864d56d-dkq9d" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:53.073989 kubelet[2720]: E0813 01:26:53.073942 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:26:53.075083 containerd[1567]: time="2025-08-13T01:26:53.074836603Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:26:53.110755 kubelet[2720]: I0813 01:26:53.110666 2720 kubelet.go:2405] "Pod admission denied" podUID="87b33f3a-e5f9-42f8-9019-c9fcff8ddff2" pod="tigera-operator/tigera-operator-747864d56d-6xvbq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:53.150314 containerd[1567]: time="2025-08-13T01:26:53.150247814Z" level=error msg="Failed to destroy network for sandbox \"f04e39557f0df9ea33628318fde19d85b5968ef0b66c5c982a7ce4ab4f33bdd0\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:53.153419 systemd[1]: run-netns-cni\x2d52883ce6\x2d6064\x2d5bbe\x2d47c2\x2d3d71648ae779.mount: Deactivated successfully. Aug 13 01:26:53.155559 containerd[1567]: time="2025-08-13T01:26:53.153417540Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"f04e39557f0df9ea33628318fde19d85b5968ef0b66c5c982a7ce4ab4f33bdd0\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:53.155637 kubelet[2720]: E0813 01:26:53.153591 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f04e39557f0df9ea33628318fde19d85b5968ef0b66c5c982a7ce4ab4f33bdd0\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:53.155637 kubelet[2720]: E0813 01:26:53.153629 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f04e39557f0df9ea33628318fde19d85b5968ef0b66c5c982a7ce4ab4f33bdd0\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:53.155637 kubelet[2720]: E0813 01:26:53.153648 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f04e39557f0df9ea33628318fde19d85b5968ef0b66c5c982a7ce4ab4f33bdd0\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:53.155637 kubelet[2720]: E0813 01:26:53.153698 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"f04e39557f0df9ea33628318fde19d85b5968ef0b66c5c982a7ce4ab4f33bdd0\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:26:53.315843 kubelet[2720]: I0813 01:26:53.315451 2720 kubelet.go:2405] "Pod admission denied" podUID="7971ba0d-b30c-47e2-b419-96c3cbc51319" pod="tigera-operator/tigera-operator-747864d56d-gzrc8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:53.403033 kubelet[2720]: I0813 01:26:53.402988 2720 kubelet.go:2405] "Pod admission denied" podUID="0af5c059-ddcf-49a8-b484-e56301de42f8" pod="tigera-operator/tigera-operator-747864d56d-vjn7l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:53.505257 kubelet[2720]: I0813 01:26:53.505211 2720 kubelet.go:2405] "Pod admission denied" podUID="2ac719e0-e878-458b-8d64-6a2942129f0c" pod="tigera-operator/tigera-operator-747864d56d-kwk4x" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:53.704489 kubelet[2720]: I0813 01:26:53.703779 2720 kubelet.go:2405] "Pod admission denied" podUID="e4f6d377-5566-47ca-9f90-b4d22ed6626d" pod="tigera-operator/tigera-operator-747864d56d-zhn8c" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:53.802792 kubelet[2720]: I0813 01:26:53.802740 2720 kubelet.go:2405] "Pod admission denied" podUID="368bd6d1-890f-4ac2-b3b6-132558f00b86" pod="tigera-operator/tigera-operator-747864d56d-sbhdb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:53.864562 kubelet[2720]: I0813 01:26:53.864511 2720 kubelet.go:2405] "Pod admission denied" podUID="410da712-1ddc-4935-aba0-05be10f5f173" pod="tigera-operator/tigera-operator-747864d56d-wnk8k" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:53.955056 kubelet[2720]: I0813 01:26:53.954513 2720 kubelet.go:2405] "Pod admission denied" podUID="6ef80fe8-4810-4605-8c79-d39b5c37c30b" pod="tigera-operator/tigera-operator-747864d56d-jglrx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:54.154427 kubelet[2720]: I0813 01:26:54.154369 2720 kubelet.go:2405] "Pod admission denied" podUID="8998a630-643d-49e8-bf7a-2b3f7684d23a" pod="tigera-operator/tigera-operator-747864d56d-z25gs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:54.256621 kubelet[2720]: I0813 01:26:54.256572 2720 kubelet.go:2405] "Pod admission denied" podUID="caabf120-5557-4ec1-86f1-52c3b87ac03c" pod="tigera-operator/tigera-operator-747864d56d-lvc44" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:54.352417 kubelet[2720]: I0813 01:26:54.352370 2720 kubelet.go:2405] "Pod admission denied" podUID="9c17d24c-eb00-461b-8afc-d59a038db43e" pod="tigera-operator/tigera-operator-747864d56d-vtr7n" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:54.456733 kubelet[2720]: I0813 01:26:54.456390 2720 kubelet.go:2405] "Pod admission denied" podUID="08ff566d-0f68-41b4-a9e0-3ce81708b05d" pod="tigera-operator/tigera-operator-747864d56d-vk5hq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:54.551428 kubelet[2720]: I0813 01:26:54.551057 2720 kubelet.go:2405] "Pod admission denied" podUID="6bc2727b-5439-468d-9f85-aa7d67ef8e69" pod="tigera-operator/tigera-operator-747864d56d-88tnf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:54.651415 kubelet[2720]: I0813 01:26:54.651372 2720 kubelet.go:2405] "Pod admission denied" podUID="47c3b8ab-d6cf-4020-b423-ae214ba09070" pod="tigera-operator/tigera-operator-747864d56d-qkp7d" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:54.752637 kubelet[2720]: I0813 01:26:54.752590 2720 kubelet.go:2405] "Pod admission denied" podUID="4471579e-88c7-45b7-bc48-87320cdd4e05" pod="tigera-operator/tigera-operator-747864d56d-pxcbw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:54.851344 kubelet[2720]: I0813 01:26:54.850737 2720 kubelet.go:2405] "Pod admission denied" podUID="8fcf4f14-b3cf-443d-be6e-aab2b2e0bcd1" pod="tigera-operator/tigera-operator-747864d56d-cdsgg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:54.956107 kubelet[2720]: I0813 01:26:54.956068 2720 kubelet.go:2405] "Pod admission denied" podUID="c7e2738a-951a-426c-aa83-b386096f91f1" pod="tigera-operator/tigera-operator-747864d56d-h9bmt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:55.050197 kubelet[2720]: I0813 01:26:55.050152 2720 kubelet.go:2405] "Pod admission denied" podUID="a7a825db-10af-4629-b51a-7374965189c8" pod="tigera-operator/tigera-operator-747864d56d-gf94f" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:55.151965 kubelet[2720]: I0813 01:26:55.151419 2720 kubelet.go:2405] "Pod admission denied" podUID="e3a69d72-48b0-4992-ae36-5fa5ad7b5f3c" pod="tigera-operator/tigera-operator-747864d56d-hq2pk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:55.252110 kubelet[2720]: I0813 01:26:55.252061 2720 kubelet.go:2405] "Pod admission denied" podUID="824a3d2c-232a-49a6-bf60-599fc24cd743" pod="tigera-operator/tigera-operator-747864d56d-cvrxz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:55.351337 kubelet[2720]: I0813 01:26:55.351291 2720 kubelet.go:2405] "Pod admission denied" podUID="da2f9a72-43b6-467a-af42-d9286b649e89" pod="tigera-operator/tigera-operator-747864d56d-4rqvv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:55.457805 kubelet[2720]: I0813 01:26:55.457415 2720 kubelet.go:2405] "Pod admission denied" podUID="e29d2a7a-f79a-43da-b45e-477eca243101" pod="tigera-operator/tigera-operator-747864d56d-vk9hr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:55.551057 kubelet[2720]: I0813 01:26:55.551013 2720 kubelet.go:2405] "Pod admission denied" podUID="a75a70cf-789e-4f1f-965e-9c08b0ae00b9" pod="tigera-operator/tigera-operator-747864d56d-h4q4z" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:55.651139 kubelet[2720]: I0813 01:26:55.651094 2720 kubelet.go:2405] "Pod admission denied" podUID="ee5f2f5e-a116-4994-9c79-895a72952f4e" pod="tigera-operator/tigera-operator-747864d56d-hkdk9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:55.752494 kubelet[2720]: I0813 01:26:55.752448 2720 kubelet.go:2405] "Pod admission denied" podUID="e9461e79-5dcc-4292-b994-7380736ed0b7" pod="tigera-operator/tigera-operator-747864d56d-hnskz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:55.854470 kubelet[2720]: I0813 01:26:55.854430 2720 kubelet.go:2405] "Pod admission denied" podUID="228a749d-337f-4b34-aff4-b9f5b3b8cab6" pod="tigera-operator/tigera-operator-747864d56d-v9f7g" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:55.973598 kubelet[2720]: I0813 01:26:55.972054 2720 kubelet.go:2405] "Pod admission denied" podUID="4126463e-5586-4796-821a-2cd7b02585ae" pod="tigera-operator/tigera-operator-747864d56d-m95mz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:56.052239 kubelet[2720]: I0813 01:26:56.052128 2720 kubelet.go:2405] "Pod admission denied" podUID="96a87e5b-6986-46d4-b164-5d45a743baf7" pod="tigera-operator/tigera-operator-747864d56d-4fzx9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:56.074585 kubelet[2720]: E0813 01:26:56.074301 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"calico-node\" with ImagePullBackOff: \"Back-off pulling image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": ErrImagePull: failed to pull and unpack image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1682018348: write /var/lib/containerd/tmpmounts/containerd-mount1682018348/usr/bin/calico-node: no space left on device\"" pod="calico-system/calico-node-2p8hj" podUID="4f334296-e8c0-427e-a090-d8fbd3582f26" Aug 13 01:26:56.156428 kubelet[2720]: I0813 01:26:56.156391 2720 kubelet.go:2405] "Pod admission denied" podUID="8bf15141-e7ea-40b9-ad83-6b5bf2683c19" pod="tigera-operator/tigera-operator-747864d56d-l6mt2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:56.252786 kubelet[2720]: I0813 01:26:56.252748 2720 kubelet.go:2405] "Pod admission denied" podUID="a2f642b1-9c9c-474c-86b6-2f3828fc97da" pod="tigera-operator/tigera-operator-747864d56d-zrmg2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:56.301492 kubelet[2720]: I0813 01:26:56.301447 2720 kubelet.go:2405] "Pod admission denied" podUID="f972a165-88fa-4479-a29d-bc9aeeb5f11a" pod="tigera-operator/tigera-operator-747864d56d-9lwv5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:56.401840 kubelet[2720]: I0813 01:26:56.401213 2720 kubelet.go:2405] "Pod admission denied" podUID="f7aa298b-0976-4f42-b05e-3b815f9c198e" pod="tigera-operator/tigera-operator-747864d56d-9qj9l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:56.500843 kubelet[2720]: I0813 01:26:56.500802 2720 kubelet.go:2405] "Pod admission denied" podUID="5e5d6c5b-da84-4dd4-a124-4f662de5c9ac" pod="tigera-operator/tigera-operator-747864d56d-4vzbp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:56.555059 kubelet[2720]: I0813 01:26:56.554184 2720 kubelet.go:2405] "Pod admission denied" podUID="08b09d8f-bee8-4594-a0a3-096c1fc50157" pod="tigera-operator/tigera-operator-747864d56d-v5j4t" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:56.650227 kubelet[2720]: I0813 01:26:56.650186 2720 kubelet.go:2405] "Pod admission denied" podUID="1e619149-8361-4eab-91d8-12c1f30d0cd6" pod="tigera-operator/tigera-operator-747864d56d-6dxgd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:56.750045 kubelet[2720]: I0813 01:26:56.750004 2720 kubelet.go:2405] "Pod admission denied" podUID="5250c987-7233-4d7e-8dfb-87ef2333bba1" pod="tigera-operator/tigera-operator-747864d56d-fjqgq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:56.801679 kubelet[2720]: I0813 01:26:56.801630 2720 kubelet.go:2405] "Pod admission denied" podUID="1363d632-6994-49b1-b965-7738c6ea0cf8" pod="tigera-operator/tigera-operator-747864d56d-b86nd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:56.901901 kubelet[2720]: I0813 01:26:56.901856 2720 kubelet.go:2405] "Pod admission denied" podUID="b0515cb6-81d5-46f6-ab86-cff7970b0f26" pod="tigera-operator/tigera-operator-747864d56d-5cdfz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:57.008839 kubelet[2720]: I0813 01:26:57.006604 2720 kubelet.go:2405] "Pod admission denied" podUID="0f1da376-e4c8-497a-b630-888d642133b4" pod="tigera-operator/tigera-operator-747864d56d-jnshc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:57.100580 kubelet[2720]: I0813 01:26:57.100533 2720 kubelet.go:2405] "Pod admission denied" podUID="811fcdfe-6d31-487c-a2f8-5325b0249353" pod="tigera-operator/tigera-operator-747864d56d-b6x58" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:57.305639 kubelet[2720]: I0813 01:26:57.305166 2720 kubelet.go:2405] "Pod admission denied" podUID="656e697f-13c0-4dd7-8d82-21ade3d4e5d1" pod="tigera-operator/tigera-operator-747864d56d-hkwlp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:57.401527 kubelet[2720]: I0813 01:26:57.401482 2720 kubelet.go:2405] "Pod admission denied" podUID="5fc3981c-1644-4568-b114-e6a6e55ad1cb" pod="tigera-operator/tigera-operator-747864d56d-ft5gs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:57.449823 kubelet[2720]: I0813 01:26:57.449785 2720 kubelet.go:2405] "Pod admission denied" podUID="e03eaba1-1604-4c8f-8216-8b500ed59c79" pod="tigera-operator/tigera-operator-747864d56d-fsdgj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:57.553734 kubelet[2720]: I0813 01:26:57.553667 2720 kubelet.go:2405] "Pod admission denied" podUID="3c7eab35-7d1e-4cc2-b8b1-5fd2e52b9f92" pod="tigera-operator/tigera-operator-747864d56d-tchmp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:57.650888 kubelet[2720]: I0813 01:26:57.650407 2720 kubelet.go:2405] "Pod admission denied" podUID="ebd0d035-ee45-4ce4-ad9b-a492556c70d7" pod="tigera-operator/tigera-operator-747864d56d-9ctm2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:57.697837 kubelet[2720]: I0813 01:26:57.697786 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:26:57.697837 kubelet[2720]: I0813 01:26:57.697820 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:26:57.700172 kubelet[2720]: I0813 01:26:57.700091 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:26:57.709578 kubelet[2720]: I0813 01:26:57.709546 2720 kubelet.go:2405] "Pod admission denied" podUID="7fc2f968-8eb0-498a-ab9e-f1053dfe5ee0" pod="tigera-operator/tigera-operator-747864d56d-kv262" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:57.716123 kubelet[2720]: I0813 01:26:57.716103 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:26:57.716220 kubelet[2720]: I0813 01:26:57.716167 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","calico-system/calico-node-2p8hj","calico-system/csi-node-driver-8bzzs","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:26:57.716220 kubelet[2720]: E0813 01:26:57.716190 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:26:57.716220 kubelet[2720]: E0813 01:26:57.716200 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:26:57.716220 kubelet[2720]: E0813 01:26:57.716205 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:26:57.716220 kubelet[2720]: E0813 01:26:57.716211 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:26:57.716220 kubelet[2720]: E0813 01:26:57.716217 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:57.716436 kubelet[2720]: E0813 01:26:57.716226 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:26:57.716436 kubelet[2720]: E0813 01:26:57.716234 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:26:57.716436 kubelet[2720]: E0813 01:26:57.716241 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:26:57.716436 kubelet[2720]: E0813 01:26:57.716247 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:26:57.716436 kubelet[2720]: E0813 01:26:57.716254 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:26:57.716436 kubelet[2720]: I0813 01:26:57.716262 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:26:57.806986 kubelet[2720]: I0813 01:26:57.806933 2720 kubelet.go:2405] "Pod admission denied" podUID="461046b8-e9ce-48ee-a590-9c47098e20d9" pod="tigera-operator/tigera-operator-747864d56d-zwg6w" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:58.007025 kubelet[2720]: I0813 01:26:58.006971 2720 kubelet.go:2405] "Pod admission denied" podUID="9ba7202e-3413-449c-ac7b-687fe57ab169" pod="tigera-operator/tigera-operator-747864d56d-82fh6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:58.074015 containerd[1567]: time="2025-08-13T01:26:58.073758296Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:26:58.131849 kubelet[2720]: I0813 01:26:58.131298 2720 kubelet.go:2405] "Pod admission denied" podUID="d4276849-1eab-4b1b-91de-aa013bb4ac8d" pod="tigera-operator/tigera-operator-747864d56d-d5kkn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:58.139018 containerd[1567]: time="2025-08-13T01:26:58.138982284Z" level=error msg="Failed to destroy network for sandbox \"302873141e9eea8ed8ab62c1a98ab7295c505bd3a85b3a385499d2aa9632a859\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:58.144921 systemd[1]: run-netns-cni\x2d81b1f2a6\x2dc0bc\x2d6235\x2d010b\x2d4ee5ab1701da.mount: Deactivated successfully. Aug 13 01:26:58.148742 containerd[1567]: time="2025-08-13T01:26:58.147058177Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"302873141e9eea8ed8ab62c1a98ab7295c505bd3a85b3a385499d2aa9632a859\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:58.148835 kubelet[2720]: E0813 01:26:58.147402 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"302873141e9eea8ed8ab62c1a98ab7295c505bd3a85b3a385499d2aa9632a859\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:26:58.148835 kubelet[2720]: E0813 01:26:58.147460 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"302873141e9eea8ed8ab62c1a98ab7295c505bd3a85b3a385499d2aa9632a859\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:58.148835 kubelet[2720]: E0813 01:26:58.147477 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"302873141e9eea8ed8ab62c1a98ab7295c505bd3a85b3a385499d2aa9632a859\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:26:58.148835 kubelet[2720]: E0813 01:26:58.147533 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"302873141e9eea8ed8ab62c1a98ab7295c505bd3a85b3a385499d2aa9632a859\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:26:58.169457 kubelet[2720]: I0813 01:26:58.169390 2720 kubelet.go:2405] "Pod admission denied" podUID="0f0e0ce1-94e9-44ab-bb71-80ebd734e595" pod="tigera-operator/tigera-operator-747864d56d-7lxhz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:58.253430 kubelet[2720]: I0813 01:26:58.253387 2720 kubelet.go:2405] "Pod admission denied" podUID="8ce109b9-3050-460d-8122-af3e79255417" pod="tigera-operator/tigera-operator-747864d56d-t7xhf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:58.454486 kubelet[2720]: I0813 01:26:58.453934 2720 kubelet.go:2405] "Pod admission denied" podUID="94f43d57-cf14-4684-a504-ba56dc52a689" pod="tigera-operator/tigera-operator-747864d56d-sdwqj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:58.551085 kubelet[2720]: I0813 01:26:58.551027 2720 kubelet.go:2405] "Pod admission denied" podUID="819f1d52-8f83-414c-92dd-373c39dbe560" pod="tigera-operator/tigera-operator-747864d56d-w6pcl" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:58.662397 kubelet[2720]: I0813 01:26:58.662054 2720 kubelet.go:2405] "Pod admission denied" podUID="c16be2f0-5c0b-48a3-bd04-f8f54d8caba5" pod="tigera-operator/tigera-operator-747864d56d-fr58p" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:58.854408 kubelet[2720]: I0813 01:26:58.854343 2720 kubelet.go:2405] "Pod admission denied" podUID="9c0e0d6c-7489-4ec5-bcd6-a1979ac818fa" pod="tigera-operator/tigera-operator-747864d56d-69nwr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:58.952645 kubelet[2720]: I0813 01:26:58.952600 2720 kubelet.go:2405] "Pod admission denied" podUID="e6598e9b-5be7-40cb-bd58-eb13661ca4bc" pod="tigera-operator/tigera-operator-747864d56d-x6svv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:59.054346 kubelet[2720]: I0813 01:26:59.054285 2720 kubelet.go:2405] "Pod admission denied" podUID="e3e61675-e1be-47ab-afb8-8570c5678a7f" pod="tigera-operator/tigera-operator-747864d56d-5m88s" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:59.153075 kubelet[2720]: I0813 01:26:59.152623 2720 kubelet.go:2405] "Pod admission denied" podUID="a4f42436-29fc-4fc4-9cc8-0d7008ce09dd" pod="tigera-operator/tigera-operator-747864d56d-hdz27" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:59.266572 kubelet[2720]: I0813 01:26:59.266512 2720 kubelet.go:2405] "Pod admission denied" podUID="9c037c67-68e6-4bff-aa8a-f8a523201bfc" pod="tigera-operator/tigera-operator-747864d56d-2j2mm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:59.354518 kubelet[2720]: I0813 01:26:59.354472 2720 kubelet.go:2405] "Pod admission denied" podUID="22a1a557-49e3-4b05-8abd-fc4e13192fb5" pod="tigera-operator/tigera-operator-747864d56d-7p4l5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:59.452976 kubelet[2720]: I0813 01:26:59.452854 2720 kubelet.go:2405] "Pod admission denied" podUID="762cbd14-d6a2-4e33-8f01-161937153e19" pod="tigera-operator/tigera-operator-747864d56d-zqrw7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:59.655338 kubelet[2720]: I0813 01:26:59.655297 2720 kubelet.go:2405] "Pod admission denied" podUID="a7365dca-e750-4594-a9e8-034f9ed0d8fe" pod="tigera-operator/tigera-operator-747864d56d-kd58g" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:59.751503 kubelet[2720]: I0813 01:26:59.751454 2720 kubelet.go:2405] "Pod admission denied" podUID="66a82bde-9be4-4ce8-8600-6c09114a001e" pod="tigera-operator/tigera-operator-747864d56d-4lxbp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:26:59.866147 kubelet[2720]: I0813 01:26:59.865442 2720 kubelet.go:2405] "Pod admission denied" podUID="f5f7d851-a733-4d78-9996-f8e9fcdf167b" pod="tigera-operator/tigera-operator-747864d56d-rs8j9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:00.053052 kubelet[2720]: I0813 01:27:00.052839 2720 kubelet.go:2405] "Pod admission denied" podUID="ce609427-797e-4d57-bb43-522f27a40dee" pod="tigera-operator/tigera-operator-747864d56d-cc8dn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:00.073823 kubelet[2720]: E0813 01:27:00.073766 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:27:00.153325 kubelet[2720]: I0813 01:27:00.153278 2720 kubelet.go:2405] "Pod admission denied" podUID="c0e07521-768f-4fb1-b554-0f04d846b34e" pod="tigera-operator/tigera-operator-747864d56d-8t5kt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:00.252679 kubelet[2720]: I0813 01:27:00.252627 2720 kubelet.go:2405] "Pod admission denied" podUID="3ecd6757-1a80-4340-a8e4-27a0765c5705" pod="tigera-operator/tigera-operator-747864d56d-llkvq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:00.353680 kubelet[2720]: I0813 01:27:00.353115 2720 kubelet.go:2405] "Pod admission denied" podUID="6adec13a-60c4-4dd7-8cb0-da64fb0bc1a8" pod="tigera-operator/tigera-operator-747864d56d-x9k2q" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:00.460735 kubelet[2720]: I0813 01:27:00.460553 2720 kubelet.go:2405] "Pod admission denied" podUID="1966e615-4c40-4377-8e38-c521cd0a7ced" pod="tigera-operator/tigera-operator-747864d56d-5kkhs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:00.551265 kubelet[2720]: I0813 01:27:00.551214 2720 kubelet.go:2405] "Pod admission denied" podUID="6e920c93-f320-4f10-bf8f-2562cadbbccb" pod="tigera-operator/tigera-operator-747864d56d-wwqxn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:00.603016 kubelet[2720]: I0813 01:27:00.602956 2720 kubelet.go:2405] "Pod admission denied" podUID="bdd7a8ee-cdb7-492a-a63f-f191c75179fb" pod="tigera-operator/tigera-operator-747864d56d-hjtc8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:00.704270 kubelet[2720]: I0813 01:27:00.703972 2720 kubelet.go:2405] "Pod admission denied" podUID="fd2ba1da-1612-4e63-9967-afcd20af69ed" pod="tigera-operator/tigera-operator-747864d56d-5wfqd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:00.802253 kubelet[2720]: I0813 01:27:00.802206 2720 kubelet.go:2405] "Pod admission denied" podUID="3c833291-fec6-4ee6-8d87-c6be477614c6" pod="tigera-operator/tigera-operator-747864d56d-7bptd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:00.911824 kubelet[2720]: I0813 01:27:00.911691 2720 kubelet.go:2405] "Pod admission denied" podUID="3a764ba3-04ec-420a-ab55-9e101bcaa6d2" pod="tigera-operator/tigera-operator-747864d56d-9sfnm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:01.001475 kubelet[2720]: I0813 01:27:01.001422 2720 kubelet.go:2405] "Pod admission denied" podUID="127e5a17-4fbb-416f-8925-94a13037093f" pod="tigera-operator/tigera-operator-747864d56d-psksd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:01.101956 kubelet[2720]: I0813 01:27:01.101905 2720 kubelet.go:2405] "Pod admission denied" podUID="6ffeebf2-05fc-4292-9fa3-e16944b3d03d" pod="tigera-operator/tigera-operator-747864d56d-cnffw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:01.203067 kubelet[2720]: I0813 01:27:01.203014 2720 kubelet.go:2405] "Pod admission denied" podUID="33d1a604-df7a-4b17-adca-40e85101aa29" pod="tigera-operator/tigera-operator-747864d56d-q8jlz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:01.306389 kubelet[2720]: I0813 01:27:01.306083 2720 kubelet.go:2405] "Pod admission denied" podUID="7f1f6a00-450a-4ea4-86fe-a8b8d1be13e2" pod="tigera-operator/tigera-operator-747864d56d-59zrl" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:01.415979 kubelet[2720]: I0813 01:27:01.414908 2720 kubelet.go:2405] "Pod admission denied" podUID="80b3b19c-cce7-4153-ae11-053a441507a2" pod="tigera-operator/tigera-operator-747864d56d-x2f79" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:01.503804 kubelet[2720]: I0813 01:27:01.503756 2720 kubelet.go:2405] "Pod admission denied" podUID="f65f2023-b7fd-4bd5-a7ae-38477dab537a" pod="tigera-operator/tigera-operator-747864d56d-w2ch2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:01.605596 kubelet[2720]: I0813 01:27:01.604936 2720 kubelet.go:2405] "Pod admission denied" podUID="26cc0ed0-ad95-4616-a6a3-11bc93427d1d" pod="tigera-operator/tigera-operator-747864d56d-qzft7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:01.705954 kubelet[2720]: I0813 01:27:01.705904 2720 kubelet.go:2405] "Pod admission denied" podUID="630a51e8-03be-48ca-a48a-f17f590a73e3" pod="tigera-operator/tigera-operator-747864d56d-gc4cc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:01.804114 kubelet[2720]: I0813 01:27:01.804068 2720 kubelet.go:2405] "Pod admission denied" podUID="c6bb2a9f-5fad-474a-b528-7d9aebd3d079" pod="tigera-operator/tigera-operator-747864d56d-zb45f" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:01.911004 kubelet[2720]: I0813 01:27:01.910285 2720 kubelet.go:2405] "Pod admission denied" podUID="b5bb9fa7-300e-4792-baa0-ce086a8e9a25" pod="tigera-operator/tigera-operator-747864d56d-2jg49" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:02.077791 kubelet[2720]: E0813 01:27:02.077765 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:27:02.078824 containerd[1567]: time="2025-08-13T01:27:02.078639770Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:27:02.121521 kubelet[2720]: I0813 01:27:02.121482 2720 kubelet.go:2405] "Pod admission denied" podUID="37426898-0a5a-461c-9879-2a2a74c929db" pod="tigera-operator/tigera-operator-747864d56d-nd6gt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:02.150040 containerd[1567]: time="2025-08-13T01:27:02.149903757Z" level=error msg="Failed to destroy network for sandbox \"b78e076382bd980e9b2bb469efca64eb87a6db9d75488ddcc0828b36c4493fef\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:02.153486 containerd[1567]: time="2025-08-13T01:27:02.153447057Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"b78e076382bd980e9b2bb469efca64eb87a6db9d75488ddcc0828b36c4493fef\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:02.153776 kubelet[2720]: E0813 01:27:02.153707 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b78e076382bd980e9b2bb469efca64eb87a6db9d75488ddcc0828b36c4493fef\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:02.153898 kubelet[2720]: E0813 01:27:02.153881 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b78e076382bd980e9b2bb469efca64eb87a6db9d75488ddcc0828b36c4493fef\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:27:02.154851 systemd[1]: run-netns-cni\x2d8c2753d1\x2d9541\x2d7010\x2de936\x2d56d78a6bb4c9.mount: Deactivated successfully. Aug 13 01:27:02.158739 kubelet[2720]: E0813 01:27:02.158378 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b78e076382bd980e9b2bb469efca64eb87a6db9d75488ddcc0828b36c4493fef\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:27:02.158739 kubelet[2720]: E0813 01:27:02.158432 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"b78e076382bd980e9b2bb469efca64eb87a6db9d75488ddcc0828b36c4493fef\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:27:02.206227 kubelet[2720]: I0813 01:27:02.206126 2720 kubelet.go:2405] "Pod admission denied" podUID="7e7dce2a-b436-4f81-8159-4a86be6d2924" pod="tigera-operator/tigera-operator-747864d56d-7pl8v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:02.302601 kubelet[2720]: I0813 01:27:02.302556 2720 kubelet.go:2405] "Pod admission denied" podUID="da62742d-7f9f-4b87-837f-d4692ac453f3" pod="tigera-operator/tigera-operator-747864d56d-wkfzw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:02.506532 kubelet[2720]: I0813 01:27:02.506493 2720 kubelet.go:2405] "Pod admission denied" podUID="8938b2e7-557a-45c5-826b-76469c862f73" pod="tigera-operator/tigera-operator-747864d56d-2rs7b" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:02.603999 kubelet[2720]: I0813 01:27:02.603953 2720 kubelet.go:2405] "Pod admission denied" podUID="cdaa2733-a593-45a1-8100-2c5468d6d984" pod="tigera-operator/tigera-operator-747864d56d-vc5hv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:02.706934 kubelet[2720]: I0813 01:27:02.706883 2720 kubelet.go:2405] "Pod admission denied" podUID="a1d15216-5118-4374-bd02-29a0dd5c17af" pod="tigera-operator/tigera-operator-747864d56d-hzq8n" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:02.812954 kubelet[2720]: I0813 01:27:02.812451 2720 kubelet.go:2405] "Pod admission denied" podUID="2381c04c-b4e2-44b7-880f-b38e9b376e3d" pod="tigera-operator/tigera-operator-747864d56d-kspjp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:02.902770 kubelet[2720]: I0813 01:27:02.902709 2720 kubelet.go:2405] "Pod admission denied" podUID="531bc7a5-3faf-410a-afbc-c464e9dbbc81" pod="tigera-operator/tigera-operator-747864d56d-mqr6q" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:03.004111 kubelet[2720]: I0813 01:27:03.004061 2720 kubelet.go:2405] "Pod admission denied" podUID="f56a752b-6d3f-431f-9378-ee2fbf5eb234" pod="tigera-operator/tigera-operator-747864d56d-qfrrl" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:03.052769 kubelet[2720]: I0813 01:27:03.052708 2720 kubelet.go:2405] "Pod admission denied" podUID="37566158-0d0c-41d8-83ee-187d85532fe4" pod="tigera-operator/tigera-operator-747864d56d-d4n9t" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:03.074029 containerd[1567]: time="2025-08-13T01:27:03.073611897Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:27:03.125258 containerd[1567]: time="2025-08-13T01:27:03.125209592Z" level=error msg="Failed to destroy network for sandbox \"10939439413d7928fd22694700a5e0b0719e99ad60361a9d19eddd7aacff7e04\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:03.127296 systemd[1]: run-netns-cni\x2d5105f1dc\x2d75d0\x2ddef6\x2d7268\x2d63f4ccd68c9a.mount: Deactivated successfully. Aug 13 01:27:03.129341 containerd[1567]: time="2025-08-13T01:27:03.129295818Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"10939439413d7928fd22694700a5e0b0719e99ad60361a9d19eddd7aacff7e04\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:03.129938 kubelet[2720]: E0813 01:27:03.129521 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"10939439413d7928fd22694700a5e0b0719e99ad60361a9d19eddd7aacff7e04\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:03.129938 kubelet[2720]: E0813 01:27:03.129574 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"10939439413d7928fd22694700a5e0b0719e99ad60361a9d19eddd7aacff7e04\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:27:03.129938 kubelet[2720]: E0813 01:27:03.129596 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"10939439413d7928fd22694700a5e0b0719e99ad60361a9d19eddd7aacff7e04\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:27:03.129938 kubelet[2720]: E0813 01:27:03.129645 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"10939439413d7928fd22694700a5e0b0719e99ad60361a9d19eddd7aacff7e04\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:27:03.158114 kubelet[2720]: I0813 01:27:03.158066 2720 kubelet.go:2405] "Pod admission denied" podUID="a2e47488-2c95-4601-9496-06149822e177" pod="tigera-operator/tigera-operator-747864d56d-mqz5g" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:03.368822 kubelet[2720]: I0813 01:27:03.367275 2720 kubelet.go:2405] "Pod admission denied" podUID="3a080705-c40c-471b-a55e-57c241c7043e" pod="tigera-operator/tigera-operator-747864d56d-shx8d" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:03.451245 kubelet[2720]: I0813 01:27:03.451195 2720 kubelet.go:2405] "Pod admission denied" podUID="c8e10b9d-28ef-41f9-9f78-75b5dca36ecc" pod="tigera-operator/tigera-operator-747864d56d-ghf5p" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:03.549985 kubelet[2720]: I0813 01:27:03.549936 2720 kubelet.go:2405] "Pod admission denied" podUID="b717d30c-f5d0-481f-8003-5f580041bd6d" pod="tigera-operator/tigera-operator-747864d56d-cg72n" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:03.754514 kubelet[2720]: I0813 01:27:03.754446 2720 kubelet.go:2405] "Pod admission denied" podUID="099fd035-52f0-4d53-b68f-b25489520a01" pod="tigera-operator/tigera-operator-747864d56d-5z2x8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:03.855301 kubelet[2720]: I0813 01:27:03.855034 2720 kubelet.go:2405] "Pod admission denied" podUID="ea561bdc-4e35-4010-80bd-55341825d43a" pod="tigera-operator/tigera-operator-747864d56d-wz6gj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:03.967182 kubelet[2720]: I0813 01:27:03.967099 2720 kubelet.go:2405] "Pod admission denied" podUID="1eefa596-c333-4ed4-9042-f8d517f4b42c" pod="tigera-operator/tigera-operator-747864d56d-ztxxl" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:04.156704 kubelet[2720]: I0813 01:27:04.155967 2720 kubelet.go:2405] "Pod admission denied" podUID="3501ae17-8e2d-411b-84cd-7df926d7d49f" pod="tigera-operator/tigera-operator-747864d56d-hzn99" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:04.257442 kubelet[2720]: I0813 01:27:04.257385 2720 kubelet.go:2405] "Pod admission denied" podUID="200447ba-e653-4614-b89e-10a63d85eab4" pod="tigera-operator/tigera-operator-747864d56d-lqmpv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:04.354434 kubelet[2720]: I0813 01:27:04.354386 2720 kubelet.go:2405] "Pod admission denied" podUID="6b71cfab-bdb3-4840-8c4b-f864e1e60ad7" pod="tigera-operator/tigera-operator-747864d56d-vv8xt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:04.455015 kubelet[2720]: I0813 01:27:04.454888 2720 kubelet.go:2405] "Pod admission denied" podUID="86783dad-9305-4340-bc15-6464963df828" pod="tigera-operator/tigera-operator-747864d56d-gmv8d" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:04.564494 kubelet[2720]: I0813 01:27:04.564136 2720 kubelet.go:2405] "Pod admission denied" podUID="1d549f38-39d7-4f7b-8699-d7c39de09f3d" pod="tigera-operator/tigera-operator-747864d56d-m5m96" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:04.757348 kubelet[2720]: I0813 01:27:04.757292 2720 kubelet.go:2405] "Pod admission denied" podUID="9c44bee1-7273-4d3d-b0d8-0df962db2a65" pod="tigera-operator/tigera-operator-747864d56d-9k8vb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:04.856961 kubelet[2720]: I0813 01:27:04.856919 2720 kubelet.go:2405] "Pod admission denied" podUID="5a7408f4-b32c-4c46-a6cf-fbc1a1f75009" pod="tigera-operator/tigera-operator-747864d56d-dbfdf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:04.905850 kubelet[2720]: I0813 01:27:04.905793 2720 kubelet.go:2405] "Pod admission denied" podUID="b99e52a6-879b-4589-9ab5-1c38a3c874af" pod="tigera-operator/tigera-operator-747864d56d-bjxb7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:05.002090 kubelet[2720]: I0813 01:27:05.002051 2720 kubelet.go:2405] "Pod admission denied" podUID="20ba9627-d72e-40e2-9c05-11e34fb743fe" pod="tigera-operator/tigera-operator-747864d56d-fgnkv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:05.116834 kubelet[2720]: I0813 01:27:05.114545 2720 kubelet.go:2405] "Pod admission denied" podUID="d08a1397-9fcf-4695-8a10-331e8a7628b0" pod="tigera-operator/tigera-operator-747864d56d-cg5hp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:05.200816 kubelet[2720]: I0813 01:27:05.200769 2720 kubelet.go:2405] "Pod admission denied" podUID="7d4a88e5-8265-49bf-89ce-eb91b043a118" pod="tigera-operator/tigera-operator-747864d56d-qmrm9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:05.305764 kubelet[2720]: I0813 01:27:05.305697 2720 kubelet.go:2405] "Pod admission denied" podUID="3bf2fcaf-4b3d-41ca-a77a-6f697e89c78a" pod="tigera-operator/tigera-operator-747864d56d-7wr69" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:05.405958 kubelet[2720]: I0813 01:27:05.405347 2720 kubelet.go:2405] "Pod admission denied" podUID="c764dd9a-9d1e-4461-889e-9441c4f4f5dc" pod="tigera-operator/tigera-operator-747864d56d-bzgs6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:05.505394 kubelet[2720]: I0813 01:27:05.505353 2720 kubelet.go:2405] "Pod admission denied" podUID="ff7f80ef-b2db-4e58-aced-e485e7a1d106" pod="tigera-operator/tigera-operator-747864d56d-s6d2c" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:05.563535 kubelet[2720]: I0813 01:27:05.563502 2720 kubelet.go:2405] "Pod admission denied" podUID="24a97fd9-27c0-4718-a960-e40dc43435de" pod="tigera-operator/tigera-operator-747864d56d-7zrqx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:05.654484 kubelet[2720]: I0813 01:27:05.654432 2720 kubelet.go:2405] "Pod admission denied" podUID="34b065d0-37b3-42b9-8c0f-643a3c53c529" pod="tigera-operator/tigera-operator-747864d56d-t7wfm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:05.758458 kubelet[2720]: I0813 01:27:05.758404 2720 kubelet.go:2405] "Pod admission denied" podUID="2335db62-43bd-4729-8dcc-5244968af4eb" pod="tigera-operator/tigera-operator-747864d56d-7frkq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:05.853952 kubelet[2720]: I0813 01:27:05.853896 2720 kubelet.go:2405] "Pod admission denied" podUID="cdcfeb50-15d7-4c3e-b011-3fd89c8be5b4" pod="tigera-operator/tigera-operator-747864d56d-ptq5q" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:05.956819 kubelet[2720]: I0813 01:27:05.956771 2720 kubelet.go:2405] "Pod admission denied" podUID="f7be8470-81d4-4ffe-8905-18f74bf80d8a" pod="tigera-operator/tigera-operator-747864d56d-qfgxh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:06.064970 kubelet[2720]: I0813 01:27:06.063850 2720 kubelet.go:2405] "Pod admission denied" podUID="80e01037-22c9-4a8c-b438-1c544ea326b4" pod="tigera-operator/tigera-operator-747864d56d-5w49s" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:06.154293 kubelet[2720]: I0813 01:27:06.154241 2720 kubelet.go:2405] "Pod admission denied" podUID="c9580b56-91b8-4c34-b3ef-f5df23c6fd91" pod="tigera-operator/tigera-operator-747864d56d-z2mgx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:06.206482 kubelet[2720]: I0813 01:27:06.206431 2720 kubelet.go:2405] "Pod admission denied" podUID="83f41023-a36f-4d9c-a1f5-a349c3425914" pod="tigera-operator/tigera-operator-747864d56d-w9d8p" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:06.306004 kubelet[2720]: I0813 01:27:06.305964 2720 kubelet.go:2405] "Pod admission denied" podUID="0c831469-1bc7-41a3-bce7-187c88ac5db7" pod="tigera-operator/tigera-operator-747864d56d-srq5p" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:06.407520 kubelet[2720]: I0813 01:27:06.407061 2720 kubelet.go:2405] "Pod admission denied" podUID="8b2d1417-8fac-4eef-bfc5-ff16eaea725e" pod="tigera-operator/tigera-operator-747864d56d-84jwf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:06.524447 kubelet[2720]: I0813 01:27:06.524395 2720 kubelet.go:2405] "Pod admission denied" podUID="a5afcc18-0693-4c93-8b17-3a5f6433541a" pod="tigera-operator/tigera-operator-747864d56d-dgfh4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:06.708157 kubelet[2720]: I0813 01:27:06.707551 2720 kubelet.go:2405] "Pod admission denied" podUID="f987bd1b-2e0c-4c20-ab4f-e6dfeba872f2" pod="tigera-operator/tigera-operator-747864d56d-vvxnw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:06.812935 kubelet[2720]: I0813 01:27:06.812874 2720 kubelet.go:2405] "Pod admission denied" podUID="00b3106b-f419-46c1-a5e4-ecd7728d8d88" pod="tigera-operator/tigera-operator-747864d56d-rjzkp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:06.904328 kubelet[2720]: I0813 01:27:06.904276 2720 kubelet.go:2405] "Pod admission denied" podUID="6e6de723-bb54-4ec0-9fa8-e6f1e11361d0" pod="tigera-operator/tigera-operator-747864d56d-qsjrk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:07.001623 kubelet[2720]: I0813 01:27:07.001581 2720 kubelet.go:2405] "Pod admission denied" podUID="b6e4c800-1782-46d1-b21f-79b9049df70d" pod="tigera-operator/tigera-operator-747864d56d-z9qqs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:07.122429 kubelet[2720]: I0813 01:27:07.122392 2720 kubelet.go:2405] "Pod admission denied" podUID="cc048290-ed5b-4ac4-9861-bfacd4ef19e2" pod="tigera-operator/tigera-operator-747864d56d-vxkzh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:07.307480 kubelet[2720]: I0813 01:27:07.307325 2720 kubelet.go:2405] "Pod admission denied" podUID="4efee35e-a364-4b51-822a-a91b24ac7082" pod="tigera-operator/tigera-operator-747864d56d-zp7td" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:07.404153 kubelet[2720]: I0813 01:27:07.404108 2720 kubelet.go:2405] "Pod admission denied" podUID="8c68fb58-7efc-4232-b191-8c0bcc621288" pod="tigera-operator/tigera-operator-747864d56d-b52zb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:07.504365 kubelet[2720]: I0813 01:27:07.504320 2720 kubelet.go:2405] "Pod admission denied" podUID="c071303a-f390-41b0-8c44-d164242b03f8" pod="tigera-operator/tigera-operator-747864d56d-wpkz6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:07.604578 kubelet[2720]: I0813 01:27:07.604459 2720 kubelet.go:2405] "Pod admission denied" podUID="e51a2fdb-0db7-408e-a66f-7f2752c34dcc" pod="tigera-operator/tigera-operator-747864d56d-mzm6d" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:07.711508 kubelet[2720]: I0813 01:27:07.710424 2720 kubelet.go:2405] "Pod admission denied" podUID="b9b6f40e-025a-4e1e-8091-f4c5cf0d5f93" pod="tigera-operator/tigera-operator-747864d56d-mvmhc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:07.733155 kubelet[2720]: I0813 01:27:07.732604 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:27:07.733155 kubelet[2720]: I0813 01:27:07.732636 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:27:07.734724 kubelet[2720]: I0813 01:27:07.734611 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:27:07.747473 kubelet[2720]: I0813 01:27:07.747435 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:27:07.747730 kubelet[2720]: I0813 01:27:07.747656 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","calico-system/calico-node-2p8hj","calico-system/csi-node-driver-8bzzs","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:27:07.747730 kubelet[2720]: E0813 01:27:07.747684 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:27:07.747730 kubelet[2720]: E0813 01:27:07.747693 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:27:07.747985 kubelet[2720]: E0813 01:27:07.747821 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:27:07.747985 kubelet[2720]: E0813 01:27:07.747832 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:27:07.747985 kubelet[2720]: E0813 01:27:07.747838 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:27:07.747985 kubelet[2720]: E0813 01:27:07.747848 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:27:07.747985 kubelet[2720]: E0813 01:27:07.747856 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:27:07.747985 kubelet[2720]: E0813 01:27:07.747863 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:27:07.748224 kubelet[2720]: E0813 01:27:07.748059 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:27:07.748224 kubelet[2720]: E0813 01:27:07.748076 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:27:07.748224 kubelet[2720]: I0813 01:27:07.748085 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:27:07.804799 kubelet[2720]: I0813 01:27:07.804754 2720 kubelet.go:2405] "Pod admission denied" podUID="834db882-656e-43a1-a624-76c55cbca5d9" pod="tigera-operator/tigera-operator-747864d56d-vsn4f" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:07.906913 kubelet[2720]: I0813 01:27:07.905149 2720 kubelet.go:2405] "Pod admission denied" podUID="05703095-7aa6-4b6a-ac8a-82b8b0cf3064" pod="tigera-operator/tigera-operator-747864d56d-dl76w" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:08.073820 kubelet[2720]: E0813 01:27:08.073780 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:27:08.075031 containerd[1567]: time="2025-08-13T01:27:08.074457703Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:27:08.116197 kubelet[2720]: I0813 01:27:08.116166 2720 kubelet.go:2405] "Pod admission denied" podUID="603650ee-06d3-4585-9bb0-0fd03b61e1bf" pod="tigera-operator/tigera-operator-747864d56d-ns8ct" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:08.145368 containerd[1567]: time="2025-08-13T01:27:08.145306346Z" level=error msg="Failed to destroy network for sandbox \"1675426b4fa7358efb893145016fe568569bf6f205f7243caa3bd737699e8332\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:08.147536 systemd[1]: run-netns-cni\x2df9e569c7\x2d64a3\x2d7cd5\x2de639\x2db98d521ebce9.mount: Deactivated successfully. Aug 13 01:27:08.148397 containerd[1567]: time="2025-08-13T01:27:08.148236194Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"1675426b4fa7358efb893145016fe568569bf6f205f7243caa3bd737699e8332\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:08.149552 kubelet[2720]: E0813 01:27:08.149511 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1675426b4fa7358efb893145016fe568569bf6f205f7243caa3bd737699e8332\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:08.149671 kubelet[2720]: E0813 01:27:08.149564 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1675426b4fa7358efb893145016fe568569bf6f205f7243caa3bd737699e8332\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:27:08.149671 kubelet[2720]: E0813 01:27:08.149586 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1675426b4fa7358efb893145016fe568569bf6f205f7243caa3bd737699e8332\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:27:08.149671 kubelet[2720]: E0813 01:27:08.149632 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"1675426b4fa7358efb893145016fe568569bf6f205f7243caa3bd737699e8332\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:27:08.208789 kubelet[2720]: I0813 01:27:08.208209 2720 kubelet.go:2405] "Pod admission denied" podUID="0790baff-dfa3-44f9-bd3b-3335e02762f1" pod="tigera-operator/tigera-operator-747864d56d-qgtm6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:08.307254 kubelet[2720]: I0813 01:27:08.307202 2720 kubelet.go:2405] "Pod admission denied" podUID="4542fdc5-9438-4db0-8405-57e4a14603a3" pod="tigera-operator/tigera-operator-747864d56d-8s5ff" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:08.508707 kubelet[2720]: I0813 01:27:08.508657 2720 kubelet.go:2405] "Pod admission denied" podUID="427806a6-904c-499c-90e5-9d8175fd9d76" pod="tigera-operator/tigera-operator-747864d56d-67k47" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:08.606034 kubelet[2720]: I0813 01:27:08.605989 2720 kubelet.go:2405] "Pod admission denied" podUID="655a8630-af45-4bc6-9d66-0f71fa1ffeca" pod="tigera-operator/tigera-operator-747864d56d-ksdd7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:08.703980 kubelet[2720]: I0813 01:27:08.703931 2720 kubelet.go:2405] "Pod admission denied" podUID="b9647fa9-a1fb-446b-87a8-a48505f3e9f7" pod="tigera-operator/tigera-operator-747864d56d-zlwcn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:08.818971 kubelet[2720]: I0813 01:27:08.818361 2720 kubelet.go:2405] "Pod admission denied" podUID="7eec3f9e-3e9e-44ba-9560-eef1a21caf97" pod="tigera-operator/tigera-operator-747864d56d-2k9zs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:08.906391 kubelet[2720]: I0813 01:27:08.906334 2720 kubelet.go:2405] "Pod admission denied" podUID="2c3ba302-3d2b-46f3-8ed2-a09f2c5faf09" pod="tigera-operator/tigera-operator-747864d56d-m5rzc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:09.005538 kubelet[2720]: I0813 01:27:09.005487 2720 kubelet.go:2405] "Pod admission denied" podUID="dfc881ce-3567-47bd-9ee1-4940a8484042" pod="tigera-operator/tigera-operator-747864d56d-t4sgj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:09.105640 kubelet[2720]: I0813 01:27:09.105327 2720 kubelet.go:2405] "Pod admission denied" podUID="c3752564-a7b9-4a18-8a38-d149a3429e02" pod="tigera-operator/tigera-operator-747864d56d-lgjhd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:09.204216 kubelet[2720]: I0813 01:27:09.204165 2720 kubelet.go:2405] "Pod admission denied" podUID="ef05253e-06f9-4d3c-b01f-2d10cd96990f" pod="tigera-operator/tigera-operator-747864d56d-bb6qm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:09.260677 kubelet[2720]: I0813 01:27:09.260603 2720 kubelet.go:2405] "Pod admission denied" podUID="6fa6b9cb-4149-4655-bdea-f7542a304dab" pod="tigera-operator/tigera-operator-747864d56d-cfnb2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:09.357310 kubelet[2720]: I0813 01:27:09.356774 2720 kubelet.go:2405] "Pod admission denied" podUID="4ca0b52f-bc12-4c90-8b6d-03ae10106b4e" pod="tigera-operator/tigera-operator-747864d56d-mscrf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:09.458766 kubelet[2720]: I0813 01:27:09.458691 2720 kubelet.go:2405] "Pod admission denied" podUID="488f6418-816d-4f72-8f10-8edfd0ec8e7d" pod="tigera-operator/tigera-operator-747864d56d-vrgd2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:09.552291 kubelet[2720]: I0813 01:27:09.552239 2720 kubelet.go:2405] "Pod admission denied" podUID="ff75badc-bdd6-47cb-a094-f1ed069424b0" pod="tigera-operator/tigera-operator-747864d56d-phpzp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:09.753578 kubelet[2720]: I0813 01:27:09.753532 2720 kubelet.go:2405] "Pod admission denied" podUID="b4ab5c56-fe47-43ea-9f16-ac204494b0a8" pod="tigera-operator/tigera-operator-747864d56d-dl6k5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:09.863087 kubelet[2720]: I0813 01:27:09.863012 2720 kubelet.go:2405] "Pod admission denied" podUID="f38b0048-e019-4584-8e1a-3044f5424598" pod="tigera-operator/tigera-operator-747864d56d-bj8gk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:09.956928 kubelet[2720]: I0813 01:27:09.956866 2720 kubelet.go:2405] "Pod admission denied" podUID="b000c6d3-cf11-4cf7-842c-93e4820a785b" pod="tigera-operator/tigera-operator-747864d56d-2p46k" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:10.055837 kubelet[2720]: I0813 01:27:10.055364 2720 kubelet.go:2405] "Pod admission denied" podUID="03fd0210-13b6-48ec-ab03-3318556c4c5c" pod="tigera-operator/tigera-operator-747864d56d-ctxsf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:10.157592 kubelet[2720]: I0813 01:27:10.157539 2720 kubelet.go:2405] "Pod admission denied" podUID="35e2f780-2ea5-4832-9f8f-f8ce0e6e75f2" pod="tigera-operator/tigera-operator-747864d56d-94wcc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:10.259181 kubelet[2720]: I0813 01:27:10.259127 2720 kubelet.go:2405] "Pod admission denied" podUID="1ecdca42-cf26-4b35-8a5a-a588bb17e928" pod="tigera-operator/tigera-operator-747864d56d-8qv98" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:10.362811 kubelet[2720]: I0813 01:27:10.362258 2720 kubelet.go:2405] "Pod admission denied" podUID="b30f8440-f41e-41cd-911b-de63217b2750" pod="tigera-operator/tigera-operator-747864d56d-899rd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:10.453821 kubelet[2720]: I0813 01:27:10.453763 2720 kubelet.go:2405] "Pod admission denied" podUID="0f043034-6e2a-4046-95de-1603ccca1836" pod="tigera-operator/tigera-operator-747864d56d-94qwm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:10.556079 kubelet[2720]: I0813 01:27:10.556024 2720 kubelet.go:2405] "Pod admission denied" podUID="29c6907d-96a2-4653-98a8-c93df20bf796" pod="tigera-operator/tigera-operator-747864d56d-tdh9n" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:10.661757 kubelet[2720]: I0813 01:27:10.661014 2720 kubelet.go:2405] "Pod admission denied" podUID="49da2efd-44b0-4928-82df-09f44157a66e" pod="tigera-operator/tigera-operator-747864d56d-nkjgk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:10.754587 kubelet[2720]: I0813 01:27:10.754541 2720 kubelet.go:2405] "Pod admission denied" podUID="d17a90de-bc35-4c84-a352-5fd8d68415ac" pod="tigera-operator/tigera-operator-747864d56d-8lxvn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:10.864092 kubelet[2720]: I0813 01:27:10.863959 2720 kubelet.go:2405] "Pod admission denied" podUID="946f0e5d-d756-4a9c-9140-2588dd6a85e4" pod="tigera-operator/tigera-operator-747864d56d-n4hh4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:10.955348 kubelet[2720]: I0813 01:27:10.954197 2720 kubelet.go:2405] "Pod admission denied" podUID="ec788228-f2eb-4a40-8e1e-7d13907dd300" pod="tigera-operator/tigera-operator-747864d56d-7tq9n" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:11.053745 kubelet[2720]: I0813 01:27:11.053685 2720 kubelet.go:2405] "Pod admission denied" podUID="37a3b055-5502-4b5b-b30e-10a21059343b" pod="tigera-operator/tigera-operator-747864d56d-5rtlh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:11.097413 containerd[1567]: time="2025-08-13T01:27:11.095034215Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/node:v3.30.2\"" Aug 13 01:27:11.156628 kubelet[2720]: I0813 01:27:11.156554 2720 kubelet.go:2405] "Pod admission denied" podUID="2dec4511-03a4-4e9f-9743-1835afd708d1" pod="tigera-operator/tigera-operator-747864d56d-ws79q" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:11.258403 kubelet[2720]: I0813 01:27:11.258344 2720 kubelet.go:2405] "Pod admission denied" podUID="c00da428-5df6-4874-ad12-03403b52d42f" pod="tigera-operator/tigera-operator-747864d56d-cmt64" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:11.365329 kubelet[2720]: I0813 01:27:11.364660 2720 kubelet.go:2405] "Pod admission denied" podUID="f4545ccf-b1ee-4530-8f1c-95c1a8474b8b" pod="tigera-operator/tigera-operator-747864d56d-xtj7t" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:11.560158 kubelet[2720]: I0813 01:27:11.560037 2720 kubelet.go:2405] "Pod admission denied" podUID="46d275c5-14a5-479b-a321-26e90f8c5fe7" pod="tigera-operator/tigera-operator-747864d56d-p4krq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:11.670266 kubelet[2720]: I0813 01:27:11.669109 2720 kubelet.go:2405] "Pod admission denied" podUID="d0c9437d-f95d-49ac-96bc-1cf70b0d9857" pod="tigera-operator/tigera-operator-747864d56d-pl2rr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:11.768396 kubelet[2720]: I0813 01:27:11.768359 2720 kubelet.go:2405] "Pod admission denied" podUID="9634d6d5-835a-4b70-8dcd-03eb2f5aeced" pod="tigera-operator/tigera-operator-747864d56d-5p292" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:11.864572 kubelet[2720]: I0813 01:27:11.864446 2720 kubelet.go:2405] "Pod admission denied" podUID="b2e7ce4c-f3f1-4ba3-bddc-dddb2456bf10" pod="tigera-operator/tigera-operator-747864d56d-cmq6h" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:11.997739 kubelet[2720]: I0813 01:27:11.995151 2720 kubelet.go:2405] "Pod admission denied" podUID="762d8f0c-ce9f-46ea-be09-082995bb17ba" pod="tigera-operator/tigera-operator-747864d56d-zgdh9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:12.059525 kubelet[2720]: I0813 01:27:12.059485 2720 kubelet.go:2405] "Pod admission denied" podUID="75c6987b-b8c7-4cd6-9b52-a1ee80898d7c" pod="tigera-operator/tigera-operator-747864d56d-b2ptg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:12.185148 kubelet[2720]: I0813 01:27:12.180480 2720 kubelet.go:2405] "Pod admission denied" podUID="702646d8-4d9f-4761-b979-e3aa3c9c7319" pod="tigera-operator/tigera-operator-747864d56d-6d2bh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:12.365362 kubelet[2720]: I0813 01:27:12.365311 2720 kubelet.go:2405] "Pod admission denied" podUID="ae3fec04-2876-4734-87c8-f6cad1a71dda" pod="tigera-operator/tigera-operator-747864d56d-xsqfp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:12.472146 kubelet[2720]: I0813 01:27:12.472020 2720 kubelet.go:2405] "Pod admission denied" podUID="7575f62a-90c9-48cb-b632-3e85a8313269" pod="tigera-operator/tigera-operator-747864d56d-ssdkc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:12.612587 kubelet[2720]: I0813 01:27:12.612526 2720 kubelet.go:2405] "Pod admission denied" podUID="50229c38-d552-4708-8de1-48acc68846e0" pod="tigera-operator/tigera-operator-747864d56d-2kpq4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:12.713771 kubelet[2720]: I0813 01:27:12.713739 2720 kubelet.go:2405] "Pod admission denied" podUID="d2788a83-b9f8-4cf4-b5fd-4791a8f24904" pod="tigera-operator/tigera-operator-747864d56d-tbhkn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:12.815386 kubelet[2720]: I0813 01:27:12.815094 2720 kubelet.go:2405] "Pod admission denied" podUID="79f823b1-a7fe-4ce0-bbee-0a27441c03f2" pod="tigera-operator/tigera-operator-747864d56d-pc6dw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:12.918652 kubelet[2720]: I0813 01:27:12.918603 2720 kubelet.go:2405] "Pod admission denied" podUID="2000d1ed-87e3-4d09-8d9a-7e0dfd0e125b" pod="tigera-operator/tigera-operator-747864d56d-qjsp2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:13.024206 kubelet[2720]: I0813 01:27:13.024151 2720 kubelet.go:2405] "Pod admission denied" podUID="79f5bb70-3777-43f8-9c80-04e774911a4e" pod="tigera-operator/tigera-operator-747864d56d-qwt7h" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:13.074132 containerd[1567]: time="2025-08-13T01:27:13.073933475Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:27:13.127363 kubelet[2720]: I0813 01:27:13.127326 2720 kubelet.go:2405] "Pod admission denied" podUID="23d43aa5-7bc9-477f-af8f-12dd01a38b5f" pod="tigera-operator/tigera-operator-747864d56d-lrnjk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:13.239761 containerd[1567]: time="2025-08-13T01:27:13.237850661Z" level=error msg="Failed to destroy network for sandbox \"7318ae1d1fac65b953fb7c298c6802db448ace6b87242e485a6833189edabfff\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:13.243834 systemd[1]: run-netns-cni\x2df5ed4f2c\x2dff07\x2d1fa8\x2d38be\x2dbd73e38adf79.mount: Deactivated successfully. Aug 13 01:27:13.245278 containerd[1567]: time="2025-08-13T01:27:13.245193833Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"7318ae1d1fac65b953fb7c298c6802db448ace6b87242e485a6833189edabfff\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:13.246062 kubelet[2720]: E0813 01:27:13.245470 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7318ae1d1fac65b953fb7c298c6802db448ace6b87242e485a6833189edabfff\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:13.246062 kubelet[2720]: E0813 01:27:13.245526 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7318ae1d1fac65b953fb7c298c6802db448ace6b87242e485a6833189edabfff\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:27:13.246062 kubelet[2720]: E0813 01:27:13.245548 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7318ae1d1fac65b953fb7c298c6802db448ace6b87242e485a6833189edabfff\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:27:13.246062 kubelet[2720]: E0813 01:27:13.245594 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"7318ae1d1fac65b953fb7c298c6802db448ace6b87242e485a6833189edabfff\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:27:13.271809 kubelet[2720]: I0813 01:27:13.271699 2720 kubelet.go:2405] "Pod admission denied" podUID="9d8ffcf5-086b-4f68-888c-2a289f46dbc6" pod="tigera-operator/tigera-operator-747864d56d-swtwc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:13.402518 kubelet[2720]: I0813 01:27:13.401845 2720 kubelet.go:2405] "Pod admission denied" podUID="5dff3e15-e0a1-438b-8518-67d6e577e8f6" pod="tigera-operator/tigera-operator-747864d56d-cpsjn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:13.542526 kubelet[2720]: I0813 01:27:13.542225 2720 kubelet.go:2405] "Pod admission denied" podUID="d27b61c7-ce8e-4211-a557-e84f09843a4a" pod="tigera-operator/tigera-operator-747864d56d-dnclj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:13.629738 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount849196455.mount: Deactivated successfully. Aug 13 01:27:13.630290 containerd[1567]: time="2025-08-13T01:27:13.629363791Z" level=error msg="PullImage \"ghcr.io/flatcar/calico/node:v3.30.2\" failed" error="rpc error: code = Unknown desc = failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount849196455: write /var/lib/containerd/tmpmounts/containerd-mount849196455/usr/bin/calico-node: no space left on device" Aug 13 01:27:13.630851 containerd[1567]: time="2025-08-13T01:27:13.630017846Z" level=info msg="stop pulling image ghcr.io/flatcar/calico/node:v3.30.2: active requests=0, bytes read=158500163" Aug 13 01:27:13.630883 kubelet[2720]: E0813 01:27:13.630531 2720 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount849196455: write /var/lib/containerd/tmpmounts/containerd-mount849196455/usr/bin/calico-node: no space left on device" image="ghcr.io/flatcar/calico/node:v3.30.2" Aug 13 01:27:13.630883 kubelet[2720]: E0813 01:27:13.630578 2720 kuberuntime_image.go:42] "Failed to pull image" err="failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount849196455: write /var/lib/containerd/tmpmounts/containerd-mount849196455/usr/bin/calico-node: no space left on device" image="ghcr.io/flatcar/calico/node:v3.30.2" Aug 13 01:27:13.632113 kubelet[2720]: E0813 01:27:13.632032 2720 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:calico-node,Image:ghcr.io/flatcar/calico/node:v3.30.2,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{},Env:[]EnvVar{EnvVar{Name:DATASTORE_TYPE,Value:kubernetes,ValueFrom:nil,},EnvVar{Name:WAIT_FOR_DATASTORE,Value:true,ValueFrom:nil,},EnvVar{Name:CLUSTER_TYPE,Value:k8s,operator,bgp,ValueFrom:nil,},EnvVar{Name:CALICO_DISABLE_FILE_LOGGING,Value:false,ValueFrom:nil,},EnvVar{Name:FELIX_DEFAULTENDPOINTTOHOSTACTION,Value:ACCEPT,ValueFrom:nil,},EnvVar{Name:FELIX_HEALTHENABLED,Value:true,ValueFrom:nil,},EnvVar{Name:FELIX_HEALTHPORT,Value:9099,ValueFrom:nil,},EnvVar{Name:NODENAME,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:spec.nodeName,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:FELIX_TYPHAK8SNAMESPACE,Value:calico-system,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHAK8SSERVICENAME,Value:calico-typha,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHACAFILE,Value:/etc/pki/tls/certs/tigera-ca-bundle.crt,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHACERTFILE,Value:/node-certs/tls.crt,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHAKEYFILE,Value:/node-certs/tls.key,ValueFrom:nil,},EnvVar{Name:NO_DEFAULT_POOLS,Value:true,ValueFrom:nil,},EnvVar{Name:FELIX_FLOWLOGSGOLDMANESERVER,Value:goldmane.calico-system.svc:7443,ValueFrom:nil,},EnvVar{Name:FELIX_FLOWLOGSFLUSHINTERVAL,Value:15,ValueFrom:nil,},EnvVar{Name:FELIX_TYPHACN,Value:typha-server,ValueFrom:nil,},EnvVar{Name:CALICO_MANAGE_CNI,Value:true,ValueFrom:nil,},EnvVar{Name:CALICO_NETWORKING_BACKEND,Value:bird,ValueFrom:nil,},EnvVar{Name:IP,Value:autodetect,ValueFrom:nil,},EnvVar{Name:IP_AUTODETECTION_METHOD,Value:first-found,ValueFrom:nil,},EnvVar{Name:IP6,Value:none,ValueFrom:nil,},EnvVar{Name:FELIX_IPV6SUPPORT,Value:false,ValueFrom:nil,},EnvVar{Name:KUBERNETES_SERVICE_HOST,Value:10.96.0.1,ValueFrom:nil,},EnvVar{Name:KUBERNETES_SERVICE_PORT,Value:443,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:cni-log-dir,ReadOnly:false,MountPath:/var/log/calico/cni,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:cni-net-dir,ReadOnly:false,MountPath:/host/etc/cni/net.d,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:lib-modules,ReadOnly:true,MountPath:/lib/modules,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:node-certs,ReadOnly:true,MountPath:/node-certs,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:policysync,ReadOnly:false,MountPath:/var/run/nodeagent,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:tigera-ca-bundle,ReadOnly:true,MountPath:/etc/pki/tls/certs,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:tigera-ca-bundle,ReadOnly:true,MountPath:/etc/pki/tls/cert.pem,SubPath:ca-bundle.crt,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:var-lib-calico,ReadOnly:false,MountPath:/var/lib/calico,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:var-run-calico,ReadOnly:false,MountPath:/var/run/calico,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:xtables-lock,ReadOnly:false,MountPath:/run/xtables.lock,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-jl8ww,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/liveness,Port:{0 9099 },Host:localhost,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:10,PeriodSeconds:60,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:&ExecAction{Command:[/bin/calico-node -bird-ready -felix-ready],},HTTPGet:nil,TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:10,PeriodSeconds:30,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:&Lifecycle{PostStart:nil,PreStop:&LifecycleHandler{Exec:&ExecAction{Command:[/bin/calico-node -shutdown],},HTTPGet:nil,TCPSocket:nil,Sleep:nil,},StopSignal:nil,},TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:*true,SELinuxOptions:nil,RunAsUser:*0,RunAsNonRoot:*false,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:*true,RunAsGroup:*0,ProcMount:nil,WindowsOptions:nil,SeccompProfile:&SeccompProfile{Type:RuntimeDefault,LocalhostProfile:nil,},AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod calico-node-2p8hj_calico-system(4f334296-e8c0-427e-a090-d8fbd3582f26): ErrImagePull: failed to pull and unpack image \"ghcr.io/flatcar/calico/node:v3.30.2\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount849196455: write /var/lib/containerd/tmpmounts/containerd-mount849196455/usr/bin/calico-node: no space left on device" logger="UnhandledError" Aug 13 01:27:13.633412 kubelet[2720]: E0813 01:27:13.633370 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"calico-node\" with ErrImagePull: \"failed to pull and unpack image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount849196455: write /var/lib/containerd/tmpmounts/containerd-mount849196455/usr/bin/calico-node: no space left on device\"" pod="calico-system/calico-node-2p8hj" podUID="4f334296-e8c0-427e-a090-d8fbd3582f26" Aug 13 01:27:13.661598 kubelet[2720]: I0813 01:27:13.661487 2720 kubelet.go:2405] "Pod admission denied" podUID="833e58ef-4442-4614-ba8d-07ec0ec563a5" pod="tigera-operator/tigera-operator-747864d56d-9pktq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:13.703584 kubelet[2720]: I0813 01:27:13.703543 2720 kubelet.go:2405] "Pod admission denied" podUID="5cf9d21f-dce9-46a6-af32-f2466715a3a6" pod="tigera-operator/tigera-operator-747864d56d-ps65h" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:13.808354 kubelet[2720]: I0813 01:27:13.808072 2720 kubelet.go:2405] "Pod admission denied" podUID="c95c6083-21af-4524-9b62-32dec1c5b01d" pod="tigera-operator/tigera-operator-747864d56d-qk7ww" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:13.907122 kubelet[2720]: I0813 01:27:13.907086 2720 kubelet.go:2405] "Pod admission denied" podUID="221315c9-33d3-4b6f-8b39-426eaf04716c" pod="tigera-operator/tigera-operator-747864d56d-h4m8v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:14.021921 kubelet[2720]: I0813 01:27:14.021875 2720 kubelet.go:2405] "Pod admission denied" podUID="23fa9c0e-320d-4dc2-9a2c-a007d06a3eb7" pod="tigera-operator/tigera-operator-747864d56d-bh782" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:14.208582 kubelet[2720]: I0813 01:27:14.208530 2720 kubelet.go:2405] "Pod admission denied" podUID="d30be2ab-bb1b-479c-977e-ea4c6d01083e" pod="tigera-operator/tigera-operator-747864d56d-tp7wv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:14.309276 kubelet[2720]: I0813 01:27:14.308519 2720 kubelet.go:2405] "Pod admission denied" podUID="0c75ff99-7a03-461e-8fb2-3f19b6350e38" pod="tigera-operator/tigera-operator-747864d56d-v6n2k" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:14.404524 kubelet[2720]: I0813 01:27:14.404473 2720 kubelet.go:2405] "Pod admission denied" podUID="312be8a9-8764-4767-a7b0-01287606f79b" pod="tigera-operator/tigera-operator-747864d56d-7mfqf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:14.508831 kubelet[2720]: I0813 01:27:14.508772 2720 kubelet.go:2405] "Pod admission denied" podUID="72344314-35ce-4ca0-ba62-1a324290eb96" pod="tigera-operator/tigera-operator-747864d56d-l5cz5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:14.607953 kubelet[2720]: I0813 01:27:14.607846 2720 kubelet.go:2405] "Pod admission denied" podUID="8a2d874b-c01f-4c30-95e6-1be211a81592" pod="tigera-operator/tigera-operator-747864d56d-vkpr9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:14.708795 kubelet[2720]: I0813 01:27:14.708744 2720 kubelet.go:2405] "Pod admission denied" podUID="d84085ab-dc74-4c13-a226-a204270068b9" pod="tigera-operator/tigera-operator-747864d56d-fnmvv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:14.767467 kubelet[2720]: I0813 01:27:14.767415 2720 kubelet.go:2405] "Pod admission denied" podUID="f24d8e77-63cd-4537-bd43-e2d374c9bb08" pod="tigera-operator/tigera-operator-747864d56d-j8hkt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:14.859789 kubelet[2720]: I0813 01:27:14.859677 2720 kubelet.go:2405] "Pod admission denied" podUID="e2fa10d9-75a1-43c7-b46e-3593253fcc1d" pod="tigera-operator/tigera-operator-747864d56d-6m5pz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:15.077740 kubelet[2720]: I0813 01:27:15.076243 2720 kubelet.go:2405] "Pod admission denied" podUID="60b2f2d7-44af-4ca2-a946-afd67b92b773" pod="tigera-operator/tigera-operator-747864d56d-6kxql" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:15.156771 kubelet[2720]: I0813 01:27:15.155413 2720 kubelet.go:2405] "Pod admission denied" podUID="7b1364aa-88da-4048-9ff4-89e2738d3968" pod="tigera-operator/tigera-operator-747864d56d-mkfhn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:15.259705 kubelet[2720]: I0813 01:27:15.259650 2720 kubelet.go:2405] "Pod admission denied" podUID="42904102-8235-40c2-9a2a-682613d06401" pod="tigera-operator/tigera-operator-747864d56d-44bg4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:15.358946 kubelet[2720]: I0813 01:27:15.358892 2720 kubelet.go:2405] "Pod admission denied" podUID="cfe9fbc5-f273-4426-b6d8-d76cfa160c34" pod="tigera-operator/tigera-operator-747864d56d-knqv2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:15.458631 kubelet[2720]: I0813 01:27:15.457820 2720 kubelet.go:2405] "Pod admission denied" podUID="faf11596-6835-401a-9a34-393c33185df0" pod="tigera-operator/tigera-operator-747864d56d-8dntg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:15.557142 kubelet[2720]: I0813 01:27:15.557088 2720 kubelet.go:2405] "Pod admission denied" podUID="960f9d57-7bc0-4edd-9d92-081c3f01792f" pod="tigera-operator/tigera-operator-747864d56d-qhc6v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:15.659198 kubelet[2720]: I0813 01:27:15.659146 2720 kubelet.go:2405] "Pod admission denied" podUID="4e11d71e-341c-4bd0-b306-721868f3f776" pod="tigera-operator/tigera-operator-747864d56d-xjln6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:15.755424 kubelet[2720]: I0813 01:27:15.755365 2720 kubelet.go:2405] "Pod admission denied" podUID="cccec95b-8c8a-495a-bcce-ffc061c0c39b" pod="tigera-operator/tigera-operator-747864d56d-7brdp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:15.858649 kubelet[2720]: I0813 01:27:15.858595 2720 kubelet.go:2405] "Pod admission denied" podUID="02ca140c-2a86-4d7e-9007-b38e31408a53" pod="tigera-operator/tigera-operator-747864d56d-qd4mn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:15.956186 kubelet[2720]: I0813 01:27:15.956130 2720 kubelet.go:2405] "Pod admission denied" podUID="653d2653-ff0f-40eb-99d4-daafbedcb33c" pod="tigera-operator/tigera-operator-747864d56d-b8tb7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:16.059798 kubelet[2720]: I0813 01:27:16.058962 2720 kubelet.go:2405] "Pod admission denied" podUID="04cfd5a5-421e-4347-b619-f3515548d43f" pod="tigera-operator/tigera-operator-747864d56d-pl4nb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:16.074741 kubelet[2720]: E0813 01:27:16.074592 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:27:16.075685 containerd[1567]: time="2025-08-13T01:27:16.075570234Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:27:16.160426 containerd[1567]: time="2025-08-13T01:27:16.160358740Z" level=error msg="Failed to destroy network for sandbox \"14ee1c06adf3c9e0490d3eeb2ad06bb52937ccbbeebdc69eaa2a0e78842d095c\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:16.163597 containerd[1567]: time="2025-08-13T01:27:16.163486100Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"14ee1c06adf3c9e0490d3eeb2ad06bb52937ccbbeebdc69eaa2a0e78842d095c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:16.163916 kubelet[2720]: E0813 01:27:16.163857 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"14ee1c06adf3c9e0490d3eeb2ad06bb52937ccbbeebdc69eaa2a0e78842d095c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:16.163983 kubelet[2720]: E0813 01:27:16.163943 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"14ee1c06adf3c9e0490d3eeb2ad06bb52937ccbbeebdc69eaa2a0e78842d095c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:27:16.163983 kubelet[2720]: E0813 01:27:16.163973 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"14ee1c06adf3c9e0490d3eeb2ad06bb52937ccbbeebdc69eaa2a0e78842d095c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:27:16.164576 kubelet[2720]: E0813 01:27:16.164041 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"14ee1c06adf3c9e0490d3eeb2ad06bb52937ccbbeebdc69eaa2a0e78842d095c\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:27:16.164501 systemd[1]: run-netns-cni\x2dd85daf61\x2d3e4b\x2dff95\x2db517\x2deb4ef92c0705.mount: Deactivated successfully. Aug 13 01:27:16.168859 kubelet[2720]: I0813 01:27:16.168825 2720 kubelet.go:2405] "Pod admission denied" podUID="fad9c351-8686-4b48-9740-c7d245103ae9" pod="tigera-operator/tigera-operator-747864d56d-xlfj9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:16.255900 kubelet[2720]: I0813 01:27:16.255855 2720 kubelet.go:2405] "Pod admission denied" podUID="8d24c189-d615-4c91-93cd-29a0b85d9f15" pod="tigera-operator/tigera-operator-747864d56d-n8zz6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:16.374110 kubelet[2720]: I0813 01:27:16.372903 2720 kubelet.go:2405] "Pod admission denied" podUID="b8fb1b6d-f1a9-4efd-80c0-109346710264" pod="tigera-operator/tigera-operator-747864d56d-flpvv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:16.455759 kubelet[2720]: I0813 01:27:16.455691 2720 kubelet.go:2405] "Pod admission denied" podUID="5fdee91b-97e2-4be0-b860-9d026cc9824a" pod="tigera-operator/tigera-operator-747864d56d-kj24p" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:16.556862 kubelet[2720]: I0813 01:27:16.556803 2720 kubelet.go:2405] "Pod admission denied" podUID="c27b1d0f-cb87-466a-97e0-99191537394a" pod="tigera-operator/tigera-operator-747864d56d-d4cm4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:16.620031 kubelet[2720]: I0813 01:27:16.619973 2720 kubelet.go:2405] "Pod admission denied" podUID="fe881c80-6b5d-4748-9a62-c4fd95f9b93f" pod="tigera-operator/tigera-operator-747864d56d-mvqk9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:16.704948 kubelet[2720]: I0813 01:27:16.704699 2720 kubelet.go:2405] "Pod admission denied" podUID="02139076-139f-45d2-8c48-8cdf2c66575b" pod="tigera-operator/tigera-operator-747864d56d-87bdp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:16.823158 kubelet[2720]: I0813 01:27:16.823117 2720 kubelet.go:2405] "Pod admission denied" podUID="d7f42b11-cab5-4c4b-9e3b-67f4be49b1f0" pod="tigera-operator/tigera-operator-747864d56d-kdzx8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:16.910434 kubelet[2720]: I0813 01:27:16.910385 2720 kubelet.go:2405] "Pod admission denied" podUID="891981a1-c11f-4e61-a299-29763ca620fb" pod="tigera-operator/tigera-operator-747864d56d-55f74" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:17.010430 kubelet[2720]: I0813 01:27:17.010375 2720 kubelet.go:2405] "Pod admission denied" podUID="761b6de2-9260-41d8-a9b5-7bb987afce9b" pod="tigera-operator/tigera-operator-747864d56d-xsr65" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:17.107990 kubelet[2720]: I0813 01:27:17.107941 2720 kubelet.go:2405] "Pod admission denied" podUID="a5f12c32-dc41-41f8-a824-6019d978990b" pod="tigera-operator/tigera-operator-747864d56d-zv7gh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:17.208190 kubelet[2720]: I0813 01:27:17.208142 2720 kubelet.go:2405] "Pod admission denied" podUID="d83b119e-da73-48aa-a857-a18fc7130442" pod="tigera-operator/tigera-operator-747864d56d-s4g4w" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:17.324413 kubelet[2720]: I0813 01:27:17.324284 2720 kubelet.go:2405] "Pod admission denied" podUID="28d73c65-35d0-4689-931f-5ada15f019fd" pod="tigera-operator/tigera-operator-747864d56d-qkcfx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:17.406932 kubelet[2720]: I0813 01:27:17.406883 2720 kubelet.go:2405] "Pod admission denied" podUID="850741d7-439c-448e-852d-d4f725012e46" pod="tigera-operator/tigera-operator-747864d56d-9444v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:17.461925 kubelet[2720]: I0813 01:27:17.461890 2720 kubelet.go:2405] "Pod admission denied" podUID="2085080e-5ae8-4381-91cc-849b2ff26406" pod="tigera-operator/tigera-operator-747864d56d-4kxpl" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:17.557679 kubelet[2720]: I0813 01:27:17.557614 2720 kubelet.go:2405] "Pod admission denied" podUID="1e940571-fd66-4bd3-9721-7f59e1810a6d" pod="tigera-operator/tigera-operator-747864d56d-6gskn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:17.662819 kubelet[2720]: I0813 01:27:17.662459 2720 kubelet.go:2405] "Pod admission denied" podUID="2069734e-28d6-4395-9846-168f7464b467" pod="tigera-operator/tigera-operator-747864d56d-fw78l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:17.763582 kubelet[2720]: I0813 01:27:17.762913 2720 kubelet.go:2405] "Pod admission denied" podUID="e240c2fd-af9e-4ec7-8896-2962d1a9429a" pod="tigera-operator/tigera-operator-747864d56d-9467l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:17.767573 kubelet[2720]: I0813 01:27:17.767532 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:27:17.767696 kubelet[2720]: I0813 01:27:17.767686 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:27:17.770462 kubelet[2720]: I0813 01:27:17.770439 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:27:17.786056 kubelet[2720]: I0813 01:27:17.785866 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:27:17.786056 kubelet[2720]: I0813 01:27:17.785932 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-node-2p8hj","calico-system/csi-node-driver-8bzzs","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:27:17.786056 kubelet[2720]: E0813 01:27:17.785957 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:27:17.786056 kubelet[2720]: E0813 01:27:17.785966 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:27:17.786056 kubelet[2720]: E0813 01:27:17.785973 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:27:17.786056 kubelet[2720]: E0813 01:27:17.785979 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:27:17.786056 kubelet[2720]: E0813 01:27:17.785985 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:27:17.786056 kubelet[2720]: E0813 01:27:17.785995 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:27:17.786056 kubelet[2720]: E0813 01:27:17.786002 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:27:17.786056 kubelet[2720]: E0813 01:27:17.786009 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:27:17.786056 kubelet[2720]: E0813 01:27:17.786016 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:27:17.786056 kubelet[2720]: E0813 01:27:17.786023 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:27:17.786056 kubelet[2720]: I0813 01:27:17.786031 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:27:17.864867 kubelet[2720]: I0813 01:27:17.864839 2720 kubelet.go:2405] "Pod admission denied" podUID="94d28894-cccf-4547-a96f-9c746a052934" pod="tigera-operator/tigera-operator-747864d56d-sd9c4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:17.908467 kubelet[2720]: I0813 01:27:17.908407 2720 kubelet.go:2405] "Pod admission denied" podUID="6250d835-f692-41b2-9f8e-de0d05e1f55a" pod="tigera-operator/tigera-operator-747864d56d-mz57m" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:18.007078 kubelet[2720]: I0813 01:27:18.007032 2720 kubelet.go:2405] "Pod admission denied" podUID="64c3728e-8a6e-4d7c-b2be-8e496de143d3" pod="tigera-operator/tigera-operator-747864d56d-b888g" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:18.073596 containerd[1567]: time="2025-08-13T01:27:18.073538248Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:27:18.129394 containerd[1567]: time="2025-08-13T01:27:18.129346777Z" level=error msg="Failed to destroy network for sandbox \"64b035a9c1e0ed76b8784db1de01ad42c52d52160a07a259c4b60810fb97d62a\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:18.131376 systemd[1]: run-netns-cni\x2d121d21e4\x2d4479\x2dbc53\x2d2d6d\x2d78b96a96e5e9.mount: Deactivated successfully. Aug 13 01:27:18.133348 containerd[1567]: time="2025-08-13T01:27:18.132884767Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"64b035a9c1e0ed76b8784db1de01ad42c52d52160a07a259c4b60810fb97d62a\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:18.133437 kubelet[2720]: E0813 01:27:18.133133 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"64b035a9c1e0ed76b8784db1de01ad42c52d52160a07a259c4b60810fb97d62a\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:18.133437 kubelet[2720]: E0813 01:27:18.133211 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"64b035a9c1e0ed76b8784db1de01ad42c52d52160a07a259c4b60810fb97d62a\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:27:18.133437 kubelet[2720]: E0813 01:27:18.133259 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"64b035a9c1e0ed76b8784db1de01ad42c52d52160a07a259c4b60810fb97d62a\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:27:18.133437 kubelet[2720]: E0813 01:27:18.133306 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"64b035a9c1e0ed76b8784db1de01ad42c52d52160a07a259c4b60810fb97d62a\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:27:18.207740 kubelet[2720]: I0813 01:27:18.207273 2720 kubelet.go:2405] "Pod admission denied" podUID="e7ebef9e-a1a3-4fc7-b3d9-7b5414eba258" pod="tigera-operator/tigera-operator-747864d56d-8wffc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:18.311800 kubelet[2720]: I0813 01:27:18.310131 2720 kubelet.go:2405] "Pod admission denied" podUID="9cf0b5e8-e223-44b9-8929-2ff00c27b15e" pod="tigera-operator/tigera-operator-747864d56d-h9hbz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:18.409239 kubelet[2720]: I0813 01:27:18.409170 2720 kubelet.go:2405] "Pod admission denied" podUID="fe7ca53a-2c4a-4833-9bbc-fd95e8fa7523" pod="tigera-operator/tigera-operator-747864d56d-z2f79" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:18.609055 kubelet[2720]: I0813 01:27:18.608249 2720 kubelet.go:2405] "Pod admission denied" podUID="eca105be-f849-4c43-aea5-6c06b19fcb6d" pod="tigera-operator/tigera-operator-747864d56d-dcvsj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:18.712048 kubelet[2720]: I0813 01:27:18.711998 2720 kubelet.go:2405] "Pod admission denied" podUID="ab805f48-464e-480c-9680-4f4f1a03bb38" pod="tigera-operator/tigera-operator-747864d56d-hhwjp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:18.806208 kubelet[2720]: I0813 01:27:18.806159 2720 kubelet.go:2405] "Pod admission denied" podUID="b38c9633-8cd2-4fa0-8d56-9c8872ddbe29" pod="tigera-operator/tigera-operator-747864d56d-vh7fv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:18.909824 kubelet[2720]: I0813 01:27:18.909682 2720 kubelet.go:2405] "Pod admission denied" podUID="9858370b-b5ab-4623-8fb3-ff43c2000a44" pod="tigera-operator/tigera-operator-747864d56d-mv85w" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:19.008917 kubelet[2720]: I0813 01:27:19.008861 2720 kubelet.go:2405] "Pod admission denied" podUID="92cd91e9-75ca-4ddd-b107-35af53710fb1" pod="tigera-operator/tigera-operator-747864d56d-88xrw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:19.211163 kubelet[2720]: I0813 01:27:19.210472 2720 kubelet.go:2405] "Pod admission denied" podUID="3703adb3-a0ec-46f0-bb37-c7fb44b9577f" pod="tigera-operator/tigera-operator-747864d56d-lp5kx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:19.307520 kubelet[2720]: I0813 01:27:19.307460 2720 kubelet.go:2405] "Pod admission denied" podUID="de6bb9ce-f998-49fa-88fa-401815096d4e" pod="tigera-operator/tigera-operator-747864d56d-2sc7l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:19.407422 kubelet[2720]: I0813 01:27:19.407268 2720 kubelet.go:2405] "Pod admission denied" podUID="0b70ae1b-d3fc-4a1b-b500-6ca67bf20fb0" pod="tigera-operator/tigera-operator-747864d56d-n627k" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:19.508307 kubelet[2720]: I0813 01:27:19.508262 2720 kubelet.go:2405] "Pod admission denied" podUID="822b1d49-819e-4a16-af11-aa841222f253" pod="tigera-operator/tigera-operator-747864d56d-k4bcj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:19.621767 kubelet[2720]: I0813 01:27:19.620970 2720 kubelet.go:2405] "Pod admission denied" podUID="65044bcd-5c61-4cfa-8721-9469bec39c9d" pod="tigera-operator/tigera-operator-747864d56d-dwnt2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:19.707702 kubelet[2720]: I0813 01:27:19.707646 2720 kubelet.go:2405] "Pod admission denied" podUID="8c0b8df8-69d3-4cbe-b506-c7af352766d5" pod="tigera-operator/tigera-operator-747864d56d-sqjr9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:19.808196 kubelet[2720]: I0813 01:27:19.806776 2720 kubelet.go:2405] "Pod admission denied" podUID="cfb0aee1-1aa7-48dd-b651-0e530757500f" pod="tigera-operator/tigera-operator-747864d56d-zn9vd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:19.906063 kubelet[2720]: I0813 01:27:19.906026 2720 kubelet.go:2405] "Pod admission denied" podUID="b992ed44-9e89-4746-a232-ae4135b7de02" pod="tigera-operator/tigera-operator-747864d56d-8wf64" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:20.005903 kubelet[2720]: I0813 01:27:20.005858 2720 kubelet.go:2405] "Pod admission denied" podUID="5494bd7e-56ad-4a6c-9c9d-b7327bb731dd" pod="tigera-operator/tigera-operator-747864d56d-zl694" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:20.208705 kubelet[2720]: I0813 01:27:20.207672 2720 kubelet.go:2405] "Pod admission denied" podUID="1bb9bfcf-cadf-4900-b02a-a02b7b24dec3" pod="tigera-operator/tigera-operator-747864d56d-zr85k" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:20.308553 kubelet[2720]: I0813 01:27:20.308501 2720 kubelet.go:2405] "Pod admission denied" podUID="58230c3c-c69b-4470-8d29-57b53c2db144" pod="tigera-operator/tigera-operator-747864d56d-pkdnh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:20.408279 kubelet[2720]: I0813 01:27:20.408235 2720 kubelet.go:2405] "Pod admission denied" podUID="a27b0502-9733-4864-bbee-adf2b0177598" pod="tigera-operator/tigera-operator-747864d56d-mcklm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:20.512208 kubelet[2720]: I0813 01:27:20.512153 2720 kubelet.go:2405] "Pod admission denied" podUID="c036d0ee-179b-432b-83a6-e2d689302f49" pod="tigera-operator/tigera-operator-747864d56d-zftjr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:20.607053 kubelet[2720]: I0813 01:27:20.607008 2720 kubelet.go:2405] "Pod admission denied" podUID="fa016e1a-c9a1-4372-b06d-6fde29425bcf" pod="tigera-operator/tigera-operator-747864d56d-qxb5w" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:20.710005 kubelet[2720]: I0813 01:27:20.709950 2720 kubelet.go:2405] "Pod admission denied" podUID="76c6668a-7588-450f-982b-cfbe75413047" pod="tigera-operator/tigera-operator-747864d56d-jw25t" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:20.773673 kubelet[2720]: I0813 01:27:20.773543 2720 kubelet.go:2405] "Pod admission denied" podUID="5c412b18-1714-42ee-b875-5868ad49dabd" pod="tigera-operator/tigera-operator-747864d56d-9l9hv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:20.857439 kubelet[2720]: I0813 01:27:20.857386 2720 kubelet.go:2405] "Pod admission denied" podUID="1d929e7b-9706-453f-bdf8-a72dc6a52f44" pod="tigera-operator/tigera-operator-747864d56d-7nffm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:20.972737 kubelet[2720]: I0813 01:27:20.972032 2720 kubelet.go:2405] "Pod admission denied" podUID="f6bd3f18-42bc-4fc8-81ec-0f8e8bcaacd1" pod="tigera-operator/tigera-operator-747864d56d-xzbsc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:21.059516 kubelet[2720]: I0813 01:27:21.058818 2720 kubelet.go:2405] "Pod admission denied" podUID="669a785e-90d5-433e-a94e-2fc2983c62ba" pod="tigera-operator/tigera-operator-747864d56d-nm2z6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:21.260769 kubelet[2720]: I0813 01:27:21.260726 2720 kubelet.go:2405] "Pod admission denied" podUID="6480978c-177b-4a54-a384-cd8f9b68e71e" pod="tigera-operator/tigera-operator-747864d56d-mcgcj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:21.361118 kubelet[2720]: I0813 01:27:21.360853 2720 kubelet.go:2405] "Pod admission denied" podUID="4cb051de-0593-40db-a56d-087c748b4419" pod="tigera-operator/tigera-operator-747864d56d-9q2w6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:21.410702 kubelet[2720]: I0813 01:27:21.409892 2720 kubelet.go:2405] "Pod admission denied" podUID="c3a3c717-cb3d-4fd1-b2be-bb4c886be1ec" pod="tigera-operator/tigera-operator-747864d56d-gmcqx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:21.507889 kubelet[2720]: I0813 01:27:21.507836 2720 kubelet.go:2405] "Pod admission denied" podUID="6e83e373-cd38-4b46-b122-50c7bb7f5857" pod="tigera-operator/tigera-operator-747864d56d-fxhqf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:21.607760 kubelet[2720]: I0813 01:27:21.607707 2720 kubelet.go:2405] "Pod admission denied" podUID="6c2a61d1-7d66-4db3-9f05-37c0156b5152" pod="tigera-operator/tigera-operator-747864d56d-wl5br" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:21.722011 kubelet[2720]: I0813 01:27:21.721968 2720 kubelet.go:2405] "Pod admission denied" podUID="1d774c71-a19b-4ee9-b942-7938de925cc2" pod="tigera-operator/tigera-operator-747864d56d-2brvz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:21.907483 kubelet[2720]: I0813 01:27:21.907430 2720 kubelet.go:2405] "Pod admission denied" podUID="200a33d2-a036-43ba-99cd-d51224f64f5d" pod="tigera-operator/tigera-operator-747864d56d-d4rdb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:22.015269 kubelet[2720]: I0813 01:27:22.014786 2720 kubelet.go:2405] "Pod admission denied" podUID="62e15210-24fd-4f3a-93df-fd7fd34b07ab" pod="tigera-operator/tigera-operator-747864d56d-pl4zh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:22.083791 kubelet[2720]: E0813 01:27:22.083766 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:27:22.085207 containerd[1567]: time="2025-08-13T01:27:22.084535404Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:27:22.150820 kubelet[2720]: I0813 01:27:22.150783 2720 kubelet.go:2405] "Pod admission denied" podUID="78035f4b-bb99-43f3-b079-8b0f7b185e63" pod="tigera-operator/tigera-operator-747864d56d-q4s7d" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:22.167043 containerd[1567]: time="2025-08-13T01:27:22.167006656Z" level=error msg="Failed to destroy network for sandbox \"69c20c7b2e2fb36719bc3bc09eabbe1e2c9d058af93b3b034ac3cada14327be6\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:22.169689 systemd[1]: run-netns-cni\x2d29e7cb8e\x2dd990\x2d82e3\x2d6c2a\x2d85d55ca8321d.mount: Deactivated successfully. Aug 13 01:27:22.171840 containerd[1567]: time="2025-08-13T01:27:22.171476802Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"69c20c7b2e2fb36719bc3bc09eabbe1e2c9d058af93b3b034ac3cada14327be6\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:22.172546 kubelet[2720]: E0813 01:27:22.171999 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"69c20c7b2e2fb36719bc3bc09eabbe1e2c9d058af93b3b034ac3cada14327be6\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:22.172546 kubelet[2720]: E0813 01:27:22.172040 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"69c20c7b2e2fb36719bc3bc09eabbe1e2c9d058af93b3b034ac3cada14327be6\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:27:22.172546 kubelet[2720]: E0813 01:27:22.172061 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"69c20c7b2e2fb36719bc3bc09eabbe1e2c9d058af93b3b034ac3cada14327be6\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:27:22.172546 kubelet[2720]: E0813 01:27:22.172099 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"69c20c7b2e2fb36719bc3bc09eabbe1e2c9d058af93b3b034ac3cada14327be6\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:27:22.225733 kubelet[2720]: I0813 01:27:22.225657 2720 kubelet.go:2405] "Pod admission denied" podUID="71705ca6-c185-4be9-aebd-64a2a91db3aa" pod="tigera-operator/tigera-operator-747864d56d-bxb2d" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:22.309755 kubelet[2720]: I0813 01:27:22.309612 2720 kubelet.go:2405] "Pod admission denied" podUID="4e22f803-cd1f-4ff8-b117-7be578d2c0af" pod="tigera-operator/tigera-operator-747864d56d-tx94m" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:22.511733 kubelet[2720]: I0813 01:27:22.510728 2720 kubelet.go:2405] "Pod admission denied" podUID="8537d321-ce3a-4648-be67-e97a440fa50f" pod="tigera-operator/tigera-operator-747864d56d-nmntk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:22.606027 kubelet[2720]: I0813 01:27:22.605791 2720 kubelet.go:2405] "Pod admission denied" podUID="38fd839a-bebf-408a-b725-b4e27a95b064" pod="tigera-operator/tigera-operator-747864d56d-2kjn6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:22.706935 kubelet[2720]: I0813 01:27:22.706873 2720 kubelet.go:2405] "Pod admission denied" podUID="5d7d91c3-3682-46ee-95da-e0ae21c76603" pod="tigera-operator/tigera-operator-747864d56d-qpvqv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:22.806026 kubelet[2720]: I0813 01:27:22.805977 2720 kubelet.go:2405] "Pod admission denied" podUID="1072f42b-c6bf-4a9b-b277-e93eae88fca9" pod="tigera-operator/tigera-operator-747864d56d-dp76k" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:22.858319 kubelet[2720]: I0813 01:27:22.857705 2720 kubelet.go:2405] "Pod admission denied" podUID="f20fa7d7-e7f3-4c3d-bdb8-859114a53d22" pod="tigera-operator/tigera-operator-747864d56d-2v5jn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:22.955528 kubelet[2720]: I0813 01:27:22.955484 2720 kubelet.go:2405] "Pod admission denied" podUID="9e2d1752-584a-4804-b858-a519aee3daa4" pod="tigera-operator/tigera-operator-747864d56d-2fzfd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:23.056494 kubelet[2720]: I0813 01:27:23.056444 2720 kubelet.go:2405] "Pod admission denied" podUID="27739fe7-0d88-4def-bef1-e82a3fad7c8d" pod="tigera-operator/tigera-operator-747864d56d-ddx5t" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:23.113565 kubelet[2720]: I0813 01:27:23.113451 2720 kubelet.go:2405] "Pod admission denied" podUID="24bd91e2-5d9f-4957-92b3-e2cf11f5f380" pod="tigera-operator/tigera-operator-747864d56d-hks72" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:23.204694 kubelet[2720]: I0813 01:27:23.204648 2720 kubelet.go:2405] "Pod admission denied" podUID="2f353780-d91a-4768-bb9f-a8bc7c6baf1e" pod="tigera-operator/tigera-operator-747864d56d-rhs8h" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:23.320468 kubelet[2720]: I0813 01:27:23.320420 2720 kubelet.go:2405] "Pod admission denied" podUID="f432e086-9a13-42c1-9bd3-9bb6bf61c017" pod="tigera-operator/tigera-operator-747864d56d-ssskt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:23.404427 kubelet[2720]: I0813 01:27:23.404325 2720 kubelet.go:2405] "Pod admission denied" podUID="7a5dd1bd-6de1-40f4-a6c5-14455e3e2d7f" pod="tigera-operator/tigera-operator-747864d56d-jflqv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:23.503849 kubelet[2720]: I0813 01:27:23.503804 2720 kubelet.go:2405] "Pod admission denied" podUID="85256183-b9db-48a9-af2f-3cd86d9ed60b" pod="tigera-operator/tigera-operator-747864d56d-zlqzl" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:23.558486 kubelet[2720]: I0813 01:27:23.558432 2720 kubelet.go:2405] "Pod admission denied" podUID="429f3409-4fc0-46ba-8708-5856829c0bdb" pod="tigera-operator/tigera-operator-747864d56d-r829x" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:23.655674 kubelet[2720]: I0813 01:27:23.655370 2720 kubelet.go:2405] "Pod admission denied" podUID="526f1f34-b705-4d7f-b027-64174eacd4ae" pod="tigera-operator/tigera-operator-747864d56d-8wtsg" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:23.773237 kubelet[2720]: I0813 01:27:23.773181 2720 kubelet.go:2405] "Pod admission denied" podUID="a81ca20c-b57c-46e8-9bdd-d94f2cff4f03" pod="tigera-operator/tigera-operator-747864d56d-vgc28" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:23.857378 kubelet[2720]: I0813 01:27:23.857335 2720 kubelet.go:2405] "Pod admission denied" podUID="ac928216-b0c7-4c35-ab45-ce647ea9bd7b" pod="tigera-operator/tigera-operator-747864d56d-hlkwf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:24.062737 kubelet[2720]: I0813 01:27:24.061877 2720 kubelet.go:2405] "Pod admission denied" podUID="0ba42409-44e2-4786-99cf-7a7cae5a8338" pod="tigera-operator/tigera-operator-747864d56d-8hvw7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:24.160864 kubelet[2720]: I0813 01:27:24.160806 2720 kubelet.go:2405] "Pod admission denied" podUID="bc2037c8-a299-408f-bfe1-f3519363d44c" pod="tigera-operator/tigera-operator-747864d56d-gdk7s" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:24.258735 kubelet[2720]: I0813 01:27:24.258684 2720 kubelet.go:2405] "Pod admission denied" podUID="2ed741ef-3c1b-4614-ad55-b001c3781f06" pod="tigera-operator/tigera-operator-747864d56d-s55gd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:24.359182 kubelet[2720]: I0813 01:27:24.359049 2720 kubelet.go:2405] "Pod admission denied" podUID="467ac8f6-33bc-49db-82df-aac0e9bdd2e8" pod="tigera-operator/tigera-operator-747864d56d-l6mln" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:24.459367 kubelet[2720]: I0813 01:27:24.459314 2720 kubelet.go:2405] "Pod admission denied" podUID="ce41011e-5988-4863-ae28-50622be0fae3" pod="tigera-operator/tigera-operator-747864d56d-4g5t4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:24.576444 kubelet[2720]: I0813 01:27:24.576384 2720 kubelet.go:2405] "Pod admission denied" podUID="d8c2cbb5-ee68-4859-ad4a-7f8397ed19db" pod="tigera-operator/tigera-operator-747864d56d-5npn5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:24.621386 kubelet[2720]: I0813 01:27:24.620910 2720 kubelet.go:2405] "Pod admission denied" podUID="cf120119-f0df-469d-a314-fe93d48f2ee5" pod="tigera-operator/tigera-operator-747864d56d-456kp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:24.710265 kubelet[2720]: I0813 01:27:24.710224 2720 kubelet.go:2405] "Pod admission denied" podUID="b5ea80b2-04d8-4903-9351-01fba92b253c" pod="tigera-operator/tigera-operator-747864d56d-5wm5m" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:24.808501 kubelet[2720]: I0813 01:27:24.808449 2720 kubelet.go:2405] "Pod admission denied" podUID="3d2034c3-0f70-4e2c-979d-cd8eeeff3a9e" pod="tigera-operator/tigera-operator-747864d56d-lkzgb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:24.916258 kubelet[2720]: I0813 01:27:24.914856 2720 kubelet.go:2405] "Pod admission denied" podUID="1ac3d5dd-0c04-485a-b1b0-a312b75c0947" pod="tigera-operator/tigera-operator-747864d56d-z5f47" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:25.009594 kubelet[2720]: I0813 01:27:25.009539 2720 kubelet.go:2405] "Pod admission denied" podUID="c1126c1f-7123-438e-b398-da60b5c4453d" pod="tigera-operator/tigera-operator-747864d56d-6kktj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:25.073852 containerd[1567]: time="2025-08-13T01:27:25.073779143Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:27:25.124646 kubelet[2720]: I0813 01:27:25.124036 2720 kubelet.go:2405] "Pod admission denied" podUID="fec98141-1d39-42a7-a7c3-04f165f4fda3" pod="tigera-operator/tigera-operator-747864d56d-xr72b" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:25.146413 containerd[1567]: time="2025-08-13T01:27:25.146272263Z" level=error msg="Failed to destroy network for sandbox \"d7ab2e2468d7a979039f7b6ec3192ce2ba1e41c08a57e4ee56940bea99816fee\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:25.149359 systemd[1]: run-netns-cni\x2d1254dd23\x2daa54\x2d5758\x2d418c\x2daa0699f64186.mount: Deactivated successfully. Aug 13 01:27:25.155354 containerd[1567]: time="2025-08-13T01:27:25.154166633Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"d7ab2e2468d7a979039f7b6ec3192ce2ba1e41c08a57e4ee56940bea99816fee\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:25.159744 kubelet[2720]: E0813 01:27:25.159574 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d7ab2e2468d7a979039f7b6ec3192ce2ba1e41c08a57e4ee56940bea99816fee\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:25.159744 kubelet[2720]: E0813 01:27:25.159637 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d7ab2e2468d7a979039f7b6ec3192ce2ba1e41c08a57e4ee56940bea99816fee\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:27:25.159744 kubelet[2720]: E0813 01:27:25.159658 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d7ab2e2468d7a979039f7b6ec3192ce2ba1e41c08a57e4ee56940bea99816fee\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:27:25.159889 kubelet[2720]: E0813 01:27:25.159702 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"d7ab2e2468d7a979039f7b6ec3192ce2ba1e41c08a57e4ee56940bea99816fee\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:27:25.210876 kubelet[2720]: I0813 01:27:25.210330 2720 kubelet.go:2405] "Pod admission denied" podUID="ccbd7288-7c83-43d1-97e1-56efce2b7ac3" pod="tigera-operator/tigera-operator-747864d56d-bkt7l" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:25.309373 kubelet[2720]: I0813 01:27:25.309313 2720 kubelet.go:2405] "Pod admission denied" podUID="62bc1bd9-d221-4804-a97c-5acbe867a240" pod="tigera-operator/tigera-operator-747864d56d-2n2t5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:25.532026 kubelet[2720]: I0813 01:27:25.531973 2720 kubelet.go:2405] "Pod admission denied" podUID="458a8cde-ac18-415d-bbeb-42987ce78a59" pod="tigera-operator/tigera-operator-747864d56d-gkf5b" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:25.606907 kubelet[2720]: I0813 01:27:25.606871 2720 kubelet.go:2405] "Pod admission denied" podUID="c51e6336-34fe-4e66-b5b6-df0b04d8be36" pod="tigera-operator/tigera-operator-747864d56d-n8p8q" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:25.716734 kubelet[2720]: I0813 01:27:25.716184 2720 kubelet.go:2405] "Pod admission denied" podUID="5286245a-98df-40dd-9851-5728c4b943e8" pod="tigera-operator/tigera-operator-747864d56d-xfctx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:25.809579 kubelet[2720]: I0813 01:27:25.809104 2720 kubelet.go:2405] "Pod admission denied" podUID="5c821ec9-9847-401c-9df7-f763945e9fc9" pod="tigera-operator/tigera-operator-747864d56d-rpdsh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:25.859553 kubelet[2720]: I0813 01:27:25.859499 2720 kubelet.go:2405] "Pod admission denied" podUID="7483ebad-65b9-482f-b4dd-537fcb46092d" pod="tigera-operator/tigera-operator-747864d56d-zgh9q" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:25.959587 kubelet[2720]: I0813 01:27:25.959529 2720 kubelet.go:2405] "Pod admission denied" podUID="6d1939fc-1f07-45aa-8dfb-8c10520d5d98" pod="tigera-operator/tigera-operator-747864d56d-59f7c" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:26.058864 kubelet[2720]: I0813 01:27:26.058817 2720 kubelet.go:2405] "Pod admission denied" podUID="535c8005-da9a-42ca-9120-66146ec9fd90" pod="tigera-operator/tigera-operator-747864d56d-276lw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:26.180002 kubelet[2720]: I0813 01:27:26.179175 2720 kubelet.go:2405] "Pod admission denied" podUID="e4dcd924-8f7f-4e70-b8d1-cd2b6a331eaf" pod="tigera-operator/tigera-operator-747864d56d-9ts9z" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:26.260965 kubelet[2720]: I0813 01:27:26.260907 2720 kubelet.go:2405] "Pod admission denied" podUID="f1327b33-e913-4ec1-b293-d05e3b4b04e9" pod="tigera-operator/tigera-operator-747864d56d-kh9h4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:26.361735 kubelet[2720]: I0813 01:27:26.361422 2720 kubelet.go:2405] "Pod admission denied" podUID="372f2719-1ffe-489a-8ab8-970240c3fc14" pod="tigera-operator/tigera-operator-747864d56d-mnnmk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:26.564927 kubelet[2720]: I0813 01:27:26.564873 2720 kubelet.go:2405] "Pod admission denied" podUID="81716b0f-4bcc-4f25-a878-8e15283d0bac" pod="tigera-operator/tigera-operator-747864d56d-ndjj7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:26.658744 kubelet[2720]: I0813 01:27:26.658670 2720 kubelet.go:2405] "Pod admission denied" podUID="59496a27-0040-4ca4-bd7c-db3191d35f66" pod="tigera-operator/tigera-operator-747864d56d-snmfl" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:26.763008 kubelet[2720]: I0813 01:27:26.762955 2720 kubelet.go:2405] "Pod admission denied" podUID="07ce1e91-6162-4d0a-ae66-852fa6d94c81" pod="tigera-operator/tigera-operator-747864d56d-tmkg6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:26.859194 kubelet[2720]: I0813 01:27:26.859068 2720 kubelet.go:2405] "Pod admission denied" podUID="a0c5d5fd-e9f6-4c53-8e2c-4180644b3bf3" pod="tigera-operator/tigera-operator-747864d56d-mjxfx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:26.978738 kubelet[2720]: I0813 01:27:26.977956 2720 kubelet.go:2405] "Pod admission denied" podUID="b401f259-3631-495f-a767-3b985d8fd1fe" pod="tigera-operator/tigera-operator-747864d56d-8nvvq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:27.209758 kubelet[2720]: I0813 01:27:27.208794 2720 kubelet.go:2405] "Pod admission denied" podUID="3aad6450-e669-4546-96c9-816189318baf" pod="tigera-operator/tigera-operator-747864d56d-bgwc8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:27.305226 kubelet[2720]: I0813 01:27:27.305177 2720 kubelet.go:2405] "Pod admission denied" podUID="fcbd6172-2dc8-4db5-af86-3977f1c856e0" pod="tigera-operator/tigera-operator-747864d56d-b9sw9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:27.408937 kubelet[2720]: I0813 01:27:27.408890 2720 kubelet.go:2405] "Pod admission denied" podUID="1322633e-ca8f-4bc8-8bca-8cf12d96393c" pod="tigera-operator/tigera-operator-747864d56d-bgkgq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:27.507763 kubelet[2720]: I0813 01:27:27.507701 2720 kubelet.go:2405] "Pod admission denied" podUID="3be6d246-0195-41ed-a5f5-06cf50a2a21e" pod="tigera-operator/tigera-operator-747864d56d-kttbm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:27.617673 kubelet[2720]: I0813 01:27:27.616903 2720 kubelet.go:2405] "Pod admission denied" podUID="dcce936a-af07-4a3e-bddf-8a7de9a094dd" pod="tigera-operator/tigera-operator-747864d56d-qxrfv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:27.707852 kubelet[2720]: I0813 01:27:27.707811 2720 kubelet.go:2405] "Pod admission denied" podUID="ebdb1203-9702-48bd-a3fc-c5d46e1f6624" pod="tigera-operator/tigera-operator-747864d56d-fz8fp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:27.804442 kubelet[2720]: I0813 01:27:27.804170 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:27:27.804442 kubelet[2720]: I0813 01:27:27.804208 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:27:27.807563 kubelet[2720]: I0813 01:27:27.807538 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:27:27.809732 kubelet[2720]: I0813 01:27:27.809684 2720 image_gc_manager.go:514] "Removing image to free bytes" imageID="sha256:1cf5f116067c67da67f97bff78c4bbc76913f59057c18627b96facaced73ea0b" size=20939036 runtimeHandler="" Aug 13 01:27:27.810052 containerd[1567]: time="2025-08-13T01:27:27.810019666Z" level=info msg="RemoveImage \"sha256:1cf5f116067c67da67f97bff78c4bbc76913f59057c18627b96facaced73ea0b\"" Aug 13 01:27:27.811597 containerd[1567]: time="2025-08-13T01:27:27.811571329Z" level=info msg="ImageDelete event name:\"registry.k8s.io/coredns/coredns:v1.12.0\"" Aug 13 01:27:27.812263 containerd[1567]: time="2025-08-13T01:27:27.812238825Z" level=info msg="ImageDelete event name:\"registry.k8s.io/coredns/coredns@sha256:40384aa1f5ea6bfdc77997d243aec73da05f27aed0c5e9d65bfa98933c519d97\"" Aug 13 01:27:27.812638 containerd[1567]: time="2025-08-13T01:27:27.812617124Z" level=info msg="RemoveImage \"sha256:1cf5f116067c67da67f97bff78c4bbc76913f59057c18627b96facaced73ea0b\" returns successfully" Aug 13 01:27:27.812739 containerd[1567]: time="2025-08-13T01:27:27.812699043Z" level=info msg="ImageDelete event name:\"sha256:1cf5f116067c67da67f97bff78c4bbc76913f59057c18627b96facaced73ea0b\"" Aug 13 01:27:27.812845 kubelet[2720]: I0813 01:27:27.812827 2720 image_gc_manager.go:514] "Removing image to free bytes" imageID="sha256:499038711c0816eda03a1ad96a8eb0440c005baa6949698223c6176b7f5077e1" size=58938593 runtimeHandler="" Aug 13 01:27:27.812988 containerd[1567]: time="2025-08-13T01:27:27.812940502Z" level=info msg="RemoveImage \"sha256:499038711c0816eda03a1ad96a8eb0440c005baa6949698223c6176b7f5077e1\"" Aug 13 01:27:27.813631 containerd[1567]: time="2025-08-13T01:27:27.813611619Z" level=info msg="ImageDelete event name:\"registry.k8s.io/etcd:3.5.21-0\"" Aug 13 01:27:27.814327 containerd[1567]: time="2025-08-13T01:27:27.814268026Z" level=info msg="ImageDelete event name:\"registry.k8s.io/etcd@sha256:d58c035df557080a27387d687092e3fc2b64c6d0e3162dc51453a115f847d121\"" Aug 13 01:27:27.814975 containerd[1567]: time="2025-08-13T01:27:27.814953142Z" level=info msg="RemoveImage \"sha256:499038711c0816eda03a1ad96a8eb0440c005baa6949698223c6176b7f5077e1\" returns successfully" Aug 13 01:27:27.815105 containerd[1567]: time="2025-08-13T01:27:27.815016712Z" level=info msg="ImageDelete event name:\"sha256:499038711c0816eda03a1ad96a8eb0440c005baa6949698223c6176b7f5077e1\"" Aug 13 01:27:27.818448 kubelet[2720]: I0813 01:27:27.818424 2720 kubelet.go:2405] "Pod admission denied" podUID="1f7ac84d-b4fe-4737-a103-94bfe28184d7" pod="tigera-operator/tigera-operator-747864d56d-jfgzl" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:27.846254 kubelet[2720]: I0813 01:27:27.846229 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:27:27.846499 kubelet[2720]: I0813 01:27:27.846483 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","calico-system/calico-node-2p8hj","calico-system/csi-node-driver-8bzzs","calico-system/calico-typha-5bf7f956b7-szw5v","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:27:27.846615 kubelet[2720]: E0813 01:27:27.846603 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:27:27.846681 kubelet[2720]: E0813 01:27:27.846671 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:27:27.846750 kubelet[2720]: E0813 01:27:27.846741 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:27:27.846795 kubelet[2720]: E0813 01:27:27.846787 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:27:27.846845 kubelet[2720]: E0813 01:27:27.846837 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:27:27.846897 kubelet[2720]: E0813 01:27:27.846889 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:27:27.846941 kubelet[2720]: E0813 01:27:27.846933 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:27:27.846990 kubelet[2720]: E0813 01:27:27.846981 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:27:27.847056 kubelet[2720]: E0813 01:27:27.847046 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:27:27.847105 kubelet[2720]: E0813 01:27:27.847097 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:27:27.847990 kubelet[2720]: I0813 01:27:27.847156 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:27:27.909870 kubelet[2720]: I0813 01:27:27.908603 2720 kubelet.go:2405] "Pod admission denied" podUID="c2c1f6c8-4ede-41ef-841f-fe956ecdcc12" pod="tigera-operator/tigera-operator-747864d56d-8fc8b" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:28.011473 kubelet[2720]: I0813 01:27:28.011427 2720 kubelet.go:2405] "Pod admission denied" podUID="3193ab3a-72a1-40d6-9fea-069018b37f99" pod="tigera-operator/tigera-operator-747864d56d-dkvdc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:28.068139 kubelet[2720]: I0813 01:27:28.068015 2720 kubelet.go:2405] "Pod admission denied" podUID="3b04df1f-1124-4e50-86f9-8c6338ccccfb" pod="tigera-operator/tigera-operator-747864d56d-4qknj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:28.076042 kubelet[2720]: E0813 01:27:28.075781 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"calico-node\" with ImagePullBackOff: \"Back-off pulling image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": ErrImagePull: failed to pull and unpack image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount849196455: write /var/lib/containerd/tmpmounts/containerd-mount849196455/usr/bin/calico-node: no space left on device\"" pod="calico-system/calico-node-2p8hj" podUID="4f334296-e8c0-427e-a090-d8fbd3582f26" Aug 13 01:27:28.158698 kubelet[2720]: I0813 01:27:28.158653 2720 kubelet.go:2405] "Pod admission denied" podUID="3307bdb6-af2d-47b5-8321-2c78662d092e" pod="tigera-operator/tigera-operator-747864d56d-skqk4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:28.256307 kubelet[2720]: I0813 01:27:28.256257 2720 kubelet.go:2405] "Pod admission denied" podUID="ffb4b6f5-5ad3-4288-9a58-5cccc78ddf07" pod="tigera-operator/tigera-operator-747864d56d-25xsv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:28.355735 kubelet[2720]: I0813 01:27:28.355615 2720 kubelet.go:2405] "Pod admission denied" podUID="23cc5b54-aaa6-4912-8f31-0bcf1a85542e" pod="tigera-operator/tigera-operator-747864d56d-5cz4z" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:28.456360 kubelet[2720]: I0813 01:27:28.456312 2720 kubelet.go:2405] "Pod admission denied" podUID="8ea7b3b7-c770-4871-a68f-ca425e44251c" pod="tigera-operator/tigera-operator-747864d56d-fnb6q" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:28.504129 kubelet[2720]: I0813 01:27:28.504084 2720 kubelet.go:2405] "Pod admission denied" podUID="59a05326-2ac3-4d7a-9ddd-be0f3307cdbc" pod="tigera-operator/tigera-operator-747864d56d-gvswp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:28.607189 kubelet[2720]: I0813 01:27:28.606920 2720 kubelet.go:2405] "Pod admission denied" podUID="ab3f1d4b-151c-421c-9fcd-c1659189e377" pod="tigera-operator/tigera-operator-747864d56d-fnxp8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:28.708371 kubelet[2720]: I0813 01:27:28.708326 2720 kubelet.go:2405] "Pod admission denied" podUID="45e5824c-3927-4b04-9968-434a43e52ad6" pod="tigera-operator/tigera-operator-747864d56d-g95fk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:28.768219 kubelet[2720]: I0813 01:27:28.768175 2720 kubelet.go:2405] "Pod admission denied" podUID="c24aaac4-6cbc-4248-842a-e51c66035643" pod="tigera-operator/tigera-operator-747864d56d-qrx64" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:28.854998 kubelet[2720]: I0813 01:27:28.854953 2720 kubelet.go:2405] "Pod admission denied" podUID="29666084-cc66-4f0d-97c3-c13c89fd43c9" pod="tigera-operator/tigera-operator-747864d56d-bn7l8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:29.058075 kubelet[2720]: I0813 01:27:29.058028 2720 kubelet.go:2405] "Pod admission denied" podUID="910e97e8-484a-4ca1-a990-e9fa3f014aff" pod="tigera-operator/tigera-operator-747864d56d-kw7pq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:29.156772 kubelet[2720]: I0813 01:27:29.156729 2720 kubelet.go:2405] "Pod admission denied" podUID="a40112bc-acf5-4b0b-9f36-56e83d1574bb" pod="tigera-operator/tigera-operator-747864d56d-45jdj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:29.266734 kubelet[2720]: I0813 01:27:29.266635 2720 kubelet.go:2405] "Pod admission denied" podUID="31f7142b-1e25-45ba-be84-396320b26978" pod="tigera-operator/tigera-operator-747864d56d-kfdxz" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:29.356546 kubelet[2720]: I0813 01:27:29.356427 2720 kubelet.go:2405] "Pod admission denied" podUID="7a0df906-8e79-4cea-85ce-36a1d3a95617" pod="tigera-operator/tigera-operator-747864d56d-t6765" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:29.457906 kubelet[2720]: I0813 01:27:29.457854 2720 kubelet.go:2405] "Pod admission denied" podUID="e4a816ca-a4b9-4ab9-81c8-7eed06d8d9e8" pod="tigera-operator/tigera-operator-747864d56d-kgqjr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:29.661072 kubelet[2720]: I0813 01:27:29.660403 2720 kubelet.go:2405] "Pod admission denied" podUID="c5a2cf29-c06e-4ddf-a91a-17e96bc4ba45" pod="tigera-operator/tigera-operator-747864d56d-bb8v6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:29.758666 kubelet[2720]: I0813 01:27:29.758618 2720 kubelet.go:2405] "Pod admission denied" podUID="85a2662f-b549-4529-a730-77f014fd335f" pod="tigera-operator/tigera-operator-747864d56d-ct6sb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:29.862199 kubelet[2720]: I0813 01:27:29.862135 2720 kubelet.go:2405] "Pod admission denied" podUID="2d097007-4688-4392-a319-38ef0c3ecfc3" pod="tigera-operator/tigera-operator-747864d56d-nplm6" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:29.960017 kubelet[2720]: I0813 01:27:29.959368 2720 kubelet.go:2405] "Pod admission denied" podUID="4c6bc956-92d4-422a-9d9b-932616d7548c" pod="tigera-operator/tigera-operator-747864d56d-2t7k8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:30.024100 kubelet[2720]: I0813 01:27:30.024039 2720 kubelet.go:2405] "Pod admission denied" podUID="dc5d10fe-8b98-4ff2-aaa3-76cbd8972950" pod="tigera-operator/tigera-operator-747864d56d-kbxp9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:30.075067 kubelet[2720]: E0813 01:27:30.073895 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:27:30.075910 containerd[1567]: time="2025-08-13T01:27:30.075856956Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:27:30.127367 kubelet[2720]: I0813 01:27:30.127343 2720 kubelet.go:2405] "Pod admission denied" podUID="75a4fe45-bafe-4626-b1b7-c30dadebafad" pod="tigera-operator/tigera-operator-747864d56d-8p8jx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:30.149157 containerd[1567]: time="2025-08-13T01:27:30.149098436Z" level=error msg="Failed to destroy network for sandbox \"e37ef052e070049f24040924d7d12b47ff12ce0870f4705cca6d2a095f704305\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:30.151490 systemd[1]: run-netns-cni\x2d21d09dc9\x2dd17b\x2d1217\x2dbd6c\x2d9683d0059763.mount: Deactivated successfully. Aug 13 01:27:30.156119 containerd[1567]: time="2025-08-13T01:27:30.153563786Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"e37ef052e070049f24040924d7d12b47ff12ce0870f4705cca6d2a095f704305\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:30.156423 kubelet[2720]: E0813 01:27:30.154341 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e37ef052e070049f24040924d7d12b47ff12ce0870f4705cca6d2a095f704305\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:30.156423 kubelet[2720]: E0813 01:27:30.154394 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e37ef052e070049f24040924d7d12b47ff12ce0870f4705cca6d2a095f704305\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:27:30.156423 kubelet[2720]: E0813 01:27:30.154416 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e37ef052e070049f24040924d7d12b47ff12ce0870f4705cca6d2a095f704305\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:27:30.156423 kubelet[2720]: E0813 01:27:30.154459 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"e37ef052e070049f24040924d7d12b47ff12ce0870f4705cca6d2a095f704305\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:27:30.227328 kubelet[2720]: I0813 01:27:30.227285 2720 kubelet.go:2405] "Pod admission denied" podUID="c84a9e1e-5ad3-4009-a5e7-b6320f9f6848" pod="tigera-operator/tigera-operator-747864d56d-f88m9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:30.310635 kubelet[2720]: I0813 01:27:30.310588 2720 kubelet.go:2405] "Pod admission denied" podUID="2517b73a-296f-4b8e-b14e-800282a9b39b" pod="tigera-operator/tigera-operator-747864d56d-fbhv2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:30.420735 kubelet[2720]: I0813 01:27:30.419834 2720 kubelet.go:2405] "Pod admission denied" podUID="ab67302f-e4ff-4caf-9e0a-847c8e050dcd" pod="tigera-operator/tigera-operator-747864d56d-vk2zr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:30.479171 kubelet[2720]: I0813 01:27:30.478840 2720 kubelet.go:2405] "Pod admission denied" podUID="603d57a6-ebda-4a02-af94-a77472935949" pod="tigera-operator/tigera-operator-747864d56d-5dnnn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:30.569954 kubelet[2720]: I0813 01:27:30.569848 2720 kubelet.go:2405] "Pod admission denied" podUID="25e2b0b7-97b7-474f-bac8-f8024e6fc90f" pod="tigera-operator/tigera-operator-747864d56d-d7wtc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:30.660315 kubelet[2720]: I0813 01:27:30.660261 2720 kubelet.go:2405] "Pod admission denied" podUID="a54e7f13-6107-47b7-b7e2-2bf4b9550e86" pod="tigera-operator/tigera-operator-747864d56d-8rn2n" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:30.758577 kubelet[2720]: I0813 01:27:30.758290 2720 kubelet.go:2405] "Pod admission denied" podUID="f1c0b2a1-faf7-4df1-9077-999722a4d1e6" pod="tigera-operator/tigera-operator-747864d56d-vhn4n" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:30.961739 kubelet[2720]: I0813 01:27:30.961673 2720 kubelet.go:2405] "Pod admission denied" podUID="af1e9c6e-812d-4b6d-8d2f-cdfa74389305" pod="tigera-operator/tigera-operator-747864d56d-h2678" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:31.078021 kubelet[2720]: I0813 01:27:31.077890 2720 kubelet.go:2405] "Pod admission denied" podUID="cbbbf772-6276-4c9a-bf8b-2a92dcf68f59" pod="tigera-operator/tigera-operator-747864d56d-qbdtm" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:31.162740 kubelet[2720]: I0813 01:27:31.162135 2720 kubelet.go:2405] "Pod admission denied" podUID="1e20a696-c359-4876-bdd9-6ad5911d8b89" pod="tigera-operator/tigera-operator-747864d56d-zvl5n" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:31.259422 kubelet[2720]: I0813 01:27:31.259374 2720 kubelet.go:2405] "Pod admission denied" podUID="0b3c85d3-0ef5-49a5-a659-ec263aca4f18" pod="tigera-operator/tigera-operator-747864d56d-lngcf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:31.363094 kubelet[2720]: I0813 01:27:31.362502 2720 kubelet.go:2405] "Pod admission denied" podUID="029d61e4-1561-4edd-9ae0-cd50f705d50e" pod="tigera-operator/tigera-operator-747864d56d-jq9nn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:31.460235 kubelet[2720]: I0813 01:27:31.460176 2720 kubelet.go:2405] "Pod admission denied" podUID="e2e6b3fd-9ebe-4b52-9e75-e146e6ae9696" pod="tigera-operator/tigera-operator-747864d56d-tbknw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:31.559880 kubelet[2720]: I0813 01:27:31.559818 2720 kubelet.go:2405] "Pod admission denied" podUID="b5582fae-ff0b-4866-855c-f08cc2ef19a8" pod="tigera-operator/tigera-operator-747864d56d-znk5j" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:31.675819 kubelet[2720]: I0813 01:27:31.675242 2720 kubelet.go:2405] "Pod admission denied" podUID="2846c18f-80c0-42d8-ab8c-51a5da88ae56" pod="tigera-operator/tigera-operator-747864d56d-2lqq2" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:31.728267 kubelet[2720]: I0813 01:27:31.728208 2720 kubelet.go:2405] "Pod admission denied" podUID="0ff86d5d-0a9a-47d1-9f77-8f35ea666dd2" pod="tigera-operator/tigera-operator-747864d56d-r6s5x" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:31.814163 kubelet[2720]: I0813 01:27:31.814116 2720 kubelet.go:2405] "Pod admission denied" podUID="2ea4609e-6b17-4c04-a4d8-fb94371e9eb6" pod="tigera-operator/tigera-operator-747864d56d-5r6hj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:31.909393 kubelet[2720]: I0813 01:27:31.909336 2720 kubelet.go:2405] "Pod admission denied" podUID="cd006f1e-5643-4cc7-9ad6-e1436ff9f3a1" pod="tigera-operator/tigera-operator-747864d56d-bb2tn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:32.017505 kubelet[2720]: I0813 01:27:32.017471 2720 kubelet.go:2405] "Pod admission denied" podUID="745464b3-de5f-4b19-8213-71196858a5f3" pod="tigera-operator/tigera-operator-747864d56d-7m7kk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:32.084004 containerd[1567]: time="2025-08-13T01:27:32.083625804Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:27:32.145091 kubelet[2720]: I0813 01:27:32.143703 2720 kubelet.go:2405] "Pod admission denied" podUID="0ef6aabb-cff8-41c4-a864-f2651921ba22" pod="tigera-operator/tigera-operator-747864d56d-lncnx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:32.171898 containerd[1567]: time="2025-08-13T01:27:32.171855491Z" level=error msg="Failed to destroy network for sandbox \"53259c06e72a6e3ee6243f75e00e0de0b3686faccb1b28e8f49668ddb088faed\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:32.175885 systemd[1]: run-netns-cni\x2de22f1f8d\x2d8650\x2d0a31\x2ddc27\x2dd4f1fdc83d48.mount: Deactivated successfully. Aug 13 01:27:32.177514 containerd[1567]: time="2025-08-13T01:27:32.177387346Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"53259c06e72a6e3ee6243f75e00e0de0b3686faccb1b28e8f49668ddb088faed\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:32.177613 kubelet[2720]: E0813 01:27:32.177554 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"53259c06e72a6e3ee6243f75e00e0de0b3686faccb1b28e8f49668ddb088faed\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:32.177613 kubelet[2720]: E0813 01:27:32.177599 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"53259c06e72a6e3ee6243f75e00e0de0b3686faccb1b28e8f49668ddb088faed\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:27:32.177680 kubelet[2720]: E0813 01:27:32.177619 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"53259c06e72a6e3ee6243f75e00e0de0b3686faccb1b28e8f49668ddb088faed\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:27:32.177680 kubelet[2720]: E0813 01:27:32.177664 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"53259c06e72a6e3ee6243f75e00e0de0b3686faccb1b28e8f49668ddb088faed\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:27:32.261901 kubelet[2720]: I0813 01:27:32.261733 2720 kubelet.go:2405] "Pod admission denied" podUID="d45d02ae-d743-49eb-8619-ea47decb6720" pod="tigera-operator/tigera-operator-747864d56d-vprzs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:32.363025 kubelet[2720]: I0813 01:27:32.362892 2720 kubelet.go:2405] "Pod admission denied" podUID="98ad1b81-78ac-41de-8e9e-1438e46ca86f" pod="tigera-operator/tigera-operator-747864d56d-l7dc8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:32.463352 kubelet[2720]: I0813 01:27:32.463309 2720 kubelet.go:2405] "Pod admission denied" podUID="5e55d591-b839-4acf-b7fa-9e66b92a26d9" pod="tigera-operator/tigera-operator-747864d56d-tqbpn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:32.560638 kubelet[2720]: I0813 01:27:32.560576 2720 kubelet.go:2405] "Pod admission denied" podUID="80a81421-5a5d-43c0-a5f1-63a6c43a3b27" pod="tigera-operator/tigera-operator-747864d56d-dbzvw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:32.661051 kubelet[2720]: I0813 01:27:32.660938 2720 kubelet.go:2405] "Pod admission denied" podUID="00af588e-ea5a-4407-8ddd-1bec1f770541" pod="tigera-operator/tigera-operator-747864d56d-lknzn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:32.761124 kubelet[2720]: I0813 01:27:32.761073 2720 kubelet.go:2405] "Pod admission denied" podUID="432c4994-7dbc-4937-acae-828b8cef806f" pod="tigera-operator/tigera-operator-747864d56d-6d857" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:32.960660 kubelet[2720]: I0813 01:27:32.960533 2720 kubelet.go:2405] "Pod admission denied" podUID="aa1893c0-560c-48e3-a933-6e3da136b940" pod="tigera-operator/tigera-operator-747864d56d-v8jh7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:33.060188 kubelet[2720]: I0813 01:27:33.060138 2720 kubelet.go:2405] "Pod admission denied" podUID="518561fa-d6dd-44d8-a8ef-cfe54e561115" pod="tigera-operator/tigera-operator-747864d56d-z48lc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:33.160805 kubelet[2720]: I0813 01:27:33.160755 2720 kubelet.go:2405] "Pod admission denied" podUID="ff53798f-711e-4ba8-9435-784f403e0634" pod="tigera-operator/tigera-operator-747864d56d-mcx5j" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:33.260574 kubelet[2720]: I0813 01:27:33.260530 2720 kubelet.go:2405] "Pod admission denied" podUID="94818c8c-3f70-458a-babd-3ec7cd07f136" pod="tigera-operator/tigera-operator-747864d56d-f8qnt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:33.358239 kubelet[2720]: I0813 01:27:33.358190 2720 kubelet.go:2405] "Pod admission denied" podUID="8914f5db-b3a1-4d2f-84a6-dbbbe4381dc3" pod="tigera-operator/tigera-operator-747864d56d-5xtnb" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:33.472494 kubelet[2720]: I0813 01:27:33.472444 2720 kubelet.go:2405] "Pod admission denied" podUID="0adc5ac5-3e5d-488b-8233-b4ae4f6f9fc7" pod="tigera-operator/tigera-operator-747864d56d-8vj2r" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:33.558164 kubelet[2720]: I0813 01:27:33.558055 2720 kubelet.go:2405] "Pod admission denied" podUID="3cf4e7af-c027-44ff-b6e9-74efad50c5b2" pod="tigera-operator/tigera-operator-747864d56d-mqn9m" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:33.661744 kubelet[2720]: I0813 01:27:33.661102 2720 kubelet.go:2405] "Pod admission denied" podUID="57c5dbd8-3ebd-4174-a394-2c434b46c4a7" pod="tigera-operator/tigera-operator-747864d56d-6fvc4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:33.760316 kubelet[2720]: I0813 01:27:33.760268 2720 kubelet.go:2405] "Pod admission denied" podUID="93ad9275-ad77-4c4e-91bd-4bc10d517ade" pod="tigera-operator/tigera-operator-747864d56d-h4v9j" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:33.865732 kubelet[2720]: I0813 01:27:33.864920 2720 kubelet.go:2405] "Pod admission denied" podUID="4eb9b0e6-64f1-44ed-b3cb-73f33bb58b05" pod="tigera-operator/tigera-operator-747864d56d-nm5tx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:33.931083 kubelet[2720]: I0813 01:27:33.931040 2720 kubelet.go:2405] "Pod admission denied" podUID="106091e9-1a9d-45c9-a92c-dbc97e087ddc" pod="tigera-operator/tigera-operator-747864d56d-xgfxs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:34.065736 kubelet[2720]: I0813 01:27:34.065285 2720 kubelet.go:2405] "Pod admission denied" podUID="d392d266-75b7-4b82-98a9-bc2224a883ca" pod="tigera-operator/tigera-operator-747864d56d-r9vkd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:34.160853 kubelet[2720]: I0813 01:27:34.160700 2720 kubelet.go:2405] "Pod admission denied" podUID="44f0767e-6254-404e-bfef-c59c051fec8e" pod="tigera-operator/tigera-operator-747864d56d-2qppd" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:34.361182 kubelet[2720]: I0813 01:27:34.361124 2720 kubelet.go:2405] "Pod admission denied" podUID="76d6f01a-5f37-4f00-8279-1dba0fe0ac27" pod="tigera-operator/tigera-operator-747864d56d-86zff" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:34.488521 kubelet[2720]: I0813 01:27:34.488464 2720 kubelet.go:2405] "Pod admission denied" podUID="ababb8ba-a759-4998-81e8-5d07051bc36e" pod="tigera-operator/tigera-operator-747864d56d-xztpk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:34.609525 kubelet[2720]: I0813 01:27:34.609475 2720 kubelet.go:2405] "Pod admission denied" podUID="30aea20f-cbff-4387-8bf6-9030b8425db2" pod="tigera-operator/tigera-operator-747864d56d-wplbx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:34.671148 kubelet[2720]: I0813 01:27:34.671087 2720 kubelet.go:2405] "Pod admission denied" podUID="d5fed0db-0214-4c2d-8793-2cb54bd97f23" pod="tigera-operator/tigera-operator-747864d56d-9vqh8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:34.762337 kubelet[2720]: I0813 01:27:34.762169 2720 kubelet.go:2405] "Pod admission denied" podUID="9278fed7-0547-43a0-a56f-831373b588d4" pod="tigera-operator/tigera-operator-747864d56d-fjnvv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:34.971966 kubelet[2720]: I0813 01:27:34.971907 2720 kubelet.go:2405] "Pod admission denied" podUID="0e8b5c2b-c40c-4009-a001-057d3573fe98" pod="tigera-operator/tigera-operator-747864d56d-hkn7t" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:35.060813 kubelet[2720]: I0813 01:27:35.060233 2720 kubelet.go:2405] "Pod admission denied" podUID="bc07fd74-2518-4726-922e-2e973de930fe" pod="tigera-operator/tigera-operator-747864d56d-cm9xc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:35.163087 kubelet[2720]: I0813 01:27:35.163013 2720 kubelet.go:2405] "Pod admission denied" podUID="08c94e82-4d9b-4f72-90af-7b37476848c8" pod="tigera-operator/tigera-operator-747864d56d-9q6zp" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:35.362544 kubelet[2720]: I0813 01:27:35.361978 2720 kubelet.go:2405] "Pod admission denied" podUID="f792fa1b-7887-4bbe-954f-e4dc32134241" pod="tigera-operator/tigera-operator-747864d56d-s8ts8" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:35.462380 kubelet[2720]: I0813 01:27:35.462338 2720 kubelet.go:2405] "Pod admission denied" podUID="f8281f78-63d1-4a21-9424-2cf56569e2ea" pod="tigera-operator/tigera-operator-747864d56d-8hzp4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:35.559741 kubelet[2720]: I0813 01:27:35.559675 2720 kubelet.go:2405] "Pod admission denied" podUID="77c250e4-c4ef-4c21-b783-50aef17db349" pod="tigera-operator/tigera-operator-747864d56d-5c9bc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:35.658549 kubelet[2720]: I0813 01:27:35.657907 2720 kubelet.go:2405] "Pod admission denied" podUID="d5ad542c-9db2-4d99-9024-756f3d8850ea" pod="tigera-operator/tigera-operator-747864d56d-msvld" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:35.735592 kubelet[2720]: I0813 01:27:35.735544 2720 kubelet.go:2405] "Pod admission denied" podUID="02cc10db-1d6f-4207-969e-85fc1cc074dd" pod="tigera-operator/tigera-operator-747864d56d-n6qwn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:35.861905 kubelet[2720]: I0813 01:27:35.861843 2720 kubelet.go:2405] "Pod admission denied" podUID="2d2d50c3-dba8-4e40-aada-f592111698f3" pod="tigera-operator/tigera-operator-747864d56d-vtx8b" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:35.990739 kubelet[2720]: I0813 01:27:35.990605 2720 kubelet.go:2405] "Pod admission denied" podUID="2f9ce54c-5d31-4336-a7a3-47d0e827fa8c" pod="tigera-operator/tigera-operator-747864d56d-9jlrx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:36.058733 kubelet[2720]: I0813 01:27:36.058677 2720 kubelet.go:2405] "Pod admission denied" podUID="29d0bf5e-02d3-4bde-b51b-10bd0fc69782" pod="tigera-operator/tigera-operator-747864d56d-dvql4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:36.076221 kubelet[2720]: E0813 01:27:36.076183 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:27:36.080535 containerd[1567]: time="2025-08-13T01:27:36.080387684Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:27:36.177171 kubelet[2720]: I0813 01:27:36.177113 2720 kubelet.go:2405] "Pod admission denied" podUID="bff77668-5a44-4ef3-98dd-a83cba3a06e7" pod="tigera-operator/tigera-operator-747864d56d-wc6dc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:36.192447 containerd[1567]: time="2025-08-13T01:27:36.191935041Z" level=error msg="Failed to destroy network for sandbox \"8d6884a733cc034195ae361b93bcb4bfe6075939a4fd2543b66f77280c25c3a3\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:36.195462 systemd[1]: run-netns-cni\x2d927b52b8\x2d9b4d\x2d83ba\x2dceca\x2dc6019fcdc652.mount: Deactivated successfully. Aug 13 01:27:36.198569 containerd[1567]: time="2025-08-13T01:27:36.198492165Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"8d6884a733cc034195ae361b93bcb4bfe6075939a4fd2543b66f77280c25c3a3\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:36.201069 kubelet[2720]: E0813 01:27:36.200826 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8d6884a733cc034195ae361b93bcb4bfe6075939a4fd2543b66f77280c25c3a3\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:36.201069 kubelet[2720]: E0813 01:27:36.200871 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8d6884a733cc034195ae361b93bcb4bfe6075939a4fd2543b66f77280c25c3a3\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:27:36.201069 kubelet[2720]: E0813 01:27:36.200890 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8d6884a733cc034195ae361b93bcb4bfe6075939a4fd2543b66f77280c25c3a3\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:27:36.201069 kubelet[2720]: E0813 01:27:36.200930 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"8d6884a733cc034195ae361b93bcb4bfe6075939a4fd2543b66f77280c25c3a3\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:27:36.415868 kubelet[2720]: I0813 01:27:36.415062 2720 kubelet.go:2405] "Pod admission denied" podUID="67c38c8d-edf4-4dee-994c-730a00792c7a" pod="tigera-operator/tigera-operator-747864d56d-w2hk9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:36.510253 kubelet[2720]: I0813 01:27:36.510200 2720 kubelet.go:2405] "Pod admission denied" podUID="4ccf4b38-7036-4f2e-8705-9f69d6cfb48f" pod="tigera-operator/tigera-operator-747864d56d-wqljw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:36.611645 kubelet[2720]: I0813 01:27:36.611597 2720 kubelet.go:2405] "Pod admission denied" podUID="26a7c5fe-e6cf-4e42-8d52-837c9dec10cb" pod="tigera-operator/tigera-operator-747864d56d-fhqrf" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:36.710992 kubelet[2720]: I0813 01:27:36.710236 2720 kubelet.go:2405] "Pod admission denied" podUID="be0c9204-8354-489a-90cb-85643545ac89" pod="tigera-operator/tigera-operator-747864d56d-rfl68" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:36.812994 kubelet[2720]: I0813 01:27:36.812960 2720 kubelet.go:2405] "Pod admission denied" podUID="d7de4319-32da-4086-b84d-61804fc0c565" pod="tigera-operator/tigera-operator-747864d56d-zxc9g" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:36.911160 kubelet[2720]: I0813 01:27:36.911111 2720 kubelet.go:2405] "Pod admission denied" podUID="d5244b60-8c40-4aee-9b2d-9b3f8e40c5fe" pod="tigera-operator/tigera-operator-747864d56d-28k55" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:37.112869 kubelet[2720]: I0813 01:27:37.112830 2720 kubelet.go:2405] "Pod admission denied" podUID="d2aa1394-1511-4d53-834c-a0a2615a9571" pod="tigera-operator/tigera-operator-747864d56d-ztrbs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:37.210601 kubelet[2720]: I0813 01:27:37.210551 2720 kubelet.go:2405] "Pod admission denied" podUID="31514a47-fa7e-4c79-b2b0-93c93aa9ef1d" pod="tigera-operator/tigera-operator-747864d56d-9wmqn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:37.308605 kubelet[2720]: I0813 01:27:37.308550 2720 kubelet.go:2405] "Pod admission denied" podUID="3a8873b5-02e2-4814-8b82-42af883cab7e" pod="tigera-operator/tigera-operator-747864d56d-7wx5v" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:37.409801 kubelet[2720]: I0813 01:27:37.409671 2720 kubelet.go:2405] "Pod admission denied" podUID="9013d9fd-2258-4547-8b25-d4ded201d6e8" pod="tigera-operator/tigera-operator-747864d56d-krcjc" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:37.507734 kubelet[2720]: I0813 01:27:37.507675 2720 kubelet.go:2405] "Pod admission denied" podUID="ecbb5056-f3f9-4378-8fd1-0e40338502c9" pod="tigera-operator/tigera-operator-747864d56d-dwtrv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:37.721798 kubelet[2720]: I0813 01:27:37.721764 2720 kubelet.go:2405] "Pod admission denied" podUID="e001caea-1e5b-41e9-b755-18e65adf48a2" pod="tigera-operator/tigera-operator-747864d56d-gfsgt" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:37.812916 kubelet[2720]: I0813 01:27:37.812880 2720 kubelet.go:2405] "Pod admission denied" podUID="a388887f-bb54-4b31-89d6-2d470126e5a0" pod="tigera-operator/tigera-operator-747864d56d-rtz9x" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:37.922731 kubelet[2720]: I0813 01:27:37.921982 2720 kubelet.go:2405] "Pod admission denied" podUID="967f4a27-e920-4217-b932-da63b41db31a" pod="tigera-operator/tigera-operator-747864d56d-hcnq7" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:38.010518 kubelet[2720]: I0813 01:27:38.010368 2720 kubelet.go:2405] "Pod admission denied" podUID="841d17c0-475b-4c54-a9a4-feb32e854631" pod="tigera-operator/tigera-operator-747864d56d-klq22" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:38.059734 kubelet[2720]: I0813 01:27:38.059395 2720 kubelet.go:2405] "Pod admission denied" podUID="c5d44a24-de8b-4ffc-b52f-df4307ca8de0" pod="tigera-operator/tigera-operator-747864d56d-wnkjr" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:38.159571 kubelet[2720]: I0813 01:27:38.159519 2720 kubelet.go:2405] "Pod admission denied" podUID="799388f6-fd83-418c-853b-eb7353a10bf2" pod="tigera-operator/tigera-operator-747864d56d-c7f48" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:38.262628 kubelet[2720]: I0813 01:27:38.261872 2720 kubelet.go:2405] "Pod admission denied" podUID="3ebaa826-35f2-4d35-a835-bd95fd80ec8c" pod="tigera-operator/tigera-operator-747864d56d-9jfw4" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:38.322978 kubelet[2720]: I0813 01:27:38.322931 2720 kubelet.go:2405] "Pod admission denied" podUID="ff0d0eca-dcf1-48b7-8688-8796e7893d8c" pod="tigera-operator/tigera-operator-747864d56d-hs6px" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:38.417174 kubelet[2720]: I0813 01:27:38.417131 2720 kubelet.go:2405] "Pod admission denied" podUID="124c8754-fc7d-48ab-9e5e-f288d22b304c" pod="tigera-operator/tigera-operator-747864d56d-d65ng" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:38.613627 kubelet[2720]: I0813 01:27:38.613064 2720 kubelet.go:2405] "Pod admission denied" podUID="e19cc7dc-fd61-4adf-aa26-77cb1fd4cf96" pod="tigera-operator/tigera-operator-747864d56d-j46pk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:38.714417 kubelet[2720]: I0813 01:27:38.714244 2720 kubelet.go:2405] "Pod admission denied" podUID="63351704-1e4a-412a-85a0-0c78f357e856" pod="tigera-operator/tigera-operator-747864d56d-cp42n" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:38.807130 kubelet[2720]: I0813 01:27:38.807084 2720 kubelet.go:2405] "Pod admission denied" podUID="6063bb77-54d8-4b1a-9a54-128850b11df1" pod="tigera-operator/tigera-operator-747864d56d-vd9mx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:38.910857 kubelet[2720]: I0813 01:27:38.910337 2720 kubelet.go:2405] "Pod admission denied" podUID="6a807bdb-5d67-4262-b875-2bda8ca72e0e" pod="tigera-operator/tigera-operator-747864d56d-v48jh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:39.008967 kubelet[2720]: I0813 01:27:39.008918 2720 kubelet.go:2405] "Pod admission denied" podUID="ec9715e9-2eaf-4e57-a6f0-5c5ab14dbc31" pod="tigera-operator/tigera-operator-747864d56d-kbtpj" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:39.108956 kubelet[2720]: I0813 01:27:39.108902 2720 kubelet.go:2405] "Pod admission denied" podUID="e7e779b0-e314-4363-becb-edb35fd2782d" pod="tigera-operator/tigera-operator-747864d56d-jrwhn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:39.210600 kubelet[2720]: I0813 01:27:39.210445 2720 kubelet.go:2405] "Pod admission denied" podUID="9092fb48-31c1-4823-a832-c2786e251f94" pod="tigera-operator/tigera-operator-747864d56d-tmq7k" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:39.310133 kubelet[2720]: I0813 01:27:39.310077 2720 kubelet.go:2405] "Pod admission denied" podUID="c1c339b6-6fa3-4a0c-a5f8-9427ab0be989" pod="tigera-operator/tigera-operator-747864d56d-npdjv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:39.409851 kubelet[2720]: I0813 01:27:39.409800 2720 kubelet.go:2405] "Pod admission denied" podUID="16758d3b-cc3e-439b-9c4a-45bfeecc66ff" pod="tigera-operator/tigera-operator-747864d56d-qgq42" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:39.509497 kubelet[2720]: I0813 01:27:39.509447 2720 kubelet.go:2405] "Pod admission denied" podUID="c90c0c63-d99b-4bb3-833e-6677971a91bf" pod="tigera-operator/tigera-operator-747864d56d-sqcn5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:39.610992 kubelet[2720]: I0813 01:27:39.610945 2720 kubelet.go:2405] "Pod admission denied" podUID="5ad493f3-fdc2-4487-a9c4-113879c81cf0" pod="tigera-operator/tigera-operator-747864d56d-t7glv" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:39.710809 kubelet[2720]: I0813 01:27:39.710767 2720 kubelet.go:2405] "Pod admission denied" podUID="9aa7403a-9507-4875-a1dd-e69975da95c7" pod="tigera-operator/tigera-operator-747864d56d-bcbk9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:39.809033 kubelet[2720]: I0813 01:27:39.808901 2720 kubelet.go:2405] "Pod admission denied" podUID="2897a8d5-cd2c-4989-abcb-10157a145e87" pod="tigera-operator/tigera-operator-747864d56d-62jk5" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:40.013434 kubelet[2720]: I0813 01:27:40.013375 2720 kubelet.go:2405] "Pod admission denied" podUID="b847cc4f-01f6-45d3-8ebe-44383711ab08" pod="tigera-operator/tigera-operator-747864d56d-7q6qq" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:40.076835 kubelet[2720]: E0813 01:27:40.075642 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"calico-node\" with ImagePullBackOff: \"Back-off pulling image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": ErrImagePull: failed to pull and unpack image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount849196455: write /var/lib/containerd/tmpmounts/containerd-mount849196455/usr/bin/calico-node: no space left on device\"" pod="calico-system/calico-node-2p8hj" podUID="4f334296-e8c0-427e-a090-d8fbd3582f26" Aug 13 01:27:40.077296 containerd[1567]: time="2025-08-13T01:27:40.076195572Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:27:40.151756 containerd[1567]: time="2025-08-13T01:27:40.149510229Z" level=error msg="Failed to destroy network for sandbox \"c41b6272612d0dacee9e1df71f1a8fb95f619062e39285f0ec481cbd74f31acd\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:40.151532 systemd[1]: run-netns-cni\x2d66a46317\x2d8352\x2dbc68\x2d6775\x2d72c4eadaff84.mount: Deactivated successfully. Aug 13 01:27:40.153969 containerd[1567]: time="2025-08-13T01:27:40.153905532Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c41b6272612d0dacee9e1df71f1a8fb95f619062e39285f0ec481cbd74f31acd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:40.154268 kubelet[2720]: E0813 01:27:40.154229 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c41b6272612d0dacee9e1df71f1a8fb95f619062e39285f0ec481cbd74f31acd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:40.154792 kubelet[2720]: E0813 01:27:40.154538 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c41b6272612d0dacee9e1df71f1a8fb95f619062e39285f0ec481cbd74f31acd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:27:40.154792 kubelet[2720]: E0813 01:27:40.154564 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c41b6272612d0dacee9e1df71f1a8fb95f619062e39285f0ec481cbd74f31acd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:27:40.154792 kubelet[2720]: E0813 01:27:40.154605 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c41b6272612d0dacee9e1df71f1a8fb95f619062e39285f0ec481cbd74f31acd\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:27:40.176403 kubelet[2720]: I0813 01:27:40.176356 2720 kubelet.go:2405] "Pod admission denied" podUID="58e61ba8-ba4e-4e7e-bcbd-f46a9d34450b" pod="tigera-operator/tigera-operator-747864d56d-jcnt9" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:40.317733 kubelet[2720]: I0813 01:27:40.317670 2720 kubelet.go:2405] "Pod admission denied" podUID="ee5525c9-6ce3-4d13-8db5-3b61c475061b" pod="tigera-operator/tigera-operator-747864d56d-vb5ck" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:40.385813 kubelet[2720]: I0813 01:27:40.385147 2720 kubelet.go:2405] "Pod admission denied" podUID="d9a11bed-32f8-4b9b-98d0-d88b16a80ed1" pod="tigera-operator/tigera-operator-747864d56d-cfhrn" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:40.460637 kubelet[2720]: I0813 01:27:40.460607 2720 kubelet.go:2405] "Pod admission denied" podUID="379acd0e-5dec-48b5-a581-7cd033bd8e13" pod="tigera-operator/tigera-operator-747864d56d-4zmqs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:40.563940 kubelet[2720]: I0813 01:27:40.563850 2720 kubelet.go:2405] "Pod admission denied" podUID="3ab6b804-0c22-4769-b2c6-c5af5f734205" pod="tigera-operator/tigera-operator-747864d56d-j2gkh" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:40.663350 kubelet[2720]: I0813 01:27:40.663228 2720 kubelet.go:2405] "Pod admission denied" podUID="5b8e11a3-5ccf-442f-a9ab-d98813b308a1" pod="tigera-operator/tigera-operator-747864d56d-bj9bw" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:40.735350 kubelet[2720]: I0813 01:27:40.735286 2720 kubelet.go:2405] "Pod admission denied" podUID="bfb42523-495a-4063-95d8-12a736406bd2" pod="tigera-operator/tigera-operator-747864d56d-zktbx" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:40.864962 kubelet[2720]: I0813 01:27:40.864559 2720 kubelet.go:2405] "Pod admission denied" podUID="20f38da9-a5b2-4287-b127-d3bc7fae47d5" pod="tigera-operator/tigera-operator-747864d56d-h7gxk" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:40.975344 kubelet[2720]: I0813 01:27:40.975275 2720 kubelet.go:2405] "Pod admission denied" podUID="3577e5d1-db71-481a-9b22-09a0e58c5bac" pod="tigera-operator/tigera-operator-747864d56d-wv4ph" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:41.060936 kubelet[2720]: I0813 01:27:41.060888 2720 kubelet.go:2405] "Pod admission denied" podUID="50278a06-e6bb-43d7-b3a6-294077978682" pod="tigera-operator/tigera-operator-747864d56d-648fs" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:41.171733 kubelet[2720]: I0813 01:27:41.171268 2720 kubelet.go:2405] "Pod admission denied" podUID="61247909-f430-4e59-a7a3-f582bf9dea31" pod="tigera-operator/tigera-operator-747864d56d-xlj4h" reason="Evicted" message="The node had condition: [DiskPressure]. " Aug 13 01:27:41.808466 systemd[1]: Started sshd@7-172.234.25.215:22-147.75.109.163:57888.service - OpenSSH per-connection server daemon (147.75.109.163:57888). Aug 13 01:27:42.143439 sshd[4735]: Accepted publickey for core from 147.75.109.163 port 57888 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:27:42.145164 sshd-session[4735]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:27:42.151102 systemd-logind[1543]: New session 8 of user core. Aug 13 01:27:42.156847 systemd[1]: Started session-8.scope - Session 8 of User core. Aug 13 01:27:42.457330 sshd[4737]: Connection closed by 147.75.109.163 port 57888 Aug 13 01:27:42.457929 sshd-session[4735]: pam_unix(sshd:session): session closed for user core Aug 13 01:27:42.462778 systemd[1]: sshd@7-172.234.25.215:22-147.75.109.163:57888.service: Deactivated successfully. Aug 13 01:27:42.465794 systemd[1]: session-8.scope: Deactivated successfully. Aug 13 01:27:42.467316 systemd-logind[1543]: Session 8 logged out. Waiting for processes to exit. Aug 13 01:27:42.469113 systemd-logind[1543]: Removed session 8. Aug 13 01:27:43.074009 kubelet[2720]: E0813 01:27:43.073965 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:27:44.073780 kubelet[2720]: E0813 01:27:44.073368 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:27:44.074589 containerd[1567]: time="2025-08-13T01:27:44.074557725Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:27:44.123351 containerd[1567]: time="2025-08-13T01:27:44.123308498Z" level=error msg="Failed to destroy network for sandbox \"86fa6069bd63652371f9b520084ce1cf09c42c41b22aafad73451a40734a7abf\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:44.125246 systemd[1]: run-netns-cni\x2dbcd29a06\x2d0df9\x2d882c\x2d557d\x2d8bdbd87c5a1f.mount: Deactivated successfully. Aug 13 01:27:44.126113 containerd[1567]: time="2025-08-13T01:27:44.126081419Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"86fa6069bd63652371f9b520084ce1cf09c42c41b22aafad73451a40734a7abf\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:44.126471 kubelet[2720]: E0813 01:27:44.126440 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"86fa6069bd63652371f9b520084ce1cf09c42c41b22aafad73451a40734a7abf\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:44.127083 kubelet[2720]: E0813 01:27:44.126784 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"86fa6069bd63652371f9b520084ce1cf09c42c41b22aafad73451a40734a7abf\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:27:44.127083 kubelet[2720]: E0813 01:27:44.126810 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"86fa6069bd63652371f9b520084ce1cf09c42c41b22aafad73451a40734a7abf\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:27:44.127083 kubelet[2720]: E0813 01:27:44.126883 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"86fa6069bd63652371f9b520084ce1cf09c42c41b22aafad73451a40734a7abf\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:27:47.521947 systemd[1]: Started sshd@8-172.234.25.215:22-147.75.109.163:57896.service - OpenSSH per-connection server daemon (147.75.109.163:57896). Aug 13 01:27:47.861833 sshd[4776]: Accepted publickey for core from 147.75.109.163 port 57896 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:27:47.863177 sshd-session[4776]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:27:47.870575 systemd-logind[1543]: New session 9 of user core. Aug 13 01:27:47.874831 systemd[1]: Started session-9.scope - Session 9 of User core. Aug 13 01:27:48.076749 containerd[1567]: time="2025-08-13T01:27:48.074314812Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:27:48.141509 containerd[1567]: time="2025-08-13T01:27:48.141388883Z" level=error msg="Failed to destroy network for sandbox \"1838e8cbf5fee936fc0c96858f8736801c866da1b620c08eadb7ba58ea08604c\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:48.143461 systemd[1]: run-netns-cni\x2de7f87499\x2d3c4f\x2dc1c3\x2d19bc\x2dbd85c8cbd23d.mount: Deactivated successfully. Aug 13 01:27:48.144966 containerd[1567]: time="2025-08-13T01:27:48.144838011Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"1838e8cbf5fee936fc0c96858f8736801c866da1b620c08eadb7ba58ea08604c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:48.145261 kubelet[2720]: E0813 01:27:48.145231 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1838e8cbf5fee936fc0c96858f8736801c866da1b620c08eadb7ba58ea08604c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:48.146007 kubelet[2720]: E0813 01:27:48.145623 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1838e8cbf5fee936fc0c96858f8736801c866da1b620c08eadb7ba58ea08604c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:27:48.146007 kubelet[2720]: E0813 01:27:48.145672 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1838e8cbf5fee936fc0c96858f8736801c866da1b620c08eadb7ba58ea08604c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:27:48.146007 kubelet[2720]: E0813 01:27:48.145763 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"1838e8cbf5fee936fc0c96858f8736801c866da1b620c08eadb7ba58ea08604c\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:27:48.180054 sshd[4778]: Connection closed by 147.75.109.163 port 57896 Aug 13 01:27:48.180939 sshd-session[4776]: pam_unix(sshd:session): session closed for user core Aug 13 01:27:48.184910 systemd[1]: sshd@8-172.234.25.215:22-147.75.109.163:57896.service: Deactivated successfully. Aug 13 01:27:48.185150 systemd-logind[1543]: Session 9 logged out. Waiting for processes to exit. Aug 13 01:27:48.187305 systemd[1]: session-9.scope: Deactivated successfully. Aug 13 01:27:48.189437 systemd-logind[1543]: Removed session 9. Aug 13 01:27:49.074005 kubelet[2720]: E0813 01:27:49.073972 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:27:49.075543 containerd[1567]: time="2025-08-13T01:27:49.075301631Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:27:49.120009 containerd[1567]: time="2025-08-13T01:27:49.119960990Z" level=error msg="Failed to destroy network for sandbox \"8e1b5ecbc664fe4617372765cc32969ae8353c8f8d36bf5ec2f78d95a20bed8c\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:49.122514 containerd[1567]: time="2025-08-13T01:27:49.122415242Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"8e1b5ecbc664fe4617372765cc32969ae8353c8f8d36bf5ec2f78d95a20bed8c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:49.122662 kubelet[2720]: E0813 01:27:49.122618 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8e1b5ecbc664fe4617372765cc32969ae8353c8f8d36bf5ec2f78d95a20bed8c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:49.122736 kubelet[2720]: E0813 01:27:49.122683 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8e1b5ecbc664fe4617372765cc32969ae8353c8f8d36bf5ec2f78d95a20bed8c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:27:49.122736 kubelet[2720]: E0813 01:27:49.122705 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8e1b5ecbc664fe4617372765cc32969ae8353c8f8d36bf5ec2f78d95a20bed8c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:27:49.122816 kubelet[2720]: E0813 01:27:49.122789 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"8e1b5ecbc664fe4617372765cc32969ae8353c8f8d36bf5ec2f78d95a20bed8c\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:27:49.122943 systemd[1]: run-netns-cni\x2d9dce7192\x2d44ea\x2da954\x2d5de7\x2da859ce1edb51.mount: Deactivated successfully. Aug 13 01:27:51.074468 kubelet[2720]: E0813 01:27:51.074417 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"calico-node\" with ImagePullBackOff: \"Back-off pulling image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": ErrImagePull: failed to pull and unpack image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount849196455: write /var/lib/containerd/tmpmounts/containerd-mount849196455/usr/bin/calico-node: no space left on device\"" pod="calico-system/calico-node-2p8hj" podUID="4f334296-e8c0-427e-a090-d8fbd3582f26" Aug 13 01:27:52.079590 containerd[1567]: time="2025-08-13T01:27:52.079527762Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:27:52.134433 containerd[1567]: time="2025-08-13T01:27:52.134388795Z" level=error msg="Failed to destroy network for sandbox \"0d2ba00df2b15843d42be5d4725fc98d6d79ec3eb3c8c745ec34842042349211\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:52.136917 containerd[1567]: time="2025-08-13T01:27:52.136863616Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"0d2ba00df2b15843d42be5d4725fc98d6d79ec3eb3c8c745ec34842042349211\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:52.137260 kubelet[2720]: E0813 01:27:52.137215 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0d2ba00df2b15843d42be5d4725fc98d6d79ec3eb3c8c745ec34842042349211\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:52.138191 kubelet[2720]: E0813 01:27:52.137791 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0d2ba00df2b15843d42be5d4725fc98d6d79ec3eb3c8c745ec34842042349211\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:27:52.138191 kubelet[2720]: E0813 01:27:52.137817 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0d2ba00df2b15843d42be5d4725fc98d6d79ec3eb3c8c745ec34842042349211\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:27:52.138191 kubelet[2720]: E0813 01:27:52.137867 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"0d2ba00df2b15843d42be5d4725fc98d6d79ec3eb3c8c745ec34842042349211\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:27:52.137973 systemd[1]: run-netns-cni\x2d58cdb400\x2d41ce\x2d0afa\x2d0aa8\x2d2f2707e5f27a.mount: Deactivated successfully. Aug 13 01:27:53.243666 systemd[1]: Started sshd@9-172.234.25.215:22-147.75.109.163:57048.service - OpenSSH per-connection server daemon (147.75.109.163:57048). Aug 13 01:27:53.584378 sshd[4871]: Accepted publickey for core from 147.75.109.163 port 57048 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:27:53.586152 sshd-session[4871]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:27:53.590596 systemd-logind[1543]: New session 10 of user core. Aug 13 01:27:53.598861 systemd[1]: Started session-10.scope - Session 10 of User core. Aug 13 01:27:53.903233 sshd[4873]: Connection closed by 147.75.109.163 port 57048 Aug 13 01:27:53.904069 sshd-session[4871]: pam_unix(sshd:session): session closed for user core Aug 13 01:27:53.909273 systemd-logind[1543]: Session 10 logged out. Waiting for processes to exit. Aug 13 01:27:53.910092 systemd[1]: sshd@9-172.234.25.215:22-147.75.109.163:57048.service: Deactivated successfully. Aug 13 01:27:53.913599 systemd[1]: session-10.scope: Deactivated successfully. Aug 13 01:27:53.915672 systemd-logind[1543]: Removed session 10. Aug 13 01:27:53.970302 systemd[1]: Started sshd@10-172.234.25.215:22-147.75.109.163:57052.service - OpenSSH per-connection server daemon (147.75.109.163:57052). Aug 13 01:27:54.319555 sshd[4886]: Accepted publickey for core from 147.75.109.163 port 57052 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:27:54.321274 sshd-session[4886]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:27:54.326935 systemd-logind[1543]: New session 11 of user core. Aug 13 01:27:54.330835 systemd[1]: Started session-11.scope - Session 11 of User core. Aug 13 01:27:54.653438 sshd[4888]: Connection closed by 147.75.109.163 port 57052 Aug 13 01:27:54.654120 sshd-session[4886]: pam_unix(sshd:session): session closed for user core Aug 13 01:27:54.660962 systemd-logind[1543]: Session 11 logged out. Waiting for processes to exit. Aug 13 01:27:54.661988 systemd[1]: sshd@10-172.234.25.215:22-147.75.109.163:57052.service: Deactivated successfully. Aug 13 01:27:54.665163 systemd[1]: session-11.scope: Deactivated successfully. Aug 13 01:27:54.668463 systemd-logind[1543]: Removed session 11. Aug 13 01:27:54.712645 systemd[1]: Started sshd@11-172.234.25.215:22-147.75.109.163:57058.service - OpenSSH per-connection server daemon (147.75.109.163:57058). Aug 13 01:27:55.054760 sshd[4898]: Accepted publickey for core from 147.75.109.163 port 57058 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:27:55.056169 sshd-session[4898]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:27:55.061490 systemd-logind[1543]: New session 12 of user core. Aug 13 01:27:55.068840 systemd[1]: Started session-12.scope - Session 12 of User core. Aug 13 01:27:55.368200 sshd[4900]: Connection closed by 147.75.109.163 port 57058 Aug 13 01:27:55.368698 sshd-session[4898]: pam_unix(sshd:session): session closed for user core Aug 13 01:27:55.373799 systemd-logind[1543]: Session 12 logged out. Waiting for processes to exit. Aug 13 01:27:55.374853 systemd[1]: sshd@11-172.234.25.215:22-147.75.109.163:57058.service: Deactivated successfully. Aug 13 01:27:55.377456 systemd[1]: session-12.scope: Deactivated successfully. Aug 13 01:27:55.379659 systemd-logind[1543]: Removed session 12. Aug 13 01:27:58.074409 kubelet[2720]: E0813 01:27:58.073962 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:27:58.075568 containerd[1567]: time="2025-08-13T01:27:58.074969215Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:27:58.131795 containerd[1567]: time="2025-08-13T01:27:58.131747966Z" level=error msg="Failed to destroy network for sandbox \"590be7caeec1e44bbc8226e0afb21c349944c8b63ca549c3111dc20724dc9ff4\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:58.133461 systemd[1]: run-netns-cni\x2d71c5dd2c\x2deef0\x2dee67\x2d7938\x2daf41bb13178a.mount: Deactivated successfully. Aug 13 01:27:58.135240 containerd[1567]: time="2025-08-13T01:27:58.135066626Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"590be7caeec1e44bbc8226e0afb21c349944c8b63ca549c3111dc20724dc9ff4\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:58.135324 kubelet[2720]: E0813 01:27:58.135244 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"590be7caeec1e44bbc8226e0afb21c349944c8b63ca549c3111dc20724dc9ff4\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:27:58.135324 kubelet[2720]: E0813 01:27:58.135290 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"590be7caeec1e44bbc8226e0afb21c349944c8b63ca549c3111dc20724dc9ff4\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:27:58.135324 kubelet[2720]: E0813 01:27:58.135311 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"590be7caeec1e44bbc8226e0afb21c349944c8b63ca549c3111dc20724dc9ff4\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:27:58.135447 kubelet[2720]: E0813 01:27:58.135354 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"590be7caeec1e44bbc8226e0afb21c349944c8b63ca549c3111dc20724dc9ff4\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:28:00.436485 systemd[1]: Started sshd@12-172.234.25.215:22-147.75.109.163:35946.service - OpenSSH per-connection server daemon (147.75.109.163:35946). Aug 13 01:28:00.774548 sshd[4941]: Accepted publickey for core from 147.75.109.163 port 35946 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:28:00.776064 sshd-session[4941]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:28:00.781366 systemd-logind[1543]: New session 13 of user core. Aug 13 01:28:00.786850 systemd[1]: Started session-13.scope - Session 13 of User core. Aug 13 01:28:01.072305 sshd[4943]: Connection closed by 147.75.109.163 port 35946 Aug 13 01:28:01.072667 sshd-session[4941]: pam_unix(sshd:session): session closed for user core Aug 13 01:28:01.073734 kubelet[2720]: E0813 01:28:01.073570 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:01.074692 containerd[1567]: time="2025-08-13T01:28:01.074662920Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:28:01.076913 containerd[1567]: time="2025-08-13T01:28:01.076588474Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:28:01.080666 systemd[1]: sshd@12-172.234.25.215:22-147.75.109.163:35946.service: Deactivated successfully. Aug 13 01:28:01.094313 systemd[1]: session-13.scope: Deactivated successfully. Aug 13 01:28:01.096770 systemd-logind[1543]: Session 13 logged out. Waiting for processes to exit. Aug 13 01:28:01.099299 systemd-logind[1543]: Removed session 13. Aug 13 01:28:01.136201 systemd[1]: Started sshd@13-172.234.25.215:22-147.75.109.163:35950.service - OpenSSH per-connection server daemon (147.75.109.163:35950). Aug 13 01:28:01.182831 containerd[1567]: time="2025-08-13T01:28:01.182796698Z" level=error msg="Failed to destroy network for sandbox \"3cd5d2f45bb9ed9d94646626142b76cec4a3dde5eb0d7742c7095e9753549b2c\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:01.186380 systemd[1]: run-netns-cni\x2d45b31dc0\x2db449\x2dcd44\x2d6b04\x2db31647660b1b.mount: Deactivated successfully. Aug 13 01:28:01.189680 containerd[1567]: time="2025-08-13T01:28:01.189225331Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"3cd5d2f45bb9ed9d94646626142b76cec4a3dde5eb0d7742c7095e9753549b2c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:01.190173 kubelet[2720]: E0813 01:28:01.189987 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3cd5d2f45bb9ed9d94646626142b76cec4a3dde5eb0d7742c7095e9753549b2c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:01.190173 kubelet[2720]: E0813 01:28:01.190056 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3cd5d2f45bb9ed9d94646626142b76cec4a3dde5eb0d7742c7095e9753549b2c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:01.190173 kubelet[2720]: E0813 01:28:01.190084 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3cd5d2f45bb9ed9d94646626142b76cec4a3dde5eb0d7742c7095e9753549b2c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:01.190173 kubelet[2720]: E0813 01:28:01.190148 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"3cd5d2f45bb9ed9d94646626142b76cec4a3dde5eb0d7742c7095e9753549b2c\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:28:01.206706 containerd[1567]: time="2025-08-13T01:28:01.206676880Z" level=error msg="Failed to destroy network for sandbox \"3691b7a1716feef8ff36c8dedcd2b5caf83025e151c0d44374859d99a6905071\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:01.210165 systemd[1]: run-netns-cni\x2d2ff15ea3\x2d4639\x2dd1c9\x2d42e9\x2dda829032288d.mount: Deactivated successfully. Aug 13 01:28:01.211771 containerd[1567]: time="2025-08-13T01:28:01.211675076Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"3691b7a1716feef8ff36c8dedcd2b5caf83025e151c0d44374859d99a6905071\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:01.212046 kubelet[2720]: E0813 01:28:01.212004 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3691b7a1716feef8ff36c8dedcd2b5caf83025e151c0d44374859d99a6905071\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:01.212151 kubelet[2720]: E0813 01:28:01.212135 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3691b7a1716feef8ff36c8dedcd2b5caf83025e151c0d44374859d99a6905071\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:28:01.212279 kubelet[2720]: E0813 01:28:01.212194 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3691b7a1716feef8ff36c8dedcd2b5caf83025e151c0d44374859d99a6905071\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:28:01.212408 kubelet[2720]: E0813 01:28:01.212369 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"3691b7a1716feef8ff36c8dedcd2b5caf83025e151c0d44374859d99a6905071\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:28:01.511275 sshd[4997]: Accepted publickey for core from 147.75.109.163 port 35950 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:28:01.512934 sshd-session[4997]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:28:01.517609 systemd-logind[1543]: New session 14 of user core. Aug 13 01:28:01.525889 systemd[1]: Started session-14.scope - Session 14 of User core. Aug 13 01:28:01.914395 sshd[5013]: Connection closed by 147.75.109.163 port 35950 Aug 13 01:28:01.915231 sshd-session[4997]: pam_unix(sshd:session): session closed for user core Aug 13 01:28:01.918965 systemd-logind[1543]: Session 14 logged out. Waiting for processes to exit. Aug 13 01:28:01.919667 systemd[1]: sshd@13-172.234.25.215:22-147.75.109.163:35950.service: Deactivated successfully. Aug 13 01:28:01.921941 systemd[1]: session-14.scope: Deactivated successfully. Aug 13 01:28:01.923942 systemd-logind[1543]: Removed session 14. Aug 13 01:28:01.975035 systemd[1]: Started sshd@14-172.234.25.215:22-147.75.109.163:35962.service - OpenSSH per-connection server daemon (147.75.109.163:35962). Aug 13 01:28:02.316781 sshd[5023]: Accepted publickey for core from 147.75.109.163 port 35962 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:28:02.318410 sshd-session[5023]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:28:02.323775 systemd-logind[1543]: New session 15 of user core. Aug 13 01:28:02.328832 systemd[1]: Started session-15.scope - Session 15 of User core. Aug 13 01:28:03.049669 sshd[5025]: Connection closed by 147.75.109.163 port 35962 Aug 13 01:28:03.050293 sshd-session[5023]: pam_unix(sshd:session): session closed for user core Aug 13 01:28:03.055203 systemd-logind[1543]: Session 15 logged out. Waiting for processes to exit. Aug 13 01:28:03.055752 systemd[1]: sshd@14-172.234.25.215:22-147.75.109.163:35962.service: Deactivated successfully. Aug 13 01:28:03.058455 systemd[1]: session-15.scope: Deactivated successfully. Aug 13 01:28:03.061604 systemd-logind[1543]: Removed session 15. Aug 13 01:28:03.112123 systemd[1]: Started sshd@15-172.234.25.215:22-147.75.109.163:35974.service - OpenSSH per-connection server daemon (147.75.109.163:35974). Aug 13 01:28:03.445696 sshd[5042]: Accepted publickey for core from 147.75.109.163 port 35974 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:28:03.446941 sshd-session[5042]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:28:03.452491 systemd-logind[1543]: New session 16 of user core. Aug 13 01:28:03.459859 systemd[1]: Started session-16.scope - Session 16 of User core. Aug 13 01:28:03.846649 sshd[5044]: Connection closed by 147.75.109.163 port 35974 Aug 13 01:28:03.847321 sshd-session[5042]: pam_unix(sshd:session): session closed for user core Aug 13 01:28:03.851430 systemd-logind[1543]: Session 16 logged out. Waiting for processes to exit. Aug 13 01:28:03.852173 systemd[1]: sshd@15-172.234.25.215:22-147.75.109.163:35974.service: Deactivated successfully. Aug 13 01:28:03.854562 systemd[1]: session-16.scope: Deactivated successfully. Aug 13 01:28:03.856790 systemd-logind[1543]: Removed session 16. Aug 13 01:28:03.906570 systemd[1]: Started sshd@16-172.234.25.215:22-147.75.109.163:35976.service - OpenSSH per-connection server daemon (147.75.109.163:35976). Aug 13 01:28:04.074251 kubelet[2720]: E0813 01:28:04.074211 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:04.240900 sshd[5054]: Accepted publickey for core from 147.75.109.163 port 35976 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:28:04.242159 sshd-session[5054]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:28:04.247379 systemd-logind[1543]: New session 17 of user core. Aug 13 01:28:04.253853 systemd[1]: Started session-17.scope - Session 17 of User core. Aug 13 01:28:04.545988 sshd[5058]: Connection closed by 147.75.109.163 port 35976 Aug 13 01:28:04.546798 sshd-session[5054]: pam_unix(sshd:session): session closed for user core Aug 13 01:28:04.551570 systemd-logind[1543]: Session 17 logged out. Waiting for processes to exit. Aug 13 01:28:04.551905 systemd[1]: sshd@16-172.234.25.215:22-147.75.109.163:35976.service: Deactivated successfully. Aug 13 01:28:04.554013 systemd[1]: session-17.scope: Deactivated successfully. Aug 13 01:28:04.556281 systemd-logind[1543]: Removed session 17. Aug 13 01:28:05.074394 kubelet[2720]: E0813 01:28:05.074359 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"calico-node\" with ImagePullBackOff: \"Back-off pulling image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": ErrImagePull: failed to pull and unpack image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount849196455: write /var/lib/containerd/tmpmounts/containerd-mount849196455/usr/bin/calico-node: no space left on device\"" pod="calico-system/calico-node-2p8hj" podUID="4f334296-e8c0-427e-a090-d8fbd3582f26" Aug 13 01:28:05.075249 containerd[1567]: time="2025-08-13T01:28:05.074382874Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:28:05.141189 containerd[1567]: time="2025-08-13T01:28:05.141112229Z" level=error msg="Failed to destroy network for sandbox \"88b0747a0d308f28d7fa4b14ab2afdc557886937ad207a298cbcafc8cd23584d\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:05.144662 containerd[1567]: time="2025-08-13T01:28:05.144408811Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"88b0747a0d308f28d7fa4b14ab2afdc557886937ad207a298cbcafc8cd23584d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:05.145493 systemd[1]: run-netns-cni\x2db21c0fbd\x2d0ec8\x2dfd6c\x2d8157\x2d82c45fb75b6e.mount: Deactivated successfully. Aug 13 01:28:05.146529 kubelet[2720]: E0813 01:28:05.145539 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"88b0747a0d308f28d7fa4b14ab2afdc557886937ad207a298cbcafc8cd23584d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:05.146529 kubelet[2720]: E0813 01:28:05.145581 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"88b0747a0d308f28d7fa4b14ab2afdc557886937ad207a298cbcafc8cd23584d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:28:05.146529 kubelet[2720]: E0813 01:28:05.145610 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"88b0747a0d308f28d7fa4b14ab2afdc557886937ad207a298cbcafc8cd23584d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:28:05.146529 kubelet[2720]: E0813 01:28:05.145647 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"88b0747a0d308f28d7fa4b14ab2afdc557886937ad207a298cbcafc8cd23584d\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:28:08.074749 kubelet[2720]: E0813 01:28:08.073861 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:09.606973 systemd[1]: Started sshd@17-172.234.25.215:22-147.75.109.163:33252.service - OpenSSH per-connection server daemon (147.75.109.163:33252). Aug 13 01:28:09.935884 sshd[5100]: Accepted publickey for core from 147.75.109.163 port 33252 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:28:09.937426 sshd-session[5100]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:28:09.942126 systemd-logind[1543]: New session 18 of user core. Aug 13 01:28:09.946856 systemd[1]: Started session-18.scope - Session 18 of User core. Aug 13 01:28:10.235336 sshd[5104]: Connection closed by 147.75.109.163 port 33252 Aug 13 01:28:10.235884 sshd-session[5100]: pam_unix(sshd:session): session closed for user core Aug 13 01:28:10.239931 systemd-logind[1543]: Session 18 logged out. Waiting for processes to exit. Aug 13 01:28:10.240686 systemd[1]: sshd@17-172.234.25.215:22-147.75.109.163:33252.service: Deactivated successfully. Aug 13 01:28:10.243194 systemd[1]: session-18.scope: Deactivated successfully. Aug 13 01:28:10.245344 systemd-logind[1543]: Removed session 18. Aug 13 01:28:11.074014 kubelet[2720]: E0813 01:28:11.073974 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:11.074892 containerd[1567]: time="2025-08-13T01:28:11.074676654Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:28:11.128638 containerd[1567]: time="2025-08-13T01:28:11.128596054Z" level=error msg="Failed to destroy network for sandbox \"f5f3b53086cd4c7c47908f272433d5c38cf928a404d55d8cca6b55ebf2660f8c\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:11.131364 containerd[1567]: time="2025-08-13T01:28:11.131246117Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5f3b53086cd4c7c47908f272433d5c38cf928a404d55d8cca6b55ebf2660f8c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:11.131592 kubelet[2720]: E0813 01:28:11.131543 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5f3b53086cd4c7c47908f272433d5c38cf928a404d55d8cca6b55ebf2660f8c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:11.131698 kubelet[2720]: E0813 01:28:11.131680 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5f3b53086cd4c7c47908f272433d5c38cf928a404d55d8cca6b55ebf2660f8c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:28:11.132675 kubelet[2720]: E0813 01:28:11.131804 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5f3b53086cd4c7c47908f272433d5c38cf928a404d55d8cca6b55ebf2660f8c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:28:11.132675 kubelet[2720]: E0813 01:28:11.131864 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"f5f3b53086cd4c7c47908f272433d5c38cf928a404d55d8cca6b55ebf2660f8c\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:28:11.132606 systemd[1]: run-netns-cni\x2d144192ba\x2d26cd\x2d6008\x2d9fd0\x2d851deec69a46.mount: Deactivated successfully. Aug 13 01:28:14.074480 kubelet[2720]: E0813 01:28:14.073928 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:15.074605 containerd[1567]: time="2025-08-13T01:28:15.074546644Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:28:15.140574 containerd[1567]: time="2025-08-13T01:28:15.140533079Z" level=error msg="Failed to destroy network for sandbox \"c6ae31a9ca430aab295b2007f09e6e24b7487aaa1fec427d1ec098076cff6948\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:15.143029 containerd[1567]: time="2025-08-13T01:28:15.142984214Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c6ae31a9ca430aab295b2007f09e6e24b7487aaa1fec427d1ec098076cff6948\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:15.143902 kubelet[2720]: E0813 01:28:15.143186 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c6ae31a9ca430aab295b2007f09e6e24b7487aaa1fec427d1ec098076cff6948\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:15.143902 kubelet[2720]: E0813 01:28:15.143232 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c6ae31a9ca430aab295b2007f09e6e24b7487aaa1fec427d1ec098076cff6948\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:15.143902 kubelet[2720]: E0813 01:28:15.143252 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c6ae31a9ca430aab295b2007f09e6e24b7487aaa1fec427d1ec098076cff6948\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:15.143902 kubelet[2720]: E0813 01:28:15.143298 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c6ae31a9ca430aab295b2007f09e6e24b7487aaa1fec427d1ec098076cff6948\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:28:15.144516 systemd[1]: run-netns-cni\x2d8f8b677e\x2d8e26\x2dab05\x2d18fe\x2d82e55603b80b.mount: Deactivated successfully. Aug 13 01:28:15.302920 systemd[1]: Started sshd@18-172.234.25.215:22-147.75.109.163:33254.service - OpenSSH per-connection server daemon (147.75.109.163:33254). Aug 13 01:28:15.643473 sshd[5168]: Accepted publickey for core from 147.75.109.163 port 33254 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:28:15.646112 sshd-session[5168]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:28:15.654271 systemd-logind[1543]: New session 19 of user core. Aug 13 01:28:15.659914 systemd[1]: Started session-19.scope - Session 19 of User core. Aug 13 01:28:15.965290 sshd[5170]: Connection closed by 147.75.109.163 port 33254 Aug 13 01:28:15.965953 sshd-session[5168]: pam_unix(sshd:session): session closed for user core Aug 13 01:28:15.971650 systemd[1]: sshd@18-172.234.25.215:22-147.75.109.163:33254.service: Deactivated successfully. Aug 13 01:28:15.973984 systemd[1]: session-19.scope: Deactivated successfully. Aug 13 01:28:15.974932 systemd-logind[1543]: Session 19 logged out. Waiting for processes to exit. Aug 13 01:28:15.976426 systemd-logind[1543]: Removed session 19. Aug 13 01:28:16.074468 kubelet[2720]: E0813 01:28:16.074290 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:16.076402 containerd[1567]: time="2025-08-13T01:28:16.075994827Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:28:16.076956 containerd[1567]: time="2025-08-13T01:28:16.076936854Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:28:16.077779 kubelet[2720]: E0813 01:28:16.077600 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"calico-node\" with ImagePullBackOff: \"Back-off pulling image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": ErrImagePull: failed to pull and unpack image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount849196455: write /var/lib/containerd/tmpmounts/containerd-mount849196455/usr/bin/calico-node: no space left on device\"" pod="calico-system/calico-node-2p8hj" podUID="4f334296-e8c0-427e-a090-d8fbd3582f26" Aug 13 01:28:16.171357 containerd[1567]: time="2025-08-13T01:28:16.171299660Z" level=error msg="Failed to destroy network for sandbox \"a0b7df0d9224785c3537e9ceedc932ee5ae784b65b63f2ad20246331276a665d\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:16.173768 containerd[1567]: time="2025-08-13T01:28:16.173250635Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"a0b7df0d9224785c3537e9ceedc932ee5ae784b65b63f2ad20246331276a665d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:16.174315 systemd[1]: run-netns-cni\x2deb10a2f9\x2dddf9\x2da9d2\x2d02bf\x2d23b95ca79390.mount: Deactivated successfully. Aug 13 01:28:16.175484 kubelet[2720]: E0813 01:28:16.175408 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a0b7df0d9224785c3537e9ceedc932ee5ae784b65b63f2ad20246331276a665d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:16.175484 kubelet[2720]: E0813 01:28:16.175458 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a0b7df0d9224785c3537e9ceedc932ee5ae784b65b63f2ad20246331276a665d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:28:16.175484 kubelet[2720]: E0813 01:28:16.175480 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a0b7df0d9224785c3537e9ceedc932ee5ae784b65b63f2ad20246331276a665d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:28:16.177784 kubelet[2720]: E0813 01:28:16.175529 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"a0b7df0d9224785c3537e9ceedc932ee5ae784b65b63f2ad20246331276a665d\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:28:16.185708 containerd[1567]: time="2025-08-13T01:28:16.185672155Z" level=error msg="Failed to destroy network for sandbox \"295e2e4a4e87421c58d824dc45b3b7cd8ca68542057a6df716d4a93c3e93d2a0\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:16.187637 systemd[1]: run-netns-cni\x2d72a76d58\x2db54f\x2d815b\x2dc341\x2df10607b35f56.mount: Deactivated successfully. Aug 13 01:28:16.188930 containerd[1567]: time="2025-08-13T01:28:16.188888936Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"295e2e4a4e87421c58d824dc45b3b7cd8ca68542057a6df716d4a93c3e93d2a0\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:16.189154 kubelet[2720]: E0813 01:28:16.189116 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"295e2e4a4e87421c58d824dc45b3b7cd8ca68542057a6df716d4a93c3e93d2a0\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:16.189196 kubelet[2720]: E0813 01:28:16.189178 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"295e2e4a4e87421c58d824dc45b3b7cd8ca68542057a6df716d4a93c3e93d2a0\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:28:16.189224 kubelet[2720]: E0813 01:28:16.189197 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"295e2e4a4e87421c58d824dc45b3b7cd8ca68542057a6df716d4a93c3e93d2a0\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:28:16.189613 kubelet[2720]: E0813 01:28:16.189250 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"295e2e4a4e87421c58d824dc45b3b7cd8ca68542057a6df716d4a93c3e93d2a0\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:28:19.073618 kubelet[2720]: E0813 01:28:19.073582 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:21.024785 systemd[1]: Started sshd@19-172.234.25.215:22-147.75.109.163:52780.service - OpenSSH per-connection server daemon (147.75.109.163:52780). Aug 13 01:28:21.356411 sshd[5237]: Accepted publickey for core from 147.75.109.163 port 52780 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:28:21.357938 sshd-session[5237]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:28:21.363363 systemd-logind[1543]: New session 20 of user core. Aug 13 01:28:21.368924 systemd[1]: Started session-20.scope - Session 20 of User core. Aug 13 01:28:21.667449 sshd[5239]: Connection closed by 147.75.109.163 port 52780 Aug 13 01:28:21.668422 sshd-session[5237]: pam_unix(sshd:session): session closed for user core Aug 13 01:28:21.672249 systemd[1]: sshd@19-172.234.25.215:22-147.75.109.163:52780.service: Deactivated successfully. Aug 13 01:28:21.674513 systemd[1]: session-20.scope: Deactivated successfully. Aug 13 01:28:21.676346 systemd-logind[1543]: Session 20 logged out. Waiting for processes to exit. Aug 13 01:28:21.678319 systemd-logind[1543]: Removed session 20. Aug 13 01:28:26.074762 kubelet[2720]: E0813 01:28:26.074480 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:26.076154 containerd[1567]: time="2025-08-13T01:28:26.076089054Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:28:26.139747 containerd[1567]: time="2025-08-13T01:28:26.139690737Z" level=error msg="Failed to destroy network for sandbox \"3df686bdf4fe5abf4683268fe192c9ea04b9cde3c6b328684e60e8b81dabe258\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:26.142475 containerd[1567]: time="2025-08-13T01:28:26.142444002Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"3df686bdf4fe5abf4683268fe192c9ea04b9cde3c6b328684e60e8b81dabe258\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:26.142569 systemd[1]: run-netns-cni\x2de5fed776\x2d48da\x2d6898\x2def7b\x2d9df4e93d962c.mount: Deactivated successfully. Aug 13 01:28:26.142985 kubelet[2720]: E0813 01:28:26.142799 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3df686bdf4fe5abf4683268fe192c9ea04b9cde3c6b328684e60e8b81dabe258\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:26.142985 kubelet[2720]: E0813 01:28:26.142843 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3df686bdf4fe5abf4683268fe192c9ea04b9cde3c6b328684e60e8b81dabe258\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:28:26.142985 kubelet[2720]: E0813 01:28:26.142863 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3df686bdf4fe5abf4683268fe192c9ea04b9cde3c6b328684e60e8b81dabe258\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:28:26.142985 kubelet[2720]: E0813 01:28:26.142901 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"3df686bdf4fe5abf4683268fe192c9ea04b9cde3c6b328684e60e8b81dabe258\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:28:26.727906 systemd[1]: Started sshd@20-172.234.25.215:22-147.75.109.163:52788.service - OpenSSH per-connection server daemon (147.75.109.163:52788). Aug 13 01:28:27.065225 sshd[5278]: Accepted publickey for core from 147.75.109.163 port 52788 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:28:27.066511 sshd-session[5278]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:28:27.071829 systemd-logind[1543]: New session 21 of user core. Aug 13 01:28:27.073489 kubelet[2720]: E0813 01:28:27.073453 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:27.075889 containerd[1567]: time="2025-08-13T01:28:27.075312218Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:28:27.075889 containerd[1567]: time="2025-08-13T01:28:27.075809027Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:28:27.076142 kubelet[2720]: E0813 01:28:27.076099 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"calico-node\" with ImagePullBackOff: \"Back-off pulling image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": ErrImagePull: failed to pull and unpack image \\\"ghcr.io/flatcar/calico/node:v3.30.2\\\": failed to extract layer sha256:0df84c1170997f4f9d64efde556fda4ea5e552df4ec6d03b6cdd975940ab2b14: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount849196455: write /var/lib/containerd/tmpmounts/containerd-mount849196455/usr/bin/calico-node: no space left on device\"" pod="calico-system/calico-node-2p8hj" podUID="4f334296-e8c0-427e-a090-d8fbd3582f26" Aug 13 01:28:27.079846 systemd[1]: Started session-21.scope - Session 21 of User core. Aug 13 01:28:27.172307 containerd[1567]: time="2025-08-13T01:28:27.172255395Z" level=error msg="Failed to destroy network for sandbox \"a6e57f10d70a51a5d0fa5e79600d52fe92db0bd5e19b83ec90fde70c810db273\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:27.176101 systemd[1]: run-netns-cni\x2df24f519f\x2d71df\x2de423\x2d0cb8\x2dae769a4edc69.mount: Deactivated successfully. Aug 13 01:28:27.178886 containerd[1567]: time="2025-08-13T01:28:27.178815400Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"a6e57f10d70a51a5d0fa5e79600d52fe92db0bd5e19b83ec90fde70c810db273\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:27.180212 kubelet[2720]: E0813 01:28:27.179082 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a6e57f10d70a51a5d0fa5e79600d52fe92db0bd5e19b83ec90fde70c810db273\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:27.180212 kubelet[2720]: E0813 01:28:27.179250 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a6e57f10d70a51a5d0fa5e79600d52fe92db0bd5e19b83ec90fde70c810db273\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:28:27.180212 kubelet[2720]: E0813 01:28:27.179271 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a6e57f10d70a51a5d0fa5e79600d52fe92db0bd5e19b83ec90fde70c810db273\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:28:27.180212 kubelet[2720]: E0813 01:28:27.179318 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"a6e57f10d70a51a5d0fa5e79600d52fe92db0bd5e19b83ec90fde70c810db273\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:28:27.192553 containerd[1567]: time="2025-08-13T01:28:27.192516048Z" level=error msg="Failed to destroy network for sandbox \"f5aee2f9230e7fb779c03a5e16362f2ac9d396470f21a5514d8651e4bd1ddc5d\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:27.194454 systemd[1]: run-netns-cni\x2d27fc8e6d\x2d23ce\x2d2e96\x2deefd\x2d06a038ca7d1f.mount: Deactivated successfully. Aug 13 01:28:27.195955 containerd[1567]: time="2025-08-13T01:28:27.195878981Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5aee2f9230e7fb779c03a5e16362f2ac9d396470f21a5514d8651e4bd1ddc5d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:27.196077 kubelet[2720]: E0813 01:28:27.196023 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5aee2f9230e7fb779c03a5e16362f2ac9d396470f21a5514d8651e4bd1ddc5d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:27.196077 kubelet[2720]: E0813 01:28:27.196060 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5aee2f9230e7fb779c03a5e16362f2ac9d396470f21a5514d8651e4bd1ddc5d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:28:27.196783 kubelet[2720]: E0813 01:28:27.196076 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5aee2f9230e7fb779c03a5e16362f2ac9d396470f21a5514d8651e4bd1ddc5d\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:28:27.196783 kubelet[2720]: E0813 01:28:27.196115 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"f5aee2f9230e7fb779c03a5e16362f2ac9d396470f21a5514d8651e4bd1ddc5d\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:28:27.378352 sshd[5288]: Connection closed by 147.75.109.163 port 52788 Aug 13 01:28:27.380016 sshd-session[5278]: pam_unix(sshd:session): session closed for user core Aug 13 01:28:27.383793 systemd[1]: sshd@20-172.234.25.215:22-147.75.109.163:52788.service: Deactivated successfully. Aug 13 01:28:27.385968 systemd[1]: session-21.scope: Deactivated successfully. Aug 13 01:28:27.386772 systemd-logind[1543]: Session 21 logged out. Waiting for processes to exit. Aug 13 01:28:27.388519 systemd-logind[1543]: Removed session 21. Aug 13 01:28:28.074200 containerd[1567]: time="2025-08-13T01:28:28.073922097Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:28:28.132215 containerd[1567]: time="2025-08-13T01:28:28.132175334Z" level=error msg="Failed to destroy network for sandbox \"a65aebe915d24f1c56c89981db4fa1380c353bd5fc80eeea395c7ce77328cb2c\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:28.134580 containerd[1567]: time="2025-08-13T01:28:28.134495499Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"a65aebe915d24f1c56c89981db4fa1380c353bd5fc80eeea395c7ce77328cb2c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:28.134935 kubelet[2720]: E0813 01:28:28.134879 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a65aebe915d24f1c56c89981db4fa1380c353bd5fc80eeea395c7ce77328cb2c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:28.135225 kubelet[2720]: E0813 01:28:28.134951 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a65aebe915d24f1c56c89981db4fa1380c353bd5fc80eeea395c7ce77328cb2c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:28.135225 kubelet[2720]: E0813 01:28:28.134969 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a65aebe915d24f1c56c89981db4fa1380c353bd5fc80eeea395c7ce77328cb2c\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:28.135225 kubelet[2720]: E0813 01:28:28.135016 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"a65aebe915d24f1c56c89981db4fa1380c353bd5fc80eeea395c7ce77328cb2c\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:28:28.135523 systemd[1]: run-netns-cni\x2d9cfb726c\x2d991c\x2ddd22\x2d7862\x2d587c85850dff.mount: Deactivated successfully. Aug 13 01:28:32.448447 systemd[1]: Started sshd@21-172.234.25.215:22-147.75.109.163:44158.service - OpenSSH per-connection server daemon (147.75.109.163:44158). Aug 13 01:28:32.785174 sshd[5378]: Accepted publickey for core from 147.75.109.163 port 44158 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:28:32.786957 sshd-session[5378]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:28:32.794153 systemd-logind[1543]: New session 22 of user core. Aug 13 01:28:32.801062 systemd[1]: Started session-22.scope - Session 22 of User core. Aug 13 01:28:33.100075 sshd[5380]: Connection closed by 147.75.109.163 port 44158 Aug 13 01:28:33.100927 sshd-session[5378]: pam_unix(sshd:session): session closed for user core Aug 13 01:28:33.106252 systemd[1]: sshd@21-172.234.25.215:22-147.75.109.163:44158.service: Deactivated successfully. Aug 13 01:28:33.110017 systemd[1]: session-22.scope: Deactivated successfully. Aug 13 01:28:33.111249 systemd-logind[1543]: Session 22 logged out. Waiting for processes to exit. Aug 13 01:28:33.113666 systemd-logind[1543]: Removed session 22. Aug 13 01:28:38.166936 systemd[1]: Started sshd@22-172.234.25.215:22-147.75.109.163:59662.service - OpenSSH per-connection server daemon (147.75.109.163:59662). Aug 13 01:28:38.500358 sshd[5392]: Accepted publickey for core from 147.75.109.163 port 59662 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:28:38.502525 sshd-session[5392]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:28:38.508415 systemd-logind[1543]: New session 23 of user core. Aug 13 01:28:38.513837 systemd[1]: Started session-23.scope - Session 23 of User core. Aug 13 01:28:38.809179 sshd[5394]: Connection closed by 147.75.109.163 port 59662 Aug 13 01:28:38.809703 sshd-session[5392]: pam_unix(sshd:session): session closed for user core Aug 13 01:28:38.814204 systemd[1]: sshd@22-172.234.25.215:22-147.75.109.163:59662.service: Deactivated successfully. Aug 13 01:28:38.816823 systemd[1]: session-23.scope: Deactivated successfully. Aug 13 01:28:38.817779 systemd-logind[1543]: Session 23 logged out. Waiting for processes to exit. Aug 13 01:28:38.819880 systemd-logind[1543]: Removed session 23. Aug 13 01:28:39.073933 containerd[1567]: time="2025-08-13T01:28:39.073788923Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:28:39.130115 containerd[1567]: time="2025-08-13T01:28:39.130069242Z" level=error msg="Failed to destroy network for sandbox \"254f1950860e9a7144eeff481e70b7d342fa6d9393d6bf42c0213f9a5d59cb18\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:39.132825 containerd[1567]: time="2025-08-13T01:28:39.132777966Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"254f1950860e9a7144eeff481e70b7d342fa6d9393d6bf42c0213f9a5d59cb18\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:39.132903 systemd[1]: run-netns-cni\x2d38d7ba44\x2dc83b\x2d0bc2\x2d80f2\x2dbe34a1fdcb74.mount: Deactivated successfully. Aug 13 01:28:39.133390 kubelet[2720]: E0813 01:28:39.133335 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"254f1950860e9a7144eeff481e70b7d342fa6d9393d6bf42c0213f9a5d59cb18\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:39.133969 kubelet[2720]: E0813 01:28:39.133697 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"254f1950860e9a7144eeff481e70b7d342fa6d9393d6bf42c0213f9a5d59cb18\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:28:39.133969 kubelet[2720]: E0813 01:28:39.133762 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"254f1950860e9a7144eeff481e70b7d342fa6d9393d6bf42c0213f9a5d59cb18\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:28:39.133969 kubelet[2720]: E0813 01:28:39.133849 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"csi-node-driver-8bzzs_calico-system(1e4523fc-9f2d-4edf-945c-d7ef00a14fdf)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"254f1950860e9a7144eeff481e70b7d342fa6d9393d6bf42c0213f9a5d59cb18\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/csi-node-driver-8bzzs" podUID="1e4523fc-9f2d-4edf-945c-d7ef00a14fdf" Aug 13 01:28:40.076489 kubelet[2720]: E0813 01:28:40.076455 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:40.077019 containerd[1567]: time="2025-08-13T01:28:40.076935116Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:28:40.178280 containerd[1567]: time="2025-08-13T01:28:40.178141220Z" level=error msg="Failed to destroy network for sandbox \"8d9ace47701a08f661c6e66c07bbcd13008a7709245f364e1b0495f4e50ab4cf\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:40.182098 containerd[1567]: time="2025-08-13T01:28:40.181984931Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"8d9ace47701a08f661c6e66c07bbcd13008a7709245f364e1b0495f4e50ab4cf\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:40.182241 kubelet[2720]: E0813 01:28:40.182198 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8d9ace47701a08f661c6e66c07bbcd13008a7709245f364e1b0495f4e50ab4cf\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:40.182526 kubelet[2720]: E0813 01:28:40.182252 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8d9ace47701a08f661c6e66c07bbcd13008a7709245f364e1b0495f4e50ab4cf\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:28:40.182526 kubelet[2720]: E0813 01:28:40.182273 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8d9ace47701a08f661c6e66c07bbcd13008a7709245f364e1b0495f4e50ab4cf\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:28:40.182574 kubelet[2720]: E0813 01:28:40.182361 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"8d9ace47701a08f661c6e66c07bbcd13008a7709245f364e1b0495f4e50ab4cf\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:28:40.184978 systemd[1]: run-netns-cni\x2d72039c4a\x2d35f0\x2db48e\x2debaf\x2dd95fbfbd9750.mount: Deactivated successfully. Aug 13 01:28:41.074779 kubelet[2720]: E0813 01:28:41.074392 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:41.075249 containerd[1567]: time="2025-08-13T01:28:41.075202780Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:28:41.076482 containerd[1567]: time="2025-08-13T01:28:41.076445536Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/node:v3.30.2\"" Aug 13 01:28:41.144363 containerd[1567]: time="2025-08-13T01:28:41.144283812Z" level=error msg="Failed to destroy network for sandbox \"c870ec38ab5482d7434e075e8fb61e6d3d3cb37dcf49ad48fc6ca3d2f6340a34\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:41.146670 containerd[1567]: time="2025-08-13T01:28:41.146567807Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c870ec38ab5482d7434e075e8fb61e6d3d3cb37dcf49ad48fc6ca3d2f6340a34\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:41.147047 kubelet[2720]: E0813 01:28:41.146942 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c870ec38ab5482d7434e075e8fb61e6d3d3cb37dcf49ad48fc6ca3d2f6340a34\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:41.147047 kubelet[2720]: E0813 01:28:41.146993 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c870ec38ab5482d7434e075e8fb61e6d3d3cb37dcf49ad48fc6ca3d2f6340a34\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:28:41.147047 kubelet[2720]: E0813 01:28:41.147012 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c870ec38ab5482d7434e075e8fb61e6d3d3cb37dcf49ad48fc6ca3d2f6340a34\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:28:41.147199 kubelet[2720]: E0813 01:28:41.147177 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c870ec38ab5482d7434e075e8fb61e6d3d3cb37dcf49ad48fc6ca3d2f6340a34\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:28:41.148135 systemd[1]: run-netns-cni\x2d06e095fa\x2d7840\x2d8de0\x2d59f1\x2dcde816b8f95f.mount: Deactivated successfully. Aug 13 01:28:42.087743 containerd[1567]: time="2025-08-13T01:28:42.087646223Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:28:42.166296 containerd[1567]: time="2025-08-13T01:28:42.166187916Z" level=error msg="Failed to destroy network for sandbox \"8445676f11026ebd00e4657249a5cc1746e8c030f4a078761492a17319aa17bd\"" error="plugin type=\"calico\" failed (delete): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:42.169125 systemd[1]: run-netns-cni\x2d4ee2600d\x2dac8f\x2ded14\x2dd9fc\x2ddcd818034a99.mount: Deactivated successfully. Aug 13 01:28:42.171519 containerd[1567]: time="2025-08-13T01:28:42.171296535Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"8445676f11026ebd00e4657249a5cc1746e8c030f4a078761492a17319aa17bd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:42.172169 kubelet[2720]: E0813 01:28:42.171830 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8445676f11026ebd00e4657249a5cc1746e8c030f4a078761492a17319aa17bd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" Aug 13 01:28:42.172169 kubelet[2720]: E0813 01:28:42.172000 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8445676f11026ebd00e4657249a5cc1746e8c030f4a078761492a17319aa17bd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:42.172169 kubelet[2720]: E0813 01:28:42.172020 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8445676f11026ebd00e4657249a5cc1746e8c030f4a078761492a17319aa17bd\": plugin type=\"calico\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:42.173306 kubelet[2720]: E0813 01:28:42.172500 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"8445676f11026ebd00e4657249a5cc1746e8c030f4a078761492a17319aa17bd\\\": plugin type=\\\"calico\\\" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:28:43.873130 systemd[1]: Started sshd@23-172.234.25.215:22-147.75.109.163:59674.service - OpenSSH per-connection server daemon (147.75.109.163:59674). Aug 13 01:28:44.223849 sshd[5517]: Accepted publickey for core from 147.75.109.163 port 59674 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:28:44.226035 sshd-session[5517]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:28:44.233209 systemd-logind[1543]: New session 24 of user core. Aug 13 01:28:44.238834 systemd[1]: Started session-24.scope - Session 24 of User core. Aug 13 01:28:44.557357 sshd[5519]: Connection closed by 147.75.109.163 port 59674 Aug 13 01:28:44.559197 sshd-session[5517]: pam_unix(sshd:session): session closed for user core Aug 13 01:28:44.566966 systemd[1]: sshd@23-172.234.25.215:22-147.75.109.163:59674.service: Deactivated successfully. Aug 13 01:28:44.571454 systemd[1]: session-24.scope: Deactivated successfully. Aug 13 01:28:44.574180 systemd-logind[1543]: Session 24 logged out. Waiting for processes to exit. Aug 13 01:28:44.577208 systemd-logind[1543]: Removed session 24. Aug 13 01:28:45.575043 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount739584668.mount: Deactivated successfully. Aug 13 01:28:45.602996 containerd[1567]: time="2025-08-13T01:28:45.602947765Z" level=info msg="ImageCreate event name:\"ghcr.io/flatcar/calico/node:v3.30.2\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:28:45.603793 containerd[1567]: time="2025-08-13T01:28:45.603475853Z" level=info msg="stop pulling image ghcr.io/flatcar/calico/node:v3.30.2: active requests=0, bytes read=158500163" Aug 13 01:28:45.604746 containerd[1567]: time="2025-08-13T01:28:45.604236872Z" level=info msg="ImageCreate event name:\"sha256:cc52550d767f73458fee2ee68db9db5de30d175e8fa4569ebdb43610127b6d20\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:28:45.606296 containerd[1567]: time="2025-08-13T01:28:45.605976389Z" level=info msg="ImageCreate event name:\"ghcr.io/flatcar/calico/node@sha256:e94d49349cc361ef2216d27dda4a097278984d778279f66e79b0616c827c6760\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:28:45.606888 containerd[1567]: time="2025-08-13T01:28:45.606591717Z" level=info msg="Pulled image \"ghcr.io/flatcar/calico/node:v3.30.2\" with image id \"sha256:cc52550d767f73458fee2ee68db9db5de30d175e8fa4569ebdb43610127b6d20\", repo tag \"ghcr.io/flatcar/calico/node:v3.30.2\", repo digest \"ghcr.io/flatcar/calico/node@sha256:e94d49349cc361ef2216d27dda4a097278984d778279f66e79b0616c827c6760\", size \"158500025\" in 4.530102081s" Aug 13 01:28:45.607052 containerd[1567]: time="2025-08-13T01:28:45.607035436Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/node:v3.30.2\" returns image reference \"sha256:cc52550d767f73458fee2ee68db9db5de30d175e8fa4569ebdb43610127b6d20\"" Aug 13 01:28:45.627546 containerd[1567]: time="2025-08-13T01:28:45.627506103Z" level=info msg="CreateContainer within sandbox \"5105f19a4fa4aed619cbcaf7a7ad5ac97da1bec5f3e441a9c5640cdaf472ed9d\" for container &ContainerMetadata{Name:calico-node,Attempt:0,}" Aug 13 01:28:45.635839 containerd[1567]: time="2025-08-13T01:28:45.634896818Z" level=info msg="Container 05a7e6943f56eff710d015cc4d373aa0238d7506c1cf61f23b36bc5ebcb7ce84: CDI devices from CRI Config.CDIDevices: []" Aug 13 01:28:45.641915 containerd[1567]: time="2025-08-13T01:28:45.641890854Z" level=info msg="CreateContainer within sandbox \"5105f19a4fa4aed619cbcaf7a7ad5ac97da1bec5f3e441a9c5640cdaf472ed9d\" for &ContainerMetadata{Name:calico-node,Attempt:0,} returns container id \"05a7e6943f56eff710d015cc4d373aa0238d7506c1cf61f23b36bc5ebcb7ce84\"" Aug 13 01:28:45.643613 containerd[1567]: time="2025-08-13T01:28:45.643594559Z" level=info msg="StartContainer for \"05a7e6943f56eff710d015cc4d373aa0238d7506c1cf61f23b36bc5ebcb7ce84\"" Aug 13 01:28:45.644850 containerd[1567]: time="2025-08-13T01:28:45.644829047Z" level=info msg="connecting to shim 05a7e6943f56eff710d015cc4d373aa0238d7506c1cf61f23b36bc5ebcb7ce84" address="unix:///run/containerd/s/80cc981cbd846a3b6e47f304d0dd0986382b4ac30c826ac9de5859d4834de77a" protocol=ttrpc version=3 Aug 13 01:28:45.671860 systemd[1]: Started cri-containerd-05a7e6943f56eff710d015cc4d373aa0238d7506c1cf61f23b36bc5ebcb7ce84.scope - libcontainer container 05a7e6943f56eff710d015cc4d373aa0238d7506c1cf61f23b36bc5ebcb7ce84. Aug 13 01:28:45.742547 containerd[1567]: time="2025-08-13T01:28:45.742507862Z" level=info msg="StartContainer for \"05a7e6943f56eff710d015cc4d373aa0238d7506c1cf61f23b36bc5ebcb7ce84\" returns successfully" Aug 13 01:28:45.826218 kernel: wireguard: WireGuard 1.0.0 loaded. See www.wireguard.com for information. Aug 13 01:28:45.826339 kernel: wireguard: Copyright (C) 2015-2019 Jason A. Donenfeld . All Rights Reserved. Aug 13 01:28:46.618096 kubelet[2720]: I0813 01:28:46.617992 2720 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="calico-system/calico-node-2p8hj" podStartSLOduration=1.588048749 podStartE2EDuration="3m7.617965202s" podCreationTimestamp="2025-08-13 01:25:39 +0000 UTC" firstStartedPulling="2025-08-13 01:25:39.578003152 +0000 UTC m=+17.598552535" lastFinishedPulling="2025-08-13 01:28:45.607919605 +0000 UTC m=+203.628468988" observedRunningTime="2025-08-13 01:28:46.587792945 +0000 UTC m=+204.608342328" watchObservedRunningTime="2025-08-13 01:28:46.617965202 +0000 UTC m=+204.638514585" Aug 13 01:28:46.677349 containerd[1567]: time="2025-08-13T01:28:46.677294388Z" level=info msg="TaskExit event in podsandbox handler container_id:\"05a7e6943f56eff710d015cc4d373aa0238d7506c1cf61f23b36bc5ebcb7ce84\" id:\"6880c27da92e330ad1557b09acd22ce78804e3456bed12e069d208f63f1f7c04\" pid:5606 exit_status:1 exited_at:{seconds:1755048526 nanos:675935101}" Aug 13 01:28:47.684156 containerd[1567]: time="2025-08-13T01:28:47.684090641Z" level=info msg="TaskExit event in podsandbox handler container_id:\"05a7e6943f56eff710d015cc4d373aa0238d7506c1cf61f23b36bc5ebcb7ce84\" id:\"16693408b903725ee5550d955bf17ef391c922179197688ff5da7e41fa6cbed3\" pid:5753 exit_status:1 exited_at:{seconds:1755048527 nanos:683836032}" Aug 13 01:28:47.916212 systemd-networkd[1449]: vxlan.calico: Link UP Aug 13 01:28:47.916220 systemd-networkd[1449]: vxlan.calico: Gained carrier Aug 13 01:28:47.982197 kubelet[2720]: I0813 01:28:47.982165 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:28:47.983835 kubelet[2720]: I0813 01:28:47.982206 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:28:47.986329 kubelet[2720]: I0813 01:28:47.985952 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:28:47.988435 kubelet[2720]: I0813 01:28:47.988407 2720 image_gc_manager.go:514] "Removing image to free bytes" imageID="sha256:8bde16470b09d1963e19456806d73180c9778a6c2b3c1fda2335c67c1cd4ce93" size=25052538 runtimeHandler="" Aug 13 01:28:47.988890 containerd[1567]: time="2025-08-13T01:28:47.988770437Z" level=info msg="RemoveImage \"sha256:8bde16470b09d1963e19456806d73180c9778a6c2b3c1fda2335c67c1cd4ce93\"" Aug 13 01:28:47.989984 containerd[1567]: time="2025-08-13T01:28:47.989964995Z" level=info msg="ImageDelete event name:\"quay.io/tigera/operator:v1.38.3\"" Aug 13 01:28:47.990580 containerd[1567]: time="2025-08-13T01:28:47.990539534Z" level=info msg="ImageDelete event name:\"quay.io/tigera/operator@sha256:dbf1bad0def7b5955dc8e4aeee96e23ead0bc5822f6872518e685cd0ed484121\"" Aug 13 01:28:47.991066 containerd[1567]: time="2025-08-13T01:28:47.990965763Z" level=info msg="RemoveImage \"sha256:8bde16470b09d1963e19456806d73180c9778a6c2b3c1fda2335c67c1cd4ce93\" returns successfully" Aug 13 01:28:47.991066 containerd[1567]: time="2025-08-13T01:28:47.991032842Z" level=info msg="ImageDelete event name:\"sha256:8bde16470b09d1963e19456806d73180c9778a6c2b3c1fda2335c67c1cd4ce93\"" Aug 13 01:28:48.004730 kubelet[2720]: I0813 01:28:48.004503 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:28:48.004730 kubelet[2720]: I0813 01:28:48.004593 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-kube-controllers-b57d47f84-lznkx","calico-system/csi-node-driver-8bzzs","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:28:48.004730 kubelet[2720]: E0813 01:28:48.004631 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:28:48.004730 kubelet[2720]: E0813 01:28:48.004639 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:28:48.004730 kubelet[2720]: E0813 01:28:48.004645 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:48.004730 kubelet[2720]: E0813 01:28:48.004652 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:28:48.004730 kubelet[2720]: E0813 01:28:48.004661 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:28:48.004730 kubelet[2720]: E0813 01:28:48.004669 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:28:48.004730 kubelet[2720]: E0813 01:28:48.004676 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:28:48.004730 kubelet[2720]: E0813 01:28:48.004683 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:28:48.004730 kubelet[2720]: E0813 01:28:48.004690 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:28:48.004730 kubelet[2720]: E0813 01:28:48.004697 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:28:48.004730 kubelet[2720]: I0813 01:28:48.004706 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:28:49.049512 systemd-networkd[1449]: vxlan.calico: Gained IPv6LL Aug 13 01:28:49.617926 systemd[1]: Started sshd@24-172.234.25.215:22-147.75.109.163:53464.service - OpenSSH per-connection server daemon (147.75.109.163:53464). Aug 13 01:28:49.952185 sshd[5831]: Accepted publickey for core from 147.75.109.163 port 53464 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:28:49.953455 sshd-session[5831]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:28:49.960658 systemd-logind[1543]: New session 25 of user core. Aug 13 01:28:49.966847 systemd[1]: Started session-25.scope - Session 25 of User core. Aug 13 01:28:50.271364 sshd[5833]: Connection closed by 147.75.109.163 port 53464 Aug 13 01:28:50.271922 sshd-session[5831]: pam_unix(sshd:session): session closed for user core Aug 13 01:28:50.277504 systemd[1]: sshd@24-172.234.25.215:22-147.75.109.163:53464.service: Deactivated successfully. Aug 13 01:28:50.280018 systemd[1]: session-25.scope: Deactivated successfully. Aug 13 01:28:50.281035 systemd-logind[1543]: Session 25 logged out. Waiting for processes to exit. Aug 13 01:28:50.283166 systemd-logind[1543]: Removed session 25. Aug 13 01:28:51.074265 containerd[1567]: time="2025-08-13T01:28:51.074218085Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,}" Aug 13 01:28:51.207487 systemd-networkd[1449]: cali42835c554cd: Link UP Aug 13 01:28:51.209118 systemd-networkd[1449]: cali42835c554cd: Gained carrier Aug 13 01:28:51.227788 containerd[1567]: 2025-08-13 01:28:51.134 [INFO][5845] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-csi--node--driver--8bzzs-eth0 csi-node-driver- calico-system 1e4523fc-9f2d-4edf-945c-d7ef00a14fdf 709 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:csi-node-driver controller-revision-hash:8967bcb6f k8s-app:csi-node-driver name:csi-node-driver pod-template-generation:1 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:csi-node-driver] map[] [] [] []} {k8s 172-234-25-215 csi-node-driver-8bzzs eth0 csi-node-driver [] [] [kns.calico-system ksa.calico-system.csi-node-driver] cali42835c554cd [] [] }} ContainerID="936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" Namespace="calico-system" Pod="csi-node-driver-8bzzs" WorkloadEndpoint="172--234--25--215-k8s-csi--node--driver--8bzzs-" Aug 13 01:28:51.227788 containerd[1567]: 2025-08-13 01:28:51.135 [INFO][5845] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" Namespace="calico-system" Pod="csi-node-driver-8bzzs" WorkloadEndpoint="172--234--25--215-k8s-csi--node--driver--8bzzs-eth0" Aug 13 01:28:51.227788 containerd[1567]: 2025-08-13 01:28:51.163 [INFO][5856] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" HandleID="k8s-pod-network.936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" Workload="172--234--25--215-k8s-csi--node--driver--8bzzs-eth0" Aug 13 01:28:51.228226 containerd[1567]: 2025-08-13 01:28:51.163 [INFO][5856] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" HandleID="k8s-pod-network.936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" Workload="172--234--25--215-k8s-csi--node--driver--8bzzs-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cd270), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"csi-node-driver-8bzzs", "timestamp":"2025-08-13 01:28:51.163521191 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:28:51.228226 containerd[1567]: 2025-08-13 01:28:51.163 [INFO][5856] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:28:51.228226 containerd[1567]: 2025-08-13 01:28:51.163 [INFO][5856] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:28:51.228226 containerd[1567]: 2025-08-13 01:28:51.163 [INFO][5856] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:28:51.228226 containerd[1567]: 2025-08-13 01:28:51.169 [INFO][5856] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" host="172-234-25-215" Aug 13 01:28:51.228226 containerd[1567]: 2025-08-13 01:28:51.173 [INFO][5856] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:28:51.228226 containerd[1567]: 2025-08-13 01:28:51.176 [INFO][5856] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:51.228226 containerd[1567]: 2025-08-13 01:28:51.178 [INFO][5856] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:51.228226 containerd[1567]: 2025-08-13 01:28:51.180 [INFO][5856] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:51.228226 containerd[1567]: 2025-08-13 01:28:51.180 [INFO][5856] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" host="172-234-25-215" Aug 13 01:28:51.229099 containerd[1567]: 2025-08-13 01:28:51.182 [INFO][5856] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143 Aug 13 01:28:51.229099 containerd[1567]: 2025-08-13 01:28:51.185 [INFO][5856] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" host="172-234-25-215" Aug 13 01:28:51.229099 containerd[1567]: 2025-08-13 01:28:51.189 [INFO][5856] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.1/26] block=192.168.62.0/26 handle="k8s-pod-network.936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" host="172-234-25-215" Aug 13 01:28:51.229099 containerd[1567]: 2025-08-13 01:28:51.190 [INFO][5856] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.1/26] handle="k8s-pod-network.936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" host="172-234-25-215" Aug 13 01:28:51.229099 containerd[1567]: 2025-08-13 01:28:51.190 [INFO][5856] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:28:51.229099 containerd[1567]: 2025-08-13 01:28:51.190 [INFO][5856] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.1/26] IPv6=[] ContainerID="936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" HandleID="k8s-pod-network.936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" Workload="172--234--25--215-k8s-csi--node--driver--8bzzs-eth0" Aug 13 01:28:51.229784 containerd[1567]: 2025-08-13 01:28:51.195 [INFO][5845] cni-plugin/k8s.go 418: Populated endpoint ContainerID="936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" Namespace="calico-system" Pod="csi-node-driver-8bzzs" WorkloadEndpoint="172--234--25--215-k8s-csi--node--driver--8bzzs-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-csi--node--driver--8bzzs-eth0", GenerateName:"csi-node-driver-", Namespace:"calico-system", SelfLink:"", UID:"1e4523fc-9f2d-4edf-945c-d7ef00a14fdf", ResourceVersion:"709", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"csi-node-driver", "controller-revision-hash":"8967bcb6f", "k8s-app":"csi-node-driver", "name":"csi-node-driver", "pod-template-generation":"1", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"csi-node-driver"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"", Pod:"csi-node-driver-8bzzs", Endpoint:"eth0", ServiceAccountName:"csi-node-driver", IPNetworks:[]string{"192.168.62.1/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.csi-node-driver"}, InterfaceName:"cali42835c554cd", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:28:51.229974 containerd[1567]: 2025-08-13 01:28:51.195 [INFO][5845] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.1/32] ContainerID="936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" Namespace="calico-system" Pod="csi-node-driver-8bzzs" WorkloadEndpoint="172--234--25--215-k8s-csi--node--driver--8bzzs-eth0" Aug 13 01:28:51.229974 containerd[1567]: 2025-08-13 01:28:51.195 [INFO][5845] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali42835c554cd ContainerID="936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" Namespace="calico-system" Pod="csi-node-driver-8bzzs" WorkloadEndpoint="172--234--25--215-k8s-csi--node--driver--8bzzs-eth0" Aug 13 01:28:51.229974 containerd[1567]: 2025-08-13 01:28:51.210 [INFO][5845] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" Namespace="calico-system" Pod="csi-node-driver-8bzzs" WorkloadEndpoint="172--234--25--215-k8s-csi--node--driver--8bzzs-eth0" Aug 13 01:28:51.230409 containerd[1567]: 2025-08-13 01:28:51.211 [INFO][5845] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" Namespace="calico-system" Pod="csi-node-driver-8bzzs" WorkloadEndpoint="172--234--25--215-k8s-csi--node--driver--8bzzs-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-csi--node--driver--8bzzs-eth0", GenerateName:"csi-node-driver-", Namespace:"calico-system", SelfLink:"", UID:"1e4523fc-9f2d-4edf-945c-d7ef00a14fdf", ResourceVersion:"709", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"csi-node-driver", "controller-revision-hash":"8967bcb6f", "k8s-app":"csi-node-driver", "name":"csi-node-driver", "pod-template-generation":"1", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"csi-node-driver"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143", Pod:"csi-node-driver-8bzzs", Endpoint:"eth0", ServiceAccountName:"csi-node-driver", IPNetworks:[]string{"192.168.62.1/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.csi-node-driver"}, InterfaceName:"cali42835c554cd", MAC:"fe:cc:dc:37:7b:e8", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:28:51.230610 containerd[1567]: 2025-08-13 01:28:51.222 [INFO][5845] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" Namespace="calico-system" Pod="csi-node-driver-8bzzs" WorkloadEndpoint="172--234--25--215-k8s-csi--node--driver--8bzzs-eth0" Aug 13 01:28:51.279028 containerd[1567]: time="2025-08-13T01:28:51.278982904Z" level=info msg="connecting to shim 936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143" address="unix:///run/containerd/s/8a78b27c0de1d1e0eedf275b06f5a22cd87e8ca883f2e982a1c0f4b77fa48a1f" namespace=k8s.io protocol=ttrpc version=3 Aug 13 01:28:51.310966 systemd[1]: Started cri-containerd-936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143.scope - libcontainer container 936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143. Aug 13 01:28:51.351353 containerd[1567]: time="2025-08-13T01:28:51.351313256Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:csi-node-driver-8bzzs,Uid:1e4523fc-9f2d-4edf-945c-d7ef00a14fdf,Namespace:calico-system,Attempt:0,} returns sandbox id \"936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143\"" Aug 13 01:28:51.353907 containerd[1567]: time="2025-08-13T01:28:51.353879671Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/csi:v3.30.2\"" Aug 13 01:28:52.029121 containerd[1567]: time="2025-08-13T01:28:52.029075906Z" level=info msg="ImageCreate event name:\"ghcr.io/flatcar/calico/csi:v3.30.2\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:28:52.029787 containerd[1567]: time="2025-08-13T01:28:52.029744214Z" level=info msg="stop pulling image ghcr.io/flatcar/calico/csi:v3.30.2: active requests=0, bytes read=8759190" Aug 13 01:28:52.030405 containerd[1567]: time="2025-08-13T01:28:52.030300484Z" level=info msg="ImageCreate event name:\"sha256:c7fd1cc652979d89a51bbcc125e28e90c9815c0bd8f922a5bd36eed4e1927c6d\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:28:52.031939 containerd[1567]: time="2025-08-13T01:28:52.031897850Z" level=info msg="ImageCreate event name:\"ghcr.io/flatcar/calico/csi@sha256:e570128aa8067a2f06b96d3cc98afa2e0a4b9790b435ee36ca051c8e72aeb8d0\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:28:52.032462 containerd[1567]: time="2025-08-13T01:28:52.032422229Z" level=info msg="Pulled image \"ghcr.io/flatcar/calico/csi:v3.30.2\" with image id \"sha256:c7fd1cc652979d89a51bbcc125e28e90c9815c0bd8f922a5bd36eed4e1927c6d\", repo tag \"ghcr.io/flatcar/calico/csi:v3.30.2\", repo digest \"ghcr.io/flatcar/calico/csi@sha256:e570128aa8067a2f06b96d3cc98afa2e0a4b9790b435ee36ca051c8e72aeb8d0\", size \"10251893\" in 678.489598ms" Aug 13 01:28:52.032506 containerd[1567]: time="2025-08-13T01:28:52.032462649Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/csi:v3.30.2\" returns image reference \"sha256:c7fd1cc652979d89a51bbcc125e28e90c9815c0bd8f922a5bd36eed4e1927c6d\"" Aug 13 01:28:52.036656 containerd[1567]: time="2025-08-13T01:28:52.036090511Z" level=info msg="CreateContainer within sandbox \"936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143\" for container &ContainerMetadata{Name:calico-csi,Attempt:0,}" Aug 13 01:28:52.043899 containerd[1567]: time="2025-08-13T01:28:52.043865075Z" level=info msg="Container 720db0db93199e5525beb448fcfc7c27f413f15e1492e36a045783d5aed4f39f: CDI devices from CRI Config.CDIDevices: []" Aug 13 01:28:52.054764 containerd[1567]: time="2025-08-13T01:28:52.054740723Z" level=info msg="CreateContainer within sandbox \"936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143\" for &ContainerMetadata{Name:calico-csi,Attempt:0,} returns container id \"720db0db93199e5525beb448fcfc7c27f413f15e1492e36a045783d5aed4f39f\"" Aug 13 01:28:52.056005 containerd[1567]: time="2025-08-13T01:28:52.055879751Z" level=info msg="StartContainer for \"720db0db93199e5525beb448fcfc7c27f413f15e1492e36a045783d5aed4f39f\"" Aug 13 01:28:52.057512 containerd[1567]: time="2025-08-13T01:28:52.057493367Z" level=info msg="connecting to shim 720db0db93199e5525beb448fcfc7c27f413f15e1492e36a045783d5aed4f39f" address="unix:///run/containerd/s/8a78b27c0de1d1e0eedf275b06f5a22cd87e8ca883f2e982a1c0f4b77fa48a1f" protocol=ttrpc version=3 Aug 13 01:28:52.089839 systemd[1]: Started cri-containerd-720db0db93199e5525beb448fcfc7c27f413f15e1492e36a045783d5aed4f39f.scope - libcontainer container 720db0db93199e5525beb448fcfc7c27f413f15e1492e36a045783d5aed4f39f. Aug 13 01:28:52.157202 containerd[1567]: time="2025-08-13T01:28:52.157160744Z" level=info msg="StartContainer for \"720db0db93199e5525beb448fcfc7c27f413f15e1492e36a045783d5aed4f39f\" returns successfully" Aug 13 01:28:52.160361 containerd[1567]: time="2025-08-13T01:28:52.160150647Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/node-driver-registrar:v3.30.2\"" Aug 13 01:28:52.633208 systemd-networkd[1449]: cali42835c554cd: Gained IPv6LL Aug 13 01:28:52.935839 containerd[1567]: time="2025-08-13T01:28:52.935664022Z" level=info msg="ImageCreate event name:\"ghcr.io/flatcar/calico/node-driver-registrar:v3.30.2\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:28:52.936596 containerd[1567]: time="2025-08-13T01:28:52.936567800Z" level=info msg="stop pulling image ghcr.io/flatcar/calico/node-driver-registrar:v3.30.2: active requests=0, bytes read=14703784" Aug 13 01:28:52.938377 containerd[1567]: time="2025-08-13T01:28:52.937540918Z" level=info msg="ImageCreate event name:\"sha256:9e48822a4fe26f4ed9231b361fdd1357ea3567f1fc0a8db4d616622fe570a866\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:28:52.939102 containerd[1567]: time="2025-08-13T01:28:52.939074925Z" level=info msg="ImageCreate event name:\"ghcr.io/flatcar/calico/node-driver-registrar@sha256:8fec2de12dfa51bae89d941938a07af2598eb8bfcab55d0dded1d9c193d7b99f\" labels:{key:\"io.cri-containerd.image\" value:\"managed\"}" Aug 13 01:28:52.939919 containerd[1567]: time="2025-08-13T01:28:52.939895063Z" level=info msg="Pulled image \"ghcr.io/flatcar/calico/node-driver-registrar:v3.30.2\" with image id \"sha256:9e48822a4fe26f4ed9231b361fdd1357ea3567f1fc0a8db4d616622fe570a866\", repo tag \"ghcr.io/flatcar/calico/node-driver-registrar:v3.30.2\", repo digest \"ghcr.io/flatcar/calico/node-driver-registrar@sha256:8fec2de12dfa51bae89d941938a07af2598eb8bfcab55d0dded1d9c193d7b99f\", size \"16196439\" in 779.717106ms" Aug 13 01:28:52.940013 containerd[1567]: time="2025-08-13T01:28:52.939997783Z" level=info msg="PullImage \"ghcr.io/flatcar/calico/node-driver-registrar:v3.30.2\" returns image reference \"sha256:9e48822a4fe26f4ed9231b361fdd1357ea3567f1fc0a8db4d616622fe570a866\"" Aug 13 01:28:52.944070 containerd[1567]: time="2025-08-13T01:28:52.944007845Z" level=info msg="CreateContainer within sandbox \"936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143\" for container &ContainerMetadata{Name:csi-node-driver-registrar,Attempt:0,}" Aug 13 01:28:52.952069 containerd[1567]: time="2025-08-13T01:28:52.952011048Z" level=info msg="Container 84ef2c0eaadfec74ce9f4fb8a9ca060b11981af97b9781c42f35395293fdd3cb: CDI devices from CRI Config.CDIDevices: []" Aug 13 01:28:52.958558 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount406021704.mount: Deactivated successfully. Aug 13 01:28:52.970131 containerd[1567]: time="2025-08-13T01:28:52.970091921Z" level=info msg="CreateContainer within sandbox \"936a0160c97245a4e4fef9003e946741c79d0e35c83bac4916158b0baace7143\" for &ContainerMetadata{Name:csi-node-driver-registrar,Attempt:0,} returns container id \"84ef2c0eaadfec74ce9f4fb8a9ca060b11981af97b9781c42f35395293fdd3cb\"" Aug 13 01:28:52.973097 containerd[1567]: time="2025-08-13T01:28:52.973066176Z" level=info msg="StartContainer for \"84ef2c0eaadfec74ce9f4fb8a9ca060b11981af97b9781c42f35395293fdd3cb\"" Aug 13 01:28:52.974881 containerd[1567]: time="2025-08-13T01:28:52.974829601Z" level=info msg="connecting to shim 84ef2c0eaadfec74ce9f4fb8a9ca060b11981af97b9781c42f35395293fdd3cb" address="unix:///run/containerd/s/8a78b27c0de1d1e0eedf275b06f5a22cd87e8ca883f2e982a1c0f4b77fa48a1f" protocol=ttrpc version=3 Aug 13 01:28:53.008025 systemd[1]: Started cri-containerd-84ef2c0eaadfec74ce9f4fb8a9ca060b11981af97b9781c42f35395293fdd3cb.scope - libcontainer container 84ef2c0eaadfec74ce9f4fb8a9ca060b11981af97b9781c42f35395293fdd3cb. Aug 13 01:28:53.060321 containerd[1567]: time="2025-08-13T01:28:53.060281067Z" level=info msg="StartContainer for \"84ef2c0eaadfec74ce9f4fb8a9ca060b11981af97b9781c42f35395293fdd3cb\" returns successfully" Aug 13 01:28:53.075133 kubelet[2720]: E0813 01:28:53.075081 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:53.075864 containerd[1567]: time="2025-08-13T01:28:53.075764916Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,}" Aug 13 01:28:53.204831 systemd-networkd[1449]: calie073abd0336: Link UP Aug 13 01:28:53.205446 systemd-networkd[1449]: calie073abd0336: Gained carrier Aug 13 01:28:53.232118 containerd[1567]: 2025-08-13 01:28:53.123 [INFO][5990] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-coredns--674b8bbfcf--jlpx4-eth0 coredns-674b8bbfcf- kube-system e3a3eaf4-0523-46b9-8d19-1303011f36f8 817 0 2025-08-13 01:25:28 +0000 UTC map[k8s-app:kube-dns pod-template-hash:674b8bbfcf projectcalico.org/namespace:kube-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:coredns] map[] [] [] []} {k8s 172-234-25-215 coredns-674b8bbfcf-jlpx4 eth0 coredns [] [] [kns.kube-system ksa.kube-system.coredns] calie073abd0336 [{dns UDP 53 0 } {dns-tcp TCP 53 0 } {metrics TCP 9153 0 }] [] }} ContainerID="3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" Namespace="kube-system" Pod="coredns-674b8bbfcf-jlpx4" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--jlpx4-" Aug 13 01:28:53.232118 containerd[1567]: 2025-08-13 01:28:53.124 [INFO][5990] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" Namespace="kube-system" Pod="coredns-674b8bbfcf-jlpx4" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--jlpx4-eth0" Aug 13 01:28:53.232118 containerd[1567]: 2025-08-13 01:28:53.161 [INFO][6002] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" HandleID="k8s-pod-network.3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" Workload="172--234--25--215-k8s-coredns--674b8bbfcf--jlpx4-eth0" Aug 13 01:28:53.233139 containerd[1567]: 2025-08-13 01:28:53.161 [INFO][6002] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" HandleID="k8s-pod-network.3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" Workload="172--234--25--215-k8s-coredns--674b8bbfcf--jlpx4-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5940), Attrs:map[string]string{"namespace":"kube-system", "node":"172-234-25-215", "pod":"coredns-674b8bbfcf-jlpx4", "timestamp":"2025-08-13 01:28:53.161547731 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:28:53.233139 containerd[1567]: 2025-08-13 01:28:53.162 [INFO][6002] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:28:53.233139 containerd[1567]: 2025-08-13 01:28:53.162 [INFO][6002] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:28:53.233139 containerd[1567]: 2025-08-13 01:28:53.162 [INFO][6002] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:28:53.233139 containerd[1567]: 2025-08-13 01:28:53.169 [INFO][6002] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" host="172-234-25-215" Aug 13 01:28:53.233139 containerd[1567]: 2025-08-13 01:28:53.174 [INFO][6002] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:28:53.233139 containerd[1567]: 2025-08-13 01:28:53.181 [INFO][6002] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:53.233139 containerd[1567]: 2025-08-13 01:28:53.183 [INFO][6002] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:53.233139 containerd[1567]: 2025-08-13 01:28:53.185 [INFO][6002] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:53.233139 containerd[1567]: 2025-08-13 01:28:53.185 [INFO][6002] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" host="172-234-25-215" Aug 13 01:28:53.234198 containerd[1567]: 2025-08-13 01:28:53.187 [INFO][6002] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a Aug 13 01:28:53.234198 containerd[1567]: 2025-08-13 01:28:53.190 [INFO][6002] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" host="172-234-25-215" Aug 13 01:28:53.234198 containerd[1567]: 2025-08-13 01:28:53.195 [INFO][6002] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.2/26] block=192.168.62.0/26 handle="k8s-pod-network.3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" host="172-234-25-215" Aug 13 01:28:53.234198 containerd[1567]: 2025-08-13 01:28:53.195 [INFO][6002] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.2/26] handle="k8s-pod-network.3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" host="172-234-25-215" Aug 13 01:28:53.234198 containerd[1567]: 2025-08-13 01:28:53.195 [INFO][6002] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:28:53.234198 containerd[1567]: 2025-08-13 01:28:53.195 [INFO][6002] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.2/26] IPv6=[] ContainerID="3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" HandleID="k8s-pod-network.3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" Workload="172--234--25--215-k8s-coredns--674b8bbfcf--jlpx4-eth0" Aug 13 01:28:53.234333 containerd[1567]: 2025-08-13 01:28:53.198 [INFO][5990] cni-plugin/k8s.go 418: Populated endpoint ContainerID="3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" Namespace="kube-system" Pod="coredns-674b8bbfcf-jlpx4" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--jlpx4-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-coredns--674b8bbfcf--jlpx4-eth0", GenerateName:"coredns-674b8bbfcf-", Namespace:"kube-system", SelfLink:"", UID:"e3a3eaf4-0523-46b9-8d19-1303011f36f8", ResourceVersion:"817", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 28, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"k8s-app":"kube-dns", "pod-template-hash":"674b8bbfcf", "projectcalico.org/namespace":"kube-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"coredns"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"", Pod:"coredns-674b8bbfcf-jlpx4", Endpoint:"eth0", ServiceAccountName:"coredns", IPNetworks:[]string{"192.168.62.2/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.kube-system", "ksa.kube-system.coredns"}, InterfaceName:"calie073abd0336", MAC:"", Ports:[]v3.WorkloadEndpointPort{v3.WorkloadEndpointPort{Name:"dns", Protocol:numorstring.Protocol{Type:1, NumVal:0x0, StrVal:"UDP"}, Port:0x35, HostPort:0x0, HostIP:""}, v3.WorkloadEndpointPort{Name:"dns-tcp", Protocol:numorstring.Protocol{Type:1, NumVal:0x0, StrVal:"TCP"}, Port:0x35, HostPort:0x0, HostIP:""}, v3.WorkloadEndpointPort{Name:"metrics", Protocol:numorstring.Protocol{Type:1, NumVal:0x0, StrVal:"TCP"}, Port:0x23c1, HostPort:0x0, HostIP:""}}, AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:28:53.234333 containerd[1567]: 2025-08-13 01:28:53.198 [INFO][5990] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.2/32] ContainerID="3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" Namespace="kube-system" Pod="coredns-674b8bbfcf-jlpx4" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--jlpx4-eth0" Aug 13 01:28:53.234333 containerd[1567]: 2025-08-13 01:28:53.198 [INFO][5990] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to calie073abd0336 ContainerID="3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" Namespace="kube-system" Pod="coredns-674b8bbfcf-jlpx4" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--jlpx4-eth0" Aug 13 01:28:53.234333 containerd[1567]: 2025-08-13 01:28:53.204 [INFO][5990] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" Namespace="kube-system" Pod="coredns-674b8bbfcf-jlpx4" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--jlpx4-eth0" Aug 13 01:28:53.234333 containerd[1567]: 2025-08-13 01:28:53.207 [INFO][5990] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" Namespace="kube-system" Pod="coredns-674b8bbfcf-jlpx4" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--jlpx4-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-coredns--674b8bbfcf--jlpx4-eth0", GenerateName:"coredns-674b8bbfcf-", Namespace:"kube-system", SelfLink:"", UID:"e3a3eaf4-0523-46b9-8d19-1303011f36f8", ResourceVersion:"817", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 28, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"k8s-app":"kube-dns", "pod-template-hash":"674b8bbfcf", "projectcalico.org/namespace":"kube-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"coredns"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a", Pod:"coredns-674b8bbfcf-jlpx4", Endpoint:"eth0", ServiceAccountName:"coredns", IPNetworks:[]string{"192.168.62.2/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.kube-system", "ksa.kube-system.coredns"}, InterfaceName:"calie073abd0336", MAC:"56:47:bb:0e:5f:bd", Ports:[]v3.WorkloadEndpointPort{v3.WorkloadEndpointPort{Name:"dns", Protocol:numorstring.Protocol{Type:1, NumVal:0x0, StrVal:"UDP"}, Port:0x35, HostPort:0x0, HostIP:""}, v3.WorkloadEndpointPort{Name:"dns-tcp", Protocol:numorstring.Protocol{Type:1, NumVal:0x0, StrVal:"TCP"}, Port:0x35, HostPort:0x0, HostIP:""}, v3.WorkloadEndpointPort{Name:"metrics", Protocol:numorstring.Protocol{Type:1, NumVal:0x0, StrVal:"TCP"}, Port:0x23c1, HostPort:0x0, HostIP:""}}, AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:28:53.234333 containerd[1567]: 2025-08-13 01:28:53.225 [INFO][5990] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" Namespace="kube-system" Pod="coredns-674b8bbfcf-jlpx4" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--jlpx4-eth0" Aug 13 01:28:53.266140 containerd[1567]: time="2025-08-13T01:28:53.266066118Z" level=info msg="connecting to shim 3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a" address="unix:///run/containerd/s/d68341fa6319e474542b39a67423e98680f4d83cc37f8e2c1a733aee639437fe" namespace=k8s.io protocol=ttrpc version=3 Aug 13 01:28:53.274330 kubelet[2720]: I0813 01:28:53.274162 2720 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: csi.tigera.io endpoint: /var/lib/kubelet/plugins/csi.tigera.io/csi.sock versions: 1.0.0 Aug 13 01:28:53.274330 kubelet[2720]: I0813 01:28:53.274198 2720 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: csi.tigera.io at endpoint: /var/lib/kubelet/plugins/csi.tigera.io/csi.sock Aug 13 01:28:53.336878 systemd[1]: Started cri-containerd-3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a.scope - libcontainer container 3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a. Aug 13 01:28:53.384064 containerd[1567]: time="2025-08-13T01:28:53.383987058Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-jlpx4,Uid:e3a3eaf4-0523-46b9-8d19-1303011f36f8,Namespace:kube-system,Attempt:0,} returns sandbox id \"3d466ce7f0ff2ea6083664874c56c2c5b2ce5871024f2d6c2a9ddcf06a09422a\"" Aug 13 01:28:53.384940 kubelet[2720]: E0813 01:28:53.384883 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:53.386209 containerd[1567]: time="2025-08-13T01:28:53.386064463Z" level=info msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\"" Aug 13 01:28:53.663351 kubelet[2720]: I0813 01:28:53.663209 2720 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="calico-system/csi-node-driver-8bzzs" podStartSLOduration=193.075668969 podStartE2EDuration="3m14.663189628s" podCreationTimestamp="2025-08-13 01:25:39 +0000 UTC" firstStartedPulling="2025-08-13 01:28:51.353169482 +0000 UTC m=+209.373718875" lastFinishedPulling="2025-08-13 01:28:52.940690141 +0000 UTC m=+210.961239534" observedRunningTime="2025-08-13 01:28:53.601364655 +0000 UTC m=+211.621914038" watchObservedRunningTime="2025-08-13 01:28:53.663189628 +0000 UTC m=+211.683739011" Aug 13 01:28:54.175972 systemd[1]: var-lib-containerd-tmpmounts-containerd\x2dmount3587612026.mount: Deactivated successfully. Aug 13 01:28:55.129084 systemd-networkd[1449]: calie073abd0336: Gained IPv6LL Aug 13 01:28:55.340796 systemd[1]: Started sshd@25-172.234.25.215:22-147.75.109.163:53470.service - OpenSSH per-connection server daemon (147.75.109.163:53470). Aug 13 01:28:55.691635 sshd[6118]: Accepted publickey for core from 147.75.109.163 port 53470 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:28:55.694435 sshd-session[6118]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:28:55.711960 systemd-logind[1543]: New session 26 of user core. Aug 13 01:28:55.719002 systemd[1]: Started session-26.scope - Session 26 of User core. Aug 13 01:28:56.060178 sshd[6124]: Connection closed by 147.75.109.163 port 53470 Aug 13 01:28:56.059813 sshd-session[6118]: pam_unix(sshd:session): session closed for user core Aug 13 01:28:56.065398 systemd[1]: sshd@25-172.234.25.215:22-147.75.109.163:53470.service: Deactivated successfully. Aug 13 01:28:56.073643 systemd[1]: session-26.scope: Deactivated successfully. Aug 13 01:28:56.076535 systemd-logind[1543]: Session 26 logged out. Waiting for processes to exit. Aug 13 01:28:56.077113 kubelet[2720]: E0813 01:28:56.077055 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:56.081439 systemd-logind[1543]: Removed session 26. Aug 13 01:28:56.082769 containerd[1567]: time="2025-08-13T01:28:56.082730676Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,}" Aug 13 01:28:56.269299 systemd-networkd[1449]: cali6e0ca3c7413: Link UP Aug 13 01:28:56.270970 systemd-networkd[1449]: cali6e0ca3c7413: Gained carrier Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.162 [INFO][6135] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-coredns--674b8bbfcf--b2ds8-eth0 coredns-674b8bbfcf- kube-system 567c4932-37b3-4053-b427-2472bc4c195b 809 0 2025-08-13 01:25:28 +0000 UTC map[k8s-app:kube-dns pod-template-hash:674b8bbfcf projectcalico.org/namespace:kube-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:coredns] map[] [] [] []} {k8s 172-234-25-215 coredns-674b8bbfcf-b2ds8 eth0 coredns [] [] [kns.kube-system ksa.kube-system.coredns] cali6e0ca3c7413 [{dns UDP 53 0 } {dns-tcp TCP 53 0 } {metrics TCP 9153 0 }] [] }} ContainerID="a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" Namespace="kube-system" Pod="coredns-674b8bbfcf-b2ds8" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--b2ds8-" Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.163 [INFO][6135] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" Namespace="kube-system" Pod="coredns-674b8bbfcf-b2ds8" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--b2ds8-eth0" Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.202 [INFO][6149] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" HandleID="k8s-pod-network.a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" Workload="172--234--25--215-k8s-coredns--674b8bbfcf--b2ds8-eth0" Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.203 [INFO][6149] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" HandleID="k8s-pod-network.a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" Workload="172--234--25--215-k8s-coredns--674b8bbfcf--b2ds8-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5600), Attrs:map[string]string{"namespace":"kube-system", "node":"172-234-25-215", "pod":"coredns-674b8bbfcf-b2ds8", "timestamp":"2025-08-13 01:28:56.202548434 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.203 [INFO][6149] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.203 [INFO][6149] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.203 [INFO][6149] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.209 [INFO][6149] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" host="172-234-25-215" Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.216 [INFO][6149] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.224 [INFO][6149] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.227 [INFO][6149] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.230 [INFO][6149] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.230 [INFO][6149] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" host="172-234-25-215" Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.233 [INFO][6149] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1 Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.241 [INFO][6149] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" host="172-234-25-215" Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.248 [INFO][6149] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.3/26] block=192.168.62.0/26 handle="k8s-pod-network.a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" host="172-234-25-215" Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.249 [INFO][6149] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.3/26] handle="k8s-pod-network.a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" host="172-234-25-215" Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.249 [INFO][6149] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:28:56.311102 containerd[1567]: 2025-08-13 01:28:56.250 [INFO][6149] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.3/26] IPv6=[] ContainerID="a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" HandleID="k8s-pod-network.a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" Workload="172--234--25--215-k8s-coredns--674b8bbfcf--b2ds8-eth0" Aug 13 01:28:56.311621 containerd[1567]: 2025-08-13 01:28:56.260 [INFO][6135] cni-plugin/k8s.go 418: Populated endpoint ContainerID="a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" Namespace="kube-system" Pod="coredns-674b8bbfcf-b2ds8" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--b2ds8-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-coredns--674b8bbfcf--b2ds8-eth0", GenerateName:"coredns-674b8bbfcf-", Namespace:"kube-system", SelfLink:"", UID:"567c4932-37b3-4053-b427-2472bc4c195b", ResourceVersion:"809", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 28, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"k8s-app":"kube-dns", "pod-template-hash":"674b8bbfcf", "projectcalico.org/namespace":"kube-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"coredns"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"", Pod:"coredns-674b8bbfcf-b2ds8", Endpoint:"eth0", ServiceAccountName:"coredns", IPNetworks:[]string{"192.168.62.3/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.kube-system", "ksa.kube-system.coredns"}, InterfaceName:"cali6e0ca3c7413", MAC:"", Ports:[]v3.WorkloadEndpointPort{v3.WorkloadEndpointPort{Name:"dns", Protocol:numorstring.Protocol{Type:1, NumVal:0x0, StrVal:"UDP"}, Port:0x35, HostPort:0x0, HostIP:""}, v3.WorkloadEndpointPort{Name:"dns-tcp", Protocol:numorstring.Protocol{Type:1, NumVal:0x0, StrVal:"TCP"}, Port:0x35, HostPort:0x0, HostIP:""}, v3.WorkloadEndpointPort{Name:"metrics", Protocol:numorstring.Protocol{Type:1, NumVal:0x0, StrVal:"TCP"}, Port:0x23c1, HostPort:0x0, HostIP:""}}, AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:28:56.311621 containerd[1567]: 2025-08-13 01:28:56.260 [INFO][6135] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.3/32] ContainerID="a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" Namespace="kube-system" Pod="coredns-674b8bbfcf-b2ds8" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--b2ds8-eth0" Aug 13 01:28:56.311621 containerd[1567]: 2025-08-13 01:28:56.261 [INFO][6135] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali6e0ca3c7413 ContainerID="a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" Namespace="kube-system" Pod="coredns-674b8bbfcf-b2ds8" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--b2ds8-eth0" Aug 13 01:28:56.311621 containerd[1567]: 2025-08-13 01:28:56.270 [INFO][6135] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" Namespace="kube-system" Pod="coredns-674b8bbfcf-b2ds8" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--b2ds8-eth0" Aug 13 01:28:56.311621 containerd[1567]: 2025-08-13 01:28:56.270 [INFO][6135] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" Namespace="kube-system" Pod="coredns-674b8bbfcf-b2ds8" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--b2ds8-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-coredns--674b8bbfcf--b2ds8-eth0", GenerateName:"coredns-674b8bbfcf-", Namespace:"kube-system", SelfLink:"", UID:"567c4932-37b3-4053-b427-2472bc4c195b", ResourceVersion:"809", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 28, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"k8s-app":"kube-dns", "pod-template-hash":"674b8bbfcf", "projectcalico.org/namespace":"kube-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"coredns"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1", Pod:"coredns-674b8bbfcf-b2ds8", Endpoint:"eth0", ServiceAccountName:"coredns", IPNetworks:[]string{"192.168.62.3/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.kube-system", "ksa.kube-system.coredns"}, InterfaceName:"cali6e0ca3c7413", MAC:"92:24:c3:98:a0:18", Ports:[]v3.WorkloadEndpointPort{v3.WorkloadEndpointPort{Name:"dns", Protocol:numorstring.Protocol{Type:1, NumVal:0x0, StrVal:"UDP"}, Port:0x35, HostPort:0x0, HostIP:""}, v3.WorkloadEndpointPort{Name:"dns-tcp", Protocol:numorstring.Protocol{Type:1, NumVal:0x0, StrVal:"TCP"}, Port:0x35, HostPort:0x0, HostIP:""}, v3.WorkloadEndpointPort{Name:"metrics", Protocol:numorstring.Protocol{Type:1, NumVal:0x0, StrVal:"TCP"}, Port:0x23c1, HostPort:0x0, HostIP:""}}, AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:28:56.311621 containerd[1567]: 2025-08-13 01:28:56.288 [INFO][6135] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" Namespace="kube-system" Pod="coredns-674b8bbfcf-b2ds8" WorkloadEndpoint="172--234--25--215-k8s-coredns--674b8bbfcf--b2ds8-eth0" Aug 13 01:28:56.370776 containerd[1567]: time="2025-08-13T01:28:56.370588705Z" level=info msg="connecting to shim a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1" address="unix:///run/containerd/s/5e4a3f1b220912e3b061546b9b7af2ecd20c58ff3e2ea48097ed9e5d5d7512ad" namespace=k8s.io protocol=ttrpc version=3 Aug 13 01:28:56.423188 systemd[1]: Started cri-containerd-a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1.scope - libcontainer container a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1. Aug 13 01:28:56.461639 containerd[1567]: time="2025-08-13T01:28:56.461605271Z" level=error msg="failed to cleanup \"extract-464493858-lv7N sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\"" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:28:56.462558 containerd[1567]: time="2025-08-13T01:28:56.462485809Z" level=error msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\" failed" error="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to extract layer sha256:25359bcca1bb70511c264e8a14c78fbc75c226344e91d19f733ed309b336949f: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/92/fs/coredns: no space left on device" Aug 13 01:28:56.462878 kubelet[2720]: E0813 01:28:56.462823 2720 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to extract layer sha256:25359bcca1bb70511c264e8a14c78fbc75c226344e91d19f733ed309b336949f: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/92/fs/coredns: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:28:56.462970 kubelet[2720]: E0813 01:28:56.462885 2720 kuberuntime_image.go:42] "Failed to pull image" err="failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to extract layer sha256:25359bcca1bb70511c264e8a14c78fbc75c226344e91d19f733ed309b336949f: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/92/fs/coredns: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:28:56.463041 containerd[1567]: time="2025-08-13T01:28:56.462526319Z" level=info msg="stop pulling image registry.k8s.io/coredns/coredns:v1.12.0: active requests=0, bytes read=20942238" Aug 13 01:28:56.463099 kubelet[2720]: E0813 01:28:56.463058 2720 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:coredns,Image:registry.k8s.io/coredns/coredns:v1.12.0,Command:[],Args:[-conf /etc/coredns/Corefile],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:dns,HostPort:0,ContainerPort:53,Protocol:UDP,HostIP:,},ContainerPort{Name:dns-tcp,HostPort:0,ContainerPort:53,Protocol:TCP,HostIP:,},ContainerPort{Name:metrics,HostPort:0,ContainerPort:9153,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{memory: {{178257920 0} {} 170Mi BinarySI},},Requests:ResourceList{cpu: {{100 -3} {} 100m DecimalSI},memory: {{73400320 0} {} 70Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:config-volume,ReadOnly:true,MountPath:/etc/coredns,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-jdpj9,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{0 8080 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:60,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:5,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/ready,Port:{0 8181 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[NET_BIND_SERVICE],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8): ErrImagePull: failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to extract layer sha256:25359bcca1bb70511c264e8a14c78fbc75c226344e91d19f733ed309b336949f: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/92/fs/coredns: no space left on device" logger="UnhandledError" Aug 13 01:28:56.465059 kubelet[2720]: E0813 01:28:56.464987 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ErrImagePull: \"failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to extract layer sha256:25359bcca1bb70511c264e8a14c78fbc75c226344e91d19f733ed309b336949f: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/92/fs/coredns: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:28:56.504524 containerd[1567]: time="2025-08-13T01:28:56.504447694Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:coredns-674b8bbfcf-b2ds8,Uid:567c4932-37b3-4053-b427-2472bc4c195b,Namespace:kube-system,Attempt:0,} returns sandbox id \"a5443a34411991df2c402c02947ba5c68467b092e80e6b3ac9d701590d5420b1\"" Aug 13 01:28:56.505899 kubelet[2720]: E0813 01:28:56.505462 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:56.508060 containerd[1567]: time="2025-08-13T01:28:56.508019237Z" level=info msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\"" Aug 13 01:28:56.510312 containerd[1567]: time="2025-08-13T01:28:56.510181293Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:28:56.599105 kubelet[2720]: E0813 01:28:56.599030 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:56.602536 kubelet[2720]: E0813 01:28:56.602498 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to extract layer sha256:25359bcca1bb70511c264e8a14c78fbc75c226344e91d19f733ed309b336949f: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/92/fs/coredns: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:28:56.739317 containerd[1567]: time="2025-08-13T01:28:56.739270820Z" level=error msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\" failed" error="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-738195822-qWjD sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-1703066941: no space left on device" Aug 13 01:28:56.739480 containerd[1567]: time="2025-08-13T01:28:56.739347609Z" level=info msg="stop pulling image registry.k8s.io/coredns/coredns:v1.12.0: active requests=0, bytes read=0" Aug 13 01:28:56.739514 kubelet[2720]: E0813 01:28:56.739463 2720 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-738195822-qWjD sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-1703066941: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:28:56.739514 kubelet[2720]: E0813 01:28:56.739499 2720 kuberuntime_image.go:42] "Failed to pull image" err="failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-738195822-qWjD sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-1703066941: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:28:56.739679 kubelet[2720]: E0813 01:28:56.739600 2720 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:coredns,Image:registry.k8s.io/coredns/coredns:v1.12.0,Command:[],Args:[-conf /etc/coredns/Corefile],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:dns,HostPort:0,ContainerPort:53,Protocol:UDP,HostIP:,},ContainerPort{Name:dns-tcp,HostPort:0,ContainerPort:53,Protocol:TCP,HostIP:,},ContainerPort{Name:metrics,HostPort:0,ContainerPort:9153,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{memory: {{178257920 0} {} 170Mi BinarySI},},Requests:ResourceList{cpu: {{100 -3} {} 100m DecimalSI},memory: {{73400320 0} {} 70Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:config-volume,ReadOnly:true,MountPath:/etc/coredns,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-lncr6,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{0 8080 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:60,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:5,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/ready,Port:{0 8181 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[NET_BIND_SERVICE],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b): ErrImagePull: failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-738195822-qWjD sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-1703066941: no space left on device" logger="UnhandledError" Aug 13 01:28:56.740794 kubelet[2720]: E0813 01:28:56.740757 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ErrImagePull: \"failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-738195822-qWjD sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-1703066941: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:28:57.073957 containerd[1567]: time="2025-08-13T01:28:57.073919614Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:28:57.231537 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:28:57.231802 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.119 [INFO][6217] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 820 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.119 [INFO][6217] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.143 [INFO][6228] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" HandleID="k8s-pod-network.ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.143 [INFO][6228] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" HandleID="k8s-pod-network.ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f8d0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:28:57.143565043 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.143 [INFO][6228] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.143 [INFO][6228] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.144 [INFO][6228] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.161 [INFO][6228] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" host="172-234-25-215" Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.174 [INFO][6228] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.181 [INFO][6228] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.183 [INFO][6228] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.188 [INFO][6228] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.188 [INFO][6228] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" host="172-234-25-215" Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.190 [INFO][6228] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509 Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.202 [INFO][6228] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" host="172-234-25-215" Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.221 [INFO][6228] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.4/26] block=192.168.62.0/26 handle="k8s-pod-network.ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" host="172-234-25-215" Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.221 [INFO][6228] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.4/26] handle="k8s-pod-network.ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" host="172-234-25-215" Aug 13 01:28:57.268015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.268015 containerd[1567]: 2025-08-13 01:28:57.221 [INFO][6228] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:28:57.271674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.271674 containerd[1567]: 2025-08-13 01:28:57.221 [INFO][6228] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.4/26] IPv6=[] ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" HandleID="k8s-pod-network.ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.271674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.271674 containerd[1567]: 2025-08-13 01:28:57.223 [INFO][6217] cni-plugin/k8s.go 418: Populated endpoint ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"820", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.4/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:28:57.271674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.271674 containerd[1567]: 2025-08-13 01:28:57.224 [INFO][6217] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.4/32] ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.271674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.271674 containerd[1567]: 2025-08-13 01:28:57.224 [INFO][6217] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.271674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.271674 containerd[1567]: 2025-08-13 01:28:57.229 [INFO][6217] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.271674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.271674 containerd[1567]: 2025-08-13 01:28:57.230 [INFO][6217] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"820", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.4/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"56:29:f7:6e:1e:84", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:28:57.271674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.271674 containerd[1567]: 2025-08-13 01:28:57.263 [INFO][6217] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.272048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.322106 containerd[1567]: time="2025-08-13T01:28:57.322038785Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.322208 containerd[1567]: time="2025-08-13T01:28:57.322110124Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.322208 containerd[1567]: time="2025-08-13T01:28:57.322137524Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.322208 containerd[1567]: time="2025-08-13T01:28:57.322159194Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.322208 containerd[1567]: time="2025-08-13T01:28:57.322179094Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.327890 containerd[1567]: time="2025-08-13T01:28:57.327796562Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.329991 containerd[1567]: time="2025-08-13T01:28:57.329956439Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.344371 containerd[1567]: time="2025-08-13T01:28:57.344309320Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.344940 containerd[1567]: time="2025-08-13T01:28:57.344675679Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.355407 containerd[1567]: time="2025-08-13T01:28:57.354369009Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.355407 containerd[1567]: time="2025-08-13T01:28:57.354413789Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.355407 containerd[1567]: time="2025-08-13T01:28:57.354437489Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.355407 containerd[1567]: time="2025-08-13T01:28:57.354457769Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.355407 containerd[1567]: time="2025-08-13T01:28:57.354478399Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.355407 containerd[1567]: time="2025-08-13T01:28:57.354498499Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.355407 containerd[1567]: time="2025-08-13T01:28:57.354519849Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.355407 containerd[1567]: time="2025-08-13T01:28:57.354541679Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.354832 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:28:57.354837 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:28:57.416100 containerd[1567]: time="2025-08-13T01:28:57.416059175Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.416100 containerd[1567]: time="2025-08-13T01:28:57.416107825Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.483952 containerd[1567]: time="2025-08-13T01:28:57.483894448Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.484063 containerd[1567]: time="2025-08-13T01:28:57.483956439Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.484063 containerd[1567]: time="2025-08-13T01:28:57.483981768Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.484063 containerd[1567]: time="2025-08-13T01:28:57.484006438Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.484063 containerd[1567]: time="2025-08-13T01:28:57.484027178Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.484063 containerd[1567]: time="2025-08-13T01:28:57.484047208Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.484180 containerd[1567]: time="2025-08-13T01:28:57.484068298Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.484180 containerd[1567]: time="2025-08-13T01:28:57.484089028Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.484180 containerd[1567]: time="2025-08-13T01:28:57.484108618Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.484180 containerd[1567]: time="2025-08-13T01:28:57.484128798Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.484180 containerd[1567]: time="2025-08-13T01:28:57.484152878Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.488047 containerd[1567]: time="2025-08-13T01:28:57.488001800Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.488447 containerd[1567]: time="2025-08-13T01:28:57.488167410Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.488447 containerd[1567]: time="2025-08-13T01:28:57.488198780Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.488447 containerd[1567]: time="2025-08-13T01:28:57.488421529Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.488559 containerd[1567]: time="2025-08-13T01:28:57.488524509Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.488804 containerd[1567]: time="2025-08-13T01:28:57.488779488Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.489056 containerd[1567]: time="2025-08-13T01:28:57.488833178Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.489090 containerd[1567]: time="2025-08-13T01:28:57.489062438Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.489492 containerd[1567]: time="2025-08-13T01:28:57.489460337Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.489548 containerd[1567]: time="2025-08-13T01:28:57.489496727Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.489548 containerd[1567]: time="2025-08-13T01:28:57.489519816Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.491629 containerd[1567]: 2025-08-13 01:28:57.351 [INFO][6249] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Aug 13 01:28:57.491629 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.491629 containerd[1567]: 2025-08-13 01:28:57.351 [INFO][6249] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" iface="eth0" netns="/var/run/netns/cni-bf0b4231-fa31-c063-46e1-131a89239dce" Aug 13 01:28:57.491629 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.491629 containerd[1567]: 2025-08-13 01:28:57.352 [INFO][6249] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" iface="eth0" netns="/var/run/netns/cni-bf0b4231-fa31-c063-46e1-131a89239dce" Aug 13 01:28:57.491629 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.491629 containerd[1567]: 2025-08-13 01:28:57.364 [INFO][6249] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" after=13.198344ms iface="eth0" netns="/var/run/netns/cni-bf0b4231-fa31-c063-46e1-131a89239dce" Aug 13 01:28:57.491629 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.491629 containerd[1567]: 2025-08-13 01:28:57.365 [INFO][6249] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Aug 13 01:28:57.491629 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.491629 containerd[1567]: 2025-08-13 01:28:57.365 [INFO][6249] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Aug 13 01:28:57.491629 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.491629 containerd[1567]: 2025-08-13 01:28:57.428 [INFO][6256] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" HandleID="k8s-pod-network.ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.491629 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.491629 containerd[1567]: 2025-08-13 01:28:57.429 [INFO][6256] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:28:57.491629 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.491629 containerd[1567]: 2025-08-13 01:28:57.429 [INFO][6256] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:28:57.491629 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.491629 containerd[1567]: 2025-08-13 01:28:57.481 [INFO][6256] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" HandleID="k8s-pod-network.ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.491629 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.491629 containerd[1567]: 2025-08-13 01:28:57.481 [INFO][6256] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" HandleID="k8s-pod-network.ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.491629 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.491629 containerd[1567]: 2025-08-13 01:28:57.484 [INFO][6256] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:28:57.491629 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.491629 containerd[1567]: 2025-08-13 01:28:57.487 [INFO][6249] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509" Aug 13 01:28:57.491629 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.496127 containerd[1567]: time="2025-08-13T01:28:57.492920860Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509-lo: no space left on device" Aug 13 01:28:57.496189 kubelet[2720]: E0813 01:28:57.493212 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509-lo: no space left on device" Aug 13 01:28:57.496189 kubelet[2720]: E0813 01:28:57.493317 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:57.496189 kubelet[2720]: E0813 01:28:57.493337 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:57.496189 kubelet[2720]: E0813 01:28:57.493409 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:28:57.496767 systemd[1]: run-netns-cni\x2dbf0b4231\x2dfa31\x2dc063\x2d46e1\x2d131a89239dce.mount: Deactivated successfully. Aug 13 01:28:57.497253 systemd-networkd[1449]: cali6e0ca3c7413: Gained IPv6LL Aug 13 01:28:57.500815 containerd[1567]: time="2025-08-13T01:28:57.500486825Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.500815 containerd[1567]: time="2025-08-13T01:28:57.500538275Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.500815 containerd[1567]: time="2025-08-13T01:28:57.500561215Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.500815 containerd[1567]: time="2025-08-13T01:28:57.500582035Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.500815 containerd[1567]: time="2025-08-13T01:28:57.500601465Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.500815 containerd[1567]: time="2025-08-13T01:28:57.500621145Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.500815 containerd[1567]: time="2025-08-13T01:28:57.500641545Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.500815 containerd[1567]: time="2025-08-13T01:28:57.500661814Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.500815 containerd[1567]: time="2025-08-13T01:28:57.500682194Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.520608 containerd[1567]: time="2025-08-13T01:28:57.520575504Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.520656 containerd[1567]: time="2025-08-13T01:28:57.520614084Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.520978 containerd[1567]: time="2025-08-13T01:28:57.520926204Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.521020 containerd[1567]: time="2025-08-13T01:28:57.520988023Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.521096 containerd[1567]: time="2025-08-13T01:28:57.521042113Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.521096 containerd[1567]: time="2025-08-13T01:28:57.521070023Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.521199 containerd[1567]: time="2025-08-13T01:28:57.521110543Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.603302 kubelet[2720]: E0813 01:28:57.603206 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:28:57.605128 kubelet[2720]: E0813 01:28:57.604641 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-738195822-qWjD sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-1703066941: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:28:57.605807 containerd[1567]: time="2025-08-13T01:28:57.605773762Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:28:57.741201 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:28:57.741891 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.668 [INFO][6270] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 11434 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.671 [INFO][6270] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.703 [INFO][6282] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" HandleID="k8s-pod-network.536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.703 [INFO][6282] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" HandleID="k8s-pod-network.536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5640), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:28:57.703091337 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.703 [INFO][6282] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.703 [INFO][6282] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.703 [INFO][6282] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.710 [INFO][6282] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" host="172-234-25-215" Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.714 [INFO][6282] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.718 [INFO][6282] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.720 [INFO][6282] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.722 [INFO][6282] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.722 [INFO][6282] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" host="172-234-25-215" Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.725 [INFO][6282] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.730 [INFO][6282] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" host="172-234-25-215" Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.735 [INFO][6282] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.5/26] block=192.168.62.0/26 handle="k8s-pod-network.536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" host="172-234-25-215" Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.755968 containerd[1567]: 2025-08-13 01:28:57.735 [INFO][6282] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.5/26] handle="k8s-pod-network.536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" host="172-234-25-215" Aug 13 01:28:57.755968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.758470 containerd[1567]: 2025-08-13 01:28:57.735 [INFO][6282] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:28:57.758470 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.758470 containerd[1567]: 2025-08-13 01:28:57.735 [INFO][6282] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.5/26] IPv6=[] ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" HandleID="k8s-pod-network.536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.758470 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.758470 containerd[1567]: 2025-08-13 01:28:57.737 [INFO][6270] cni-plugin/k8s.go 418: Populated endpoint ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11434", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ce29a9ea1d30753b55d87ba8eed6188f058bd7fd25160d9cc618178fd8c6e509", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.5/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:28:57.758470 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.758470 containerd[1567]: 2025-08-13 01:28:57.738 [INFO][6270] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.5/32] ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.758470 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.758470 containerd[1567]: 2025-08-13 01:28:57.738 [INFO][6270] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.758470 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.758470 containerd[1567]: 2025-08-13 01:28:57.742 [INFO][6270] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.758470 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.758470 containerd[1567]: 2025-08-13 01:28:57.742 [INFO][6270] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11434", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.5/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"1a:9e:8a:89:5f:7b", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:28:57.758470 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.758975 containerd[1567]: 2025-08-13 01:28:57.752 [INFO][6270] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.758975 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.758975 containerd[1567]: time="2025-08-13T01:28:57.756753569Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.759325 containerd[1567]: time="2025-08-13T01:28:57.759295303Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.759943 containerd[1567]: time="2025-08-13T01:28:57.759904632Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.759985 containerd[1567]: time="2025-08-13T01:28:57.759942732Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.759985 containerd[1567]: time="2025-08-13T01:28:57.759968512Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.760166 containerd[1567]: time="2025-08-13T01:28:57.760140501Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.760972 containerd[1567]: time="2025-08-13T01:28:57.760934420Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.763419 containerd[1567]: time="2025-08-13T01:28:57.763383105Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.763466 containerd[1567]: time="2025-08-13T01:28:57.763423335Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.763466 containerd[1567]: time="2025-08-13T01:28:57.763447195Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.764075 containerd[1567]: time="2025-08-13T01:28:57.764048543Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.764226 containerd[1567]: time="2025-08-13T01:28:57.764176334Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.765243 containerd[1567]: time="2025-08-13T01:28:57.765181301Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.765348 containerd[1567]: time="2025-08-13T01:28:57.765316851Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.789760 containerd[1567]: time="2025-08-13T01:28:57.789706142Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.789819 containerd[1567]: time="2025-08-13T01:28:57.789772572Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.818836 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:28:57.818844 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:28:57.820559 containerd[1567]: time="2025-08-13T01:28:57.820520600Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.821264 containerd[1567]: time="2025-08-13T01:28:57.821012319Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.821264 containerd[1567]: time="2025-08-13T01:28:57.821056399Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.821264 containerd[1567]: time="2025-08-13T01:28:57.821080469Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.821264 containerd[1567]: time="2025-08-13T01:28:57.821107039Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.821264 containerd[1567]: time="2025-08-13T01:28:57.821127109Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.821264 containerd[1567]: time="2025-08-13T01:28:57.821161379Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.821264 containerd[1567]: time="2025-08-13T01:28:57.821182449Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.821264 containerd[1567]: time="2025-08-13T01:28:57.821204749Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.821264 containerd[1567]: time="2025-08-13T01:28:57.821223909Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.821264 containerd[1567]: time="2025-08-13T01:28:57.821243889Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.824021 containerd[1567]: time="2025-08-13T01:28:57.823971053Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.824021 containerd[1567]: time="2025-08-13T01:28:57.824006893Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.839553 containerd[1567]: time="2025-08-13T01:28:57.839522922Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.839708 containerd[1567]: time="2025-08-13T01:28:57.839664701Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.844428 containerd[1567]: time="2025-08-13T01:28:57.844371992Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.860079 containerd[1567]: time="2025-08-13T01:28:57.859347302Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.860079 containerd[1567]: time="2025-08-13T01:28:57.859428081Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.861658 containerd[1567]: time="2025-08-13T01:28:57.861509457Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.862018 containerd[1567]: time="2025-08-13T01:28:57.861764727Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.862052 containerd[1567]: time="2025-08-13T01:28:57.862032576Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.862100 containerd[1567]: time="2025-08-13T01:28:57.862067056Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.862100 containerd[1567]: time="2025-08-13T01:28:57.862090396Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.898429 containerd[1567]: 2025-08-13 01:28:57.814 [INFO][6306] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Aug 13 01:28:57.898429 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.898429 containerd[1567]: 2025-08-13 01:28:57.815 [INFO][6306] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" iface="eth0" netns="/var/run/netns/cni-a9999b90-8193-4b95-c451-3d6e38504bed" Aug 13 01:28:57.898429 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.898429 containerd[1567]: 2025-08-13 01:28:57.815 [INFO][6306] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" iface="eth0" netns="/var/run/netns/cni-a9999b90-8193-4b95-c451-3d6e38504bed" Aug 13 01:28:57.898429 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.898429 containerd[1567]: 2025-08-13 01:28:57.822 [INFO][6306] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" after=7.186966ms iface="eth0" netns="/var/run/netns/cni-a9999b90-8193-4b95-c451-3d6e38504bed" Aug 13 01:28:57.898429 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.898429 containerd[1567]: 2025-08-13 01:28:57.822 [INFO][6306] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Aug 13 01:28:57.898429 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.898429 containerd[1567]: 2025-08-13 01:28:57.823 [INFO][6306] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Aug 13 01:28:57.898429 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.898429 containerd[1567]: 2025-08-13 01:28:57.860 [INFO][6316] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" HandleID="k8s-pod-network.536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.898429 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.898429 containerd[1567]: 2025-08-13 01:28:57.861 [INFO][6316] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:28:57.898429 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.898429 containerd[1567]: 2025-08-13 01:28:57.861 [INFO][6316] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:28:57.898429 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.898429 containerd[1567]: 2025-08-13 01:28:57.891 [INFO][6316] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" HandleID="k8s-pod-network.536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.898429 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.898429 containerd[1567]: 2025-08-13 01:28:57.891 [INFO][6316] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" HandleID="k8s-pod-network.536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:57.898429 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.898429 containerd[1567]: 2025-08-13 01:28:57.893 [INFO][6316] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:28:57.898429 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.898429 containerd[1567]: 2025-08-13 01:28:57.896 [INFO][6306] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e" Aug 13 01:28:57.898429 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:57.900612 systemd[1]: run-netns-cni\x2da9999b90\x2d8193\x2d4b95\x2dc451\x2d3d6e38504bed.mount: Deactivated successfully. Aug 13 01:28:57.903669 containerd[1567]: time="2025-08-13T01:28:57.903626072Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e-lo: no space left on device" Aug 13 01:28:57.904269 kubelet[2720]: E0813 01:28:57.904163 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e-lo: no space left on device" Aug 13 01:28:57.904597 kubelet[2720]: E0813 01:28:57.904496 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:57.905032 kubelet[2720]: E0813 01:28:57.904805 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:57.906148 kubelet[2720]: E0813 01:28:57.905807 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:28:57.918521 containerd[1567]: time="2025-08-13T01:28:57.918463053Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.918935 containerd[1567]: time="2025-08-13T01:28:57.918907222Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.919772 containerd[1567]: time="2025-08-13T01:28:57.919677170Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.920766 containerd[1567]: time="2025-08-13T01:28:57.919895420Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.920882 containerd[1567]: time="2025-08-13T01:28:57.920863378Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.920994 containerd[1567]: time="2025-08-13T01:28:57.920964138Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.921077 containerd[1567]: time="2025-08-13T01:28:57.921061198Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:57.921184 containerd[1567]: time="2025-08-13T01:28:57.921164047Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.029812 kubelet[2720]: I0813 01:28:58.029741 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:28:58.029812 kubelet[2720]: I0813 01:28:58.029788 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:28:58.031693 kubelet[2720]: I0813 01:28:58.031675 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:28:58.044048 kubelet[2720]: I0813 01:28:58.044027 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:28:58.044172 kubelet[2720]: I0813 01:28:58.044153 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-kube-controllers-b57d47f84-lznkx","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:28:58.044258 kubelet[2720]: E0813 01:28:58.044182 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:28:58.044258 kubelet[2720]: E0813 01:28:58.044192 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:28:58.044258 kubelet[2720]: E0813 01:28:58.044198 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:58.044258 kubelet[2720]: E0813 01:28:58.044207 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:28:58.044258 kubelet[2720]: E0813 01:28:58.044216 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:28:58.044258 kubelet[2720]: E0813 01:28:58.044223 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:28:58.044258 kubelet[2720]: E0813 01:28:58.044230 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:28:58.044258 kubelet[2720]: E0813 01:28:58.044240 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:28:58.044258 kubelet[2720]: E0813 01:28:58.044248 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:28:58.044258 kubelet[2720]: E0813 01:28:58.044256 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:28:58.044258 kubelet[2720]: I0813 01:28:58.044266 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:28:58.604994 containerd[1567]: time="2025-08-13T01:28:58.604920973Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:28:58.710230 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:28:58.712221 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.640 [INFO][6331] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 11465 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.640 [INFO][6331] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.673 [INFO][6345] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" HandleID="k8s-pod-network.099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.674 [INFO][6345] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" HandleID="k8s-pod-network.099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:28:58.673555575 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.674 [INFO][6345] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.674 [INFO][6345] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.674 [INFO][6345] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.679 [INFO][6345] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" host="172-234-25-215" Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.683 [INFO][6345] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.687 [INFO][6345] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.689 [INFO][6345] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.691 [INFO][6345] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.691 [INFO][6345] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" host="172-234-25-215" Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.692 [INFO][6345] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321 Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.697 [INFO][6345] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" host="172-234-25-215" Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.702 [INFO][6345] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.6/26] block=192.168.62.0/26 handle="k8s-pod-network.099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" host="172-234-25-215" Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.726874 containerd[1567]: 2025-08-13 01:28:58.702 [INFO][6345] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.6/26] handle="k8s-pod-network.099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" host="172-234-25-215" Aug 13 01:28:58.726874 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.728515 containerd[1567]: 2025-08-13 01:28:58.702 [INFO][6345] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:28:58.728515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.728515 containerd[1567]: 2025-08-13 01:28:58.703 [INFO][6345] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.6/26] IPv6=[] ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" HandleID="k8s-pod-network.099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:58.728515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.728515 containerd[1567]: 2025-08-13 01:28:58.705 [INFO][6331] cni-plugin/k8s.go 418: Populated endpoint ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11465", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"536d5bf2d443fc76d44a20739a5cb4e0c3aefd7b40a1fdfc671fae236729090e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.6/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:28:58.728515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.728515 containerd[1567]: 2025-08-13 01:28:58.706 [INFO][6331] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.6/32] ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:58.728515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.728515 containerd[1567]: 2025-08-13 01:28:58.706 [INFO][6331] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:58.728515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.728515 containerd[1567]: 2025-08-13 01:28:58.715 [INFO][6331] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:58.728515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.728515 containerd[1567]: 2025-08-13 01:28:58.715 [INFO][6331] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11465", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.6/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"f2:11:02:ca:0c:48", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:28:58.728515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.729584 containerd[1567]: 2025-08-13 01:28:58.723 [INFO][6331] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:58.729584 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.729584 containerd[1567]: time="2025-08-13T01:28:58.728019466Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.729897 containerd[1567]: time="2025-08-13T01:28:58.729312123Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.730088 containerd[1567]: time="2025-08-13T01:28:58.730066121Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.730204 containerd[1567]: time="2025-08-13T01:28:58.730186031Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.730433 containerd[1567]: time="2025-08-13T01:28:58.730366631Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.730433 containerd[1567]: time="2025-08-13T01:28:58.730395631Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.730433 containerd[1567]: time="2025-08-13T01:28:58.730418321Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.731966 containerd[1567]: time="2025-08-13T01:28:58.731932668Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.732248 containerd[1567]: time="2025-08-13T01:28:58.732139997Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.732248 containerd[1567]: time="2025-08-13T01:28:58.732169647Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.732248 containerd[1567]: time="2025-08-13T01:28:58.732191327Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.732248 containerd[1567]: time="2025-08-13T01:28:58.732211217Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.732502 containerd[1567]: time="2025-08-13T01:28:58.732450256Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.732571 containerd[1567]: time="2025-08-13T01:28:58.732483536Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.779060 containerd[1567]: time="2025-08-13T01:28:58.778939284Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.779060 containerd[1567]: time="2025-08-13T01:28:58.778989004Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.788412 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:28:58.792973 containerd[1567]: time="2025-08-13T01:28:58.792631066Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.788419 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:28:58.803091 containerd[1567]: time="2025-08-13T01:28:58.802843895Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.803091 containerd[1567]: time="2025-08-13T01:28:58.802878466Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.803091 containerd[1567]: time="2025-08-13T01:28:58.802902216Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.803091 containerd[1567]: time="2025-08-13T01:28:58.802926186Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.803091 containerd[1567]: time="2025-08-13T01:28:58.802949566Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.803091 containerd[1567]: time="2025-08-13T01:28:58.802970316Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.803091 containerd[1567]: time="2025-08-13T01:28:58.802991725Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.803091 containerd[1567]: time="2025-08-13T01:28:58.803012215Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.803091 containerd[1567]: time="2025-08-13T01:28:58.803032455Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.803091 containerd[1567]: time="2025-08-13T01:28:58.803051965Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.803091 containerd[1567]: time="2025-08-13T01:28:58.803071295Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.817816 containerd[1567]: time="2025-08-13T01:28:58.817368536Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.817816 containerd[1567]: time="2025-08-13T01:28:58.817404286Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.817816 containerd[1567]: time="2025-08-13T01:28:58.817427046Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.835506 containerd[1567]: time="2025-08-13T01:28:58.835484050Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.835628 containerd[1567]: time="2025-08-13T01:28:58.835606780Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.835874 containerd[1567]: time="2025-08-13T01:28:58.835842149Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.837337 containerd[1567]: time="2025-08-13T01:28:58.836785987Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.837337 containerd[1567]: time="2025-08-13T01:28:58.836832167Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.837337 containerd[1567]: time="2025-08-13T01:28:58.836859446Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.837337 containerd[1567]: time="2025-08-13T01:28:58.836881686Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.837337 containerd[1567]: time="2025-08-13T01:28:58.836901777Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.884759 containerd[1567]: 2025-08-13 01:28:58.785 [INFO][6372] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Aug 13 01:28:58.884759 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.884759 containerd[1567]: 2025-08-13 01:28:58.785 [INFO][6372] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" iface="eth0" netns="/var/run/netns/cni-3ed536c6-101f-8ef3-eade-9ff34175d090" Aug 13 01:28:58.884759 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.884759 containerd[1567]: 2025-08-13 01:28:58.786 [INFO][6372] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" iface="eth0" netns="/var/run/netns/cni-3ed536c6-101f-8ef3-eade-9ff34175d090" Aug 13 01:28:58.884759 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.884759 containerd[1567]: 2025-08-13 01:28:58.794 [INFO][6372] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" after=8.630533ms iface="eth0" netns="/var/run/netns/cni-3ed536c6-101f-8ef3-eade-9ff34175d090" Aug 13 01:28:58.884759 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.884759 containerd[1567]: 2025-08-13 01:28:58.794 [INFO][6372] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Aug 13 01:28:58.884759 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.884759 containerd[1567]: 2025-08-13 01:28:58.794 [INFO][6372] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Aug 13 01:28:58.884759 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.884759 containerd[1567]: 2025-08-13 01:28:58.838 [INFO][6380] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" HandleID="k8s-pod-network.099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:58.884759 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.884759 containerd[1567]: 2025-08-13 01:28:58.839 [INFO][6380] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:28:58.884759 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.884759 containerd[1567]: 2025-08-13 01:28:58.839 [INFO][6380] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:28:58.884759 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.884759 containerd[1567]: 2025-08-13 01:28:58.878 [INFO][6380] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" HandleID="k8s-pod-network.099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:58.884759 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.884759 containerd[1567]: 2025-08-13 01:28:58.878 [INFO][6380] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" HandleID="k8s-pod-network.099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:58.884759 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.884759 containerd[1567]: 2025-08-13 01:28:58.879 [INFO][6380] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:28:58.884759 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.884759 containerd[1567]: 2025-08-13 01:28:58.882 [INFO][6372] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321" Aug 13 01:28:58.884759 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:58.887647 systemd[1]: run-netns-cni\x2d3ed536c6\x2d101f\x2d8ef3\x2deade\x2d9ff34175d090.mount: Deactivated successfully. Aug 13 01:28:58.888778 containerd[1567]: time="2025-08-13T01:28:58.888633593Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321-lo: no space left on device" Aug 13 01:28:58.889402 kubelet[2720]: E0813 01:28:58.889115 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321-lo: no space left on device" Aug 13 01:28:58.889402 kubelet[2720]: E0813 01:28:58.889204 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:58.889402 kubelet[2720]: E0813 01:28:58.889228 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:58.889402 kubelet[2720]: E0813 01:28:58.889295 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:28:58.893016 containerd[1567]: time="2025-08-13T01:28:58.892979385Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.893207 containerd[1567]: time="2025-08-13T01:28:58.893174354Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.893312 containerd[1567]: time="2025-08-13T01:28:58.893294214Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.893382 containerd[1567]: time="2025-08-13T01:28:58.893367014Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.893457 containerd[1567]: time="2025-08-13T01:28:58.893442764Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.893605 containerd[1567]: time="2025-08-13T01:28:58.893588723Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.893691 containerd[1567]: time="2025-08-13T01:28:58.893675413Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:58.893794 containerd[1567]: time="2025-08-13T01:28:58.893777483Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.477539 containerd[1567]: time="2025-08-13T01:28:59.477482263Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:28:59.607292 containerd[1567]: time="2025-08-13T01:28:59.607253742Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:28:59.716310 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:28:59.718964 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:28:59.732369 containerd[1567]: time="2025-08-13T01:28:59.731965213Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.732683 containerd[1567]: time="2025-08-13T01:28:59.732522471Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.732893 containerd[1567]: time="2025-08-13T01:28:59.732873920Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.733090 containerd[1567]: time="2025-08-13T01:28:59.733034100Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.733394 containerd[1567]: time="2025-08-13T01:28:59.733254599Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.733394 containerd[1567]: time="2025-08-13T01:28:59.733283929Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.733599 containerd[1567]: time="2025-08-13T01:28:59.733493029Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.733830 containerd[1567]: time="2025-08-13T01:28:59.733767859Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.733989 containerd[1567]: time="2025-08-13T01:28:59.733934759Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.639 [INFO][6390] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 11538 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.639 [INFO][6390] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.673 [INFO][6405] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" HandleID="k8s-pod-network.70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.673 [INFO][6405] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" HandleID="k8s-pod-network.70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5640), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:28:59.673456459 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.673 [INFO][6405] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.673 [INFO][6405] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.674 [INFO][6405] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.680 [INFO][6405] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" host="172-234-25-215" Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.684 [INFO][6405] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.689 [INFO][6405] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.690 [INFO][6405] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.692 [INFO][6405] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.692 [INFO][6405] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" host="172-234-25-215" Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.693 [INFO][6405] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8 Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.699 [INFO][6405] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" host="172-234-25-215" Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.706 [INFO][6405] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.7/26] block=192.168.62.0/26 handle="k8s-pod-network.70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" host="172-234-25-215" Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.734531 containerd[1567]: 2025-08-13 01:28:59.706 [INFO][6405] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.7/26] handle="k8s-pod-network.70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" host="172-234-25-215" Aug 13 01:28:59.734531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.735451 containerd[1567]: 2025-08-13 01:28:59.706 [INFO][6405] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:28:59.735451 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.735451 containerd[1567]: 2025-08-13 01:28:59.706 [INFO][6405] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.7/26] IPv6=[] ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" HandleID="k8s-pod-network.70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:59.735451 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.735451 containerd[1567]: 2025-08-13 01:28:59.709 [INFO][6390] cni-plugin/k8s.go 418: Populated endpoint ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11538", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"099bb0abb30f2288d4b4e3be24a4d5040a93b62698e25cddb3a2d41a7bb38321", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.7/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:28:59.735451 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.735451 containerd[1567]: 2025-08-13 01:28:59.709 [INFO][6390] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.7/32] ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:59.735451 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.735451 containerd[1567]: 2025-08-13 01:28:59.709 [INFO][6390] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:59.735451 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.735451 containerd[1567]: 2025-08-13 01:28:59.719 [INFO][6390] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:59.735451 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.735451 containerd[1567]: 2025-08-13 01:28:59.720 [INFO][6390] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11538", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.7/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"52:86:03:4f:99:1c", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:28:59.735451 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.737975 containerd[1567]: 2025-08-13 01:28:59.727 [INFO][6390] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:59.737975 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.737975 containerd[1567]: time="2025-08-13T01:28:59.734143448Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.737975 containerd[1567]: time="2025-08-13T01:28:59.734214358Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.737975 containerd[1567]: time="2025-08-13T01:28:59.734342868Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.737975 containerd[1567]: time="2025-08-13T01:28:59.734369948Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.737975 containerd[1567]: time="2025-08-13T01:28:59.734390788Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.782563 containerd[1567]: time="2025-08-13T01:28:59.782480831Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.782563 containerd[1567]: time="2025-08-13T01:28:59.782532591Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.807670 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:28:59.807679 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:28:59.810538 containerd[1567]: time="2025-08-13T01:28:59.809594617Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.812014 containerd[1567]: time="2025-08-13T01:28:59.811981642Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.812314 containerd[1567]: time="2025-08-13T01:28:59.812296212Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.812649 containerd[1567]: time="2025-08-13T01:28:59.812547361Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.812943 containerd[1567]: time="2025-08-13T01:28:59.812870050Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.814587 containerd[1567]: time="2025-08-13T01:28:59.814469897Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.819264 containerd[1567]: time="2025-08-13T01:28:59.819233347Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.819764 containerd[1567]: time="2025-08-13T01:28:59.819362037Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.819764 containerd[1567]: time="2025-08-13T01:28:59.819497937Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.819764 containerd[1567]: time="2025-08-13T01:28:59.819530416Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.819764 containerd[1567]: time="2025-08-13T01:28:59.819574777Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.819764 containerd[1567]: time="2025-08-13T01:28:59.819601267Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.819764 containerd[1567]: time="2025-08-13T01:28:59.819642477Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.819764 containerd[1567]: time="2025-08-13T01:28:59.819668467Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.819764 containerd[1567]: time="2025-08-13T01:28:59.819689937Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.819764 containerd[1567]: time="2025-08-13T01:28:59.819745227Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.867467 containerd[1567]: time="2025-08-13T01:28:59.867413730Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.867467 containerd[1567]: time="2025-08-13T01:28:59.867465700Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.868109 containerd[1567]: time="2025-08-13T01:28:59.867491510Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.868397 containerd[1567]: time="2025-08-13T01:28:59.868369339Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.868574 containerd[1567]: time="2025-08-13T01:28:59.868484249Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.869274 containerd[1567]: time="2025-08-13T01:28:59.868833368Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.869417 containerd[1567]: time="2025-08-13T01:28:59.869377777Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.879968 containerd[1567]: 2025-08-13 01:28:59.803 [INFO][6428] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Aug 13 01:28:59.879968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.879968 containerd[1567]: 2025-08-13 01:28:59.804 [INFO][6428] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" iface="eth0" netns="/var/run/netns/cni-48e678c1-1986-e785-a32c-3663984e9b6c" Aug 13 01:28:59.879968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.879968 containerd[1567]: 2025-08-13 01:28:59.805 [INFO][6428] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" iface="eth0" netns="/var/run/netns/cni-48e678c1-1986-e785-a32c-3663984e9b6c" Aug 13 01:28:59.879968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.879968 containerd[1567]: 2025-08-13 01:28:59.815 [INFO][6428] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" after=11.491228ms iface="eth0" netns="/var/run/netns/cni-48e678c1-1986-e785-a32c-3663984e9b6c" Aug 13 01:28:59.879968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.879968 containerd[1567]: 2025-08-13 01:28:59.815 [INFO][6428] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Aug 13 01:28:59.879968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.879968 containerd[1567]: 2025-08-13 01:28:59.815 [INFO][6428] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Aug 13 01:28:59.879968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.879968 containerd[1567]: 2025-08-13 01:28:59.843 [INFO][6438] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" HandleID="k8s-pod-network.70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:59.879968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.879968 containerd[1567]: 2025-08-13 01:28:59.843 [INFO][6438] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:28:59.879968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.879968 containerd[1567]: 2025-08-13 01:28:59.843 [INFO][6438] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:28:59.879968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.879968 containerd[1567]: 2025-08-13 01:28:59.873 [INFO][6438] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" HandleID="k8s-pod-network.70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:59.879968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.879968 containerd[1567]: 2025-08-13 01:28:59.873 [INFO][6438] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" HandleID="k8s-pod-network.70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:28:59.879968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.879968 containerd[1567]: 2025-08-13 01:28:59.875 [INFO][6438] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:28:59.879968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.879968 containerd[1567]: 2025-08-13 01:28:59.877 [INFO][6428] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8" Aug 13 01:28:59.879968 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:28:59.883538 containerd[1567]: time="2025-08-13T01:28:59.883500198Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8-lo: no space left on device" Aug 13 01:28:59.883858 systemd[1]: run-netns-cni\x2d48e678c1\x2d1986\x2de785\x2da32c\x2d3663984e9b6c.mount: Deactivated successfully. Aug 13 01:28:59.883939 kubelet[2720]: E0813 01:28:59.883817 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8-lo: no space left on device" Aug 13 01:28:59.883939 kubelet[2720]: E0813 01:28:59.883891 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:59.883939 kubelet[2720]: E0813 01:28:59.883929 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:28:59.884435 kubelet[2720]: E0813 01:28:59.884394 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:28:59.910605 containerd[1567]: time="2025-08-13T01:28:59.910496965Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.910874 containerd[1567]: time="2025-08-13T01:28:59.910818834Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.911020 containerd[1567]: time="2025-08-13T01:28:59.910986733Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.913083 containerd[1567]: time="2025-08-13T01:28:59.912969019Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.913083 containerd[1567]: time="2025-08-13T01:28:59.913001559Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.913083 containerd[1567]: time="2025-08-13T01:28:59.913024209Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.913083 containerd[1567]: time="2025-08-13T01:28:59.913044609Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:28:59.913083 containerd[1567]: time="2025-08-13T01:28:59.913064719Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.349171 containerd[1567]: time="2025-08-13T01:29:00.349110557Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.608677 containerd[1567]: time="2025-08-13T01:29:00.608553659Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:00.722411 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:00.723260 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:00.738678 containerd[1567]: time="2025-08-13T01:29:00.738070220Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.738678 containerd[1567]: time="2025-08-13T01:29:00.738115260Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.738678 containerd[1567]: time="2025-08-13T01:29:00.738143699Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.738678 containerd[1567]: time="2025-08-13T01:29:00.738164619Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.738998 containerd[1567]: time="2025-08-13T01:29:00.738976568Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.739081 containerd[1567]: time="2025-08-13T01:29:00.739064928Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.744575 containerd[1567]: time="2025-08-13T01:29:00.743953088Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.744575 containerd[1567]: time="2025-08-13T01:29:00.744011968Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.744575 containerd[1567]: time="2025-08-13T01:29:00.744038098Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.744575 containerd[1567]: time="2025-08-13T01:29:00.744063158Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.744575 containerd[1567]: time="2025-08-13T01:29:00.744084158Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.744575 containerd[1567]: time="2025-08-13T01:29:00.744103608Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.744575 containerd[1567]: time="2025-08-13T01:29:00.744127058Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.744575 containerd[1567]: time="2025-08-13T01:29:00.744152017Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.644 [INFO][6448] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 11608 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.644 [INFO][6448] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.678 [INFO][6461] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" HandleID="k8s-pod-network.517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.678 [INFO][6461] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" HandleID="k8s-pod-network.517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5600), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:00.678525449 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.678 [INFO][6461] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.678 [INFO][6461] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.679 [INFO][6461] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.686 [INFO][6461] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" host="172-234-25-215" Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.690 [INFO][6461] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.697 [INFO][6461] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.699 [INFO][6461] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.702 [INFO][6461] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.702 [INFO][6461] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" host="172-234-25-215" Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.705 [INFO][6461] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.709 [INFO][6461] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" host="172-234-25-215" Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.713 [INFO][6461] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.8/26] block=192.168.62.0/26 handle="k8s-pod-network.517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" host="172-234-25-215" Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.746278 containerd[1567]: 2025-08-13 01:29:00.714 [INFO][6461] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.8/26] handle="k8s-pod-network.517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" host="172-234-25-215" Aug 13 01:29:00.746278 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.747703 containerd[1567]: 2025-08-13 01:29:00.714 [INFO][6461] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:00.747703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.747703 containerd[1567]: 2025-08-13 01:29:00.714 [INFO][6461] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.8/26] IPv6=[] ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" HandleID="k8s-pod-network.517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:00.747703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.747703 containerd[1567]: 2025-08-13 01:29:00.718 [INFO][6448] cni-plugin/k8s.go 418: Populated endpoint ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11608", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"70f2943454ffa9c0ceb3e091ab487409a35b82e2059fdad45baf621a502208d8", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.8/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:00.747703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.747703 containerd[1567]: 2025-08-13 01:29:00.718 [INFO][6448] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.8/32] ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:00.747703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.747703 containerd[1567]: 2025-08-13 01:29:00.718 [INFO][6448] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:00.747703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.747703 containerd[1567]: 2025-08-13 01:29:00.722 [INFO][6448] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:00.747703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.747703 containerd[1567]: 2025-08-13 01:29:00.723 [INFO][6448] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11608", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.8/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"46:0e:35:45:c8:35", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:00.747703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.748042 containerd[1567]: 2025-08-13 01:29:00.736 [INFO][6448] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:00.748042 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.778861 containerd[1567]: time="2025-08-13T01:29:00.778822709Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.779111 containerd[1567]: time="2025-08-13T01:29:00.778873269Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.811752 containerd[1567]: time="2025-08-13T01:29:00.811340004Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.811449 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:00.811454 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:00.816970 containerd[1567]: time="2025-08-13T01:29:00.815614585Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.816970 containerd[1567]: time="2025-08-13T01:29:00.815675105Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.816970 containerd[1567]: time="2025-08-13T01:29:00.815700885Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.816970 containerd[1567]: time="2025-08-13T01:29:00.815769265Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.816970 containerd[1567]: time="2025-08-13T01:29:00.815791534Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.816970 containerd[1567]: time="2025-08-13T01:29:00.815811844Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.816970 containerd[1567]: time="2025-08-13T01:29:00.815831234Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.816970 containerd[1567]: time="2025-08-13T01:29:00.815850794Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.816970 containerd[1567]: time="2025-08-13T01:29:00.815870394Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.816970 containerd[1567]: time="2025-08-13T01:29:00.815892394Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.816970 containerd[1567]: time="2025-08-13T01:29:00.816103904Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.817876 containerd[1567]: time="2025-08-13T01:29:00.817855360Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.825724 containerd[1567]: time="2025-08-13T01:29:00.824909797Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.825724 containerd[1567]: time="2025-08-13T01:29:00.824948007Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.825724 containerd[1567]: time="2025-08-13T01:29:00.824971147Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.861788 containerd[1567]: time="2025-08-13T01:29:00.861393514Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.861974 containerd[1567]: time="2025-08-13T01:29:00.861885592Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.861974 containerd[1567]: time="2025-08-13T01:29:00.861919642Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.861974 containerd[1567]: time="2025-08-13T01:29:00.861942292Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.862563 containerd[1567]: time="2025-08-13T01:29:00.862220851Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.862930 containerd[1567]: time="2025-08-13T01:29:00.862680131Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.863755 containerd[1567]: time="2025-08-13T01:29:00.863734119Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.912246 containerd[1567]: time="2025-08-13T01:29:00.911952262Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.912829 containerd[1567]: time="2025-08-13T01:29:00.912788901Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.912940 containerd[1567]: time="2025-08-13T01:29:00.912918581Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.913115 containerd[1567]: time="2025-08-13T01:29:00.913014411Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.913115 containerd[1567]: time="2025-08-13T01:29:00.913043651Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.913115 containerd[1567]: time="2025-08-13T01:29:00.913065600Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.913115 containerd[1567]: time="2025-08-13T01:29:00.913085840Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.913452 containerd[1567]: time="2025-08-13T01:29:00.913434900Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:00.915880 containerd[1567]: 2025-08-13 01:29:00.808 [INFO][6485] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Aug 13 01:29:00.915880 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.915880 containerd[1567]: 2025-08-13 01:29:00.809 [INFO][6485] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" iface="eth0" netns="/var/run/netns/cni-50acb9eb-9d5b-77dc-2999-3b070aba9ac7" Aug 13 01:29:00.915880 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.915880 containerd[1567]: 2025-08-13 01:29:00.809 [INFO][6485] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" iface="eth0" netns="/var/run/netns/cni-50acb9eb-9d5b-77dc-2999-3b070aba9ac7" Aug 13 01:29:00.915880 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.915880 containerd[1567]: 2025-08-13 01:29:00.817 [INFO][6485] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" after=7.823335ms iface="eth0" netns="/var/run/netns/cni-50acb9eb-9d5b-77dc-2999-3b070aba9ac7" Aug 13 01:29:00.915880 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.915880 containerd[1567]: 2025-08-13 01:29:00.817 [INFO][6485] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Aug 13 01:29:00.915880 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.915880 containerd[1567]: 2025-08-13 01:29:00.818 [INFO][6485] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Aug 13 01:29:00.915880 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.915880 containerd[1567]: 2025-08-13 01:29:00.855 [INFO][6493] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" HandleID="k8s-pod-network.517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:00.915880 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.915880 containerd[1567]: 2025-08-13 01:29:00.856 [INFO][6493] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:00.915880 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.915880 containerd[1567]: 2025-08-13 01:29:00.857 [INFO][6493] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:00.915880 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.915880 containerd[1567]: 2025-08-13 01:29:00.905 [INFO][6493] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" HandleID="k8s-pod-network.517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:00.915880 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.915880 containerd[1567]: 2025-08-13 01:29:00.905 [INFO][6493] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" HandleID="k8s-pod-network.517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:00.915880 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.915880 containerd[1567]: 2025-08-13 01:29:00.907 [INFO][6493] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:00.915880 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.915880 containerd[1567]: 2025-08-13 01:29:00.913 [INFO][6485] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c" Aug 13 01:29:00.915880 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:00.920394 containerd[1567]: time="2025-08-13T01:29:00.919675717Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c-lo: no space left on device" Aug 13 01:29:00.920176 systemd[1]: run-netns-cni\x2d50acb9eb\x2d9d5b\x2d77dc\x2d2999\x2d3b070aba9ac7.mount: Deactivated successfully. Aug 13 01:29:00.921878 kubelet[2720]: E0813 01:29:00.921822 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c-lo: no space left on device" Aug 13 01:29:00.922130 kubelet[2720]: E0813 01:29:00.921891 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:00.922130 kubelet[2720]: E0813 01:29:00.921914 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:00.922130 kubelet[2720]: E0813 01:29:00.921974 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:01.118893 systemd[1]: Started sshd@26-172.234.25.215:22-147.75.109.163:53580.service - OpenSSH per-connection server daemon (147.75.109.163:53580). Aug 13 01:29:01.467533 sshd[6504]: Accepted publickey for core from 147.75.109.163 port 53580 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:29:01.469634 sshd-session[6504]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:29:01.476577 systemd-logind[1543]: New session 27 of user core. Aug 13 01:29:01.482874 systemd[1]: Started session-27.scope - Session 27 of User core. Aug 13 01:29:01.611087 containerd[1567]: time="2025-08-13T01:29:01.610805309Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:01.756787 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:01.760637 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.648 [INFO][6508] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 11675 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.648 [INFO][6508] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.689 [INFO][6521] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" HandleID="k8s-pod-network.8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.690 [INFO][6521] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" HandleID="k8s-pod-network.8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002c51e0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:01.689252483 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.690 [INFO][6521] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.690 [INFO][6521] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.690 [INFO][6521] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.699 [INFO][6521] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" host="172-234-25-215" Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.707 [INFO][6521] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.714 [INFO][6521] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.716 [INFO][6521] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.720 [INFO][6521] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.720 [INFO][6521] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" host="172-234-25-215" Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.722 [INFO][6521] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658 Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.730 [INFO][6521] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" host="172-234-25-215" Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.740 [INFO][6521] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.9/26] block=192.168.62.0/26 handle="k8s-pod-network.8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" host="172-234-25-215" Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.784765 containerd[1567]: 2025-08-13 01:29:01.740 [INFO][6521] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.9/26] handle="k8s-pod-network.8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" host="172-234-25-215" Aug 13 01:29:01.784765 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.785462 containerd[1567]: 2025-08-13 01:29:01.740 [INFO][6521] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:01.785462 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.785462 containerd[1567]: 2025-08-13 01:29:01.740 [INFO][6521] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.9/26] IPv6=[] ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" HandleID="k8s-pod-network.8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:01.785462 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.785462 containerd[1567]: 2025-08-13 01:29:01.745 [INFO][6508] cni-plugin/k8s.go 418: Populated endpoint ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11675", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"517c24ad0da9ff651784d12b6f643d4cb5c78640b6e99fe0b9d8c46a98fd6c7c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.9/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:01.785462 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.785462 containerd[1567]: 2025-08-13 01:29:01.746 [INFO][6508] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.9/32] ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:01.785462 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.785462 containerd[1567]: 2025-08-13 01:29:01.747 [INFO][6508] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:01.785462 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.785462 containerd[1567]: 2025-08-13 01:29:01.761 [INFO][6508] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:01.785462 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.785462 containerd[1567]: 2025-08-13 01:29:01.763 [INFO][6508] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11675", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.9/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"6e:69:62:19:b0:61", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:01.785462 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.786351 containerd[1567]: 2025-08-13 01:29:01.781 [INFO][6508] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:01.786351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.786351 containerd[1567]: time="2025-08-13T01:29:01.784895171Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.786351 containerd[1567]: time="2025-08-13T01:29:01.785171121Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.786351 containerd[1567]: time="2025-08-13T01:29:01.785235551Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.788361 containerd[1567]: time="2025-08-13T01:29:01.787747326Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.788361 containerd[1567]: time="2025-08-13T01:29:01.787800776Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.788361 containerd[1567]: time="2025-08-13T01:29:01.787825566Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.788361 containerd[1567]: time="2025-08-13T01:29:01.787846616Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.798899 containerd[1567]: time="2025-08-13T01:29:01.798060305Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.798899 containerd[1567]: time="2025-08-13T01:29:01.798213155Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.798899 containerd[1567]: time="2025-08-13T01:29:01.798239755Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.798899 containerd[1567]: time="2025-08-13T01:29:01.798260865Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.798899 containerd[1567]: time="2025-08-13T01:29:01.798284765Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.798899 containerd[1567]: time="2025-08-13T01:29:01.798307195Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.798899 containerd[1567]: time="2025-08-13T01:29:01.798327255Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.826793 sshd[6506]: Connection closed by 147.75.109.163 port 53580 Aug 13 01:29:01.827906 sshd-session[6504]: pam_unix(sshd:session): session closed for user core Aug 13 01:29:01.837530 systemd[1]: sshd@26-172.234.25.215:22-147.75.109.163:53580.service: Deactivated successfully. Aug 13 01:29:01.838221 systemd-logind[1543]: Session 27 logged out. Waiting for processes to exit. Aug 13 01:29:01.844470 systemd[1]: session-27.scope: Deactivated successfully. Aug 13 01:29:01.850764 systemd-logind[1543]: Removed session 27. Aug 13 01:29:01.854440 containerd[1567]: time="2025-08-13T01:29:01.853856084Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.854440 containerd[1567]: time="2025-08-13T01:29:01.853908734Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.864509 containerd[1567]: time="2025-08-13T01:29:01.864368483Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.865083 containerd[1567]: time="2025-08-13T01:29:01.864786362Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.865232 containerd[1567]: time="2025-08-13T01:29:01.865201502Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.866631 containerd[1567]: time="2025-08-13T01:29:01.866592958Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.868098 containerd[1567]: time="2025-08-13T01:29:01.867702057Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.868405 containerd[1567]: time="2025-08-13T01:29:01.868319475Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.868681 containerd[1567]: time="2025-08-13T01:29:01.868629924Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.869098 containerd[1567]: time="2025-08-13T01:29:01.868947373Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.869262 containerd[1567]: time="2025-08-13T01:29:01.869238584Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.869649 containerd[1567]: time="2025-08-13T01:29:01.869439843Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.878277 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:01.878283 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:01.882507 containerd[1567]: time="2025-08-13T01:29:01.882205467Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.882507 containerd[1567]: time="2025-08-13T01:29:01.882239027Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.882507 containerd[1567]: time="2025-08-13T01:29:01.882261167Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.882507 containerd[1567]: time="2025-08-13T01:29:01.882284267Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.882507 containerd[1567]: time="2025-08-13T01:29:01.882305157Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.882507 containerd[1567]: time="2025-08-13T01:29:01.882325177Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.882507 containerd[1567]: time="2025-08-13T01:29:01.882347407Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.882507 containerd[1567]: time="2025-08-13T01:29:01.882366917Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.882507 containerd[1567]: time="2025-08-13T01:29:01.882386767Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.882507 containerd[1567]: time="2025-08-13T01:29:01.882407677Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.882507 containerd[1567]: time="2025-08-13T01:29:01.882428287Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.882507 containerd[1567]: time="2025-08-13T01:29:01.882448797Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.882507 containerd[1567]: time="2025-08-13T01:29:01.882468207Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.882507 containerd[1567]: time="2025-08-13T01:29:01.882487407Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:01.963416 containerd[1567]: 2025-08-13 01:29:01.874 [INFO][6550] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Aug 13 01:29:01.963416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.963416 containerd[1567]: 2025-08-13 01:29:01.874 [INFO][6550] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" iface="eth0" netns="/var/run/netns/cni-f9b5a5d1-27f5-720e-4d1f-7eb833875c4b" Aug 13 01:29:01.963416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.963416 containerd[1567]: 2025-08-13 01:29:01.877 [INFO][6550] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" iface="eth0" netns="/var/run/netns/cni-f9b5a5d1-27f5-720e-4d1f-7eb833875c4b" Aug 13 01:29:01.963416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.963416 containerd[1567]: 2025-08-13 01:29:01.885 [INFO][6550] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" after=11.505487ms iface="eth0" netns="/var/run/netns/cni-f9b5a5d1-27f5-720e-4d1f-7eb833875c4b" Aug 13 01:29:01.963416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.963416 containerd[1567]: 2025-08-13 01:29:01.886 [INFO][6550] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Aug 13 01:29:01.963416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.963416 containerd[1567]: 2025-08-13 01:29:01.886 [INFO][6550] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Aug 13 01:29:01.963416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.963416 containerd[1567]: 2025-08-13 01:29:01.927 [INFO][6564] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" HandleID="k8s-pod-network.8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:01.963416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.963416 containerd[1567]: 2025-08-13 01:29:01.927 [INFO][6564] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:01.963416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.963416 containerd[1567]: 2025-08-13 01:29:01.927 [INFO][6564] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:01.963416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.963416 containerd[1567]: 2025-08-13 01:29:01.957 [INFO][6564] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" HandleID="k8s-pod-network.8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:01.963416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.963416 containerd[1567]: 2025-08-13 01:29:01.957 [INFO][6564] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" HandleID="k8s-pod-network.8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:01.963416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.963416 containerd[1567]: 2025-08-13 01:29:01.958 [INFO][6564] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:01.963416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.963416 containerd[1567]: 2025-08-13 01:29:01.961 [INFO][6550] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658" Aug 13 01:29:01.963416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:01.968421 containerd[1567]: time="2025-08-13T01:29:01.964530303Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658-lo: no space left on device" Aug 13 01:29:01.967500 systemd[1]: run-netns-cni\x2df9b5a5d1\x2d27f5\x2d720e\x2d4d1f\x2d7eb833875c4b.mount: Deactivated successfully. Aug 13 01:29:01.968780 kubelet[2720]: E0813 01:29:01.964901 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658-lo: no space left on device" Aug 13 01:29:01.968780 kubelet[2720]: E0813 01:29:01.964976 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:01.968780 kubelet[2720]: E0813 01:29:01.964998 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:01.968780 kubelet[2720]: E0813 01:29:01.965079 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:02.612564 containerd[1567]: time="2025-08-13T01:29:02.612494995Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:02.725583 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:02.726657 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.651 [INFO][6576] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 11748 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.653 [INFO][6576] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.685 [INFO][6589] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" HandleID="k8s-pod-network.e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.685 [INFO][6589] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" HandleID="k8s-pod-network.e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5860), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:02.685435619 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.685 [INFO][6589] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.686 [INFO][6589] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.686 [INFO][6589] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.692 [INFO][6589] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" host="172-234-25-215" Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.698 [INFO][6589] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.702 [INFO][6589] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.706 [INFO][6589] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.708 [INFO][6589] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.708 [INFO][6589] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" host="172-234-25-215" Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.710 [INFO][6589] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006 Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.713 [INFO][6589] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" host="172-234-25-215" Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.718 [INFO][6589] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.10/26] block=192.168.62.0/26 handle="k8s-pod-network.e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" host="172-234-25-215" Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.739301 containerd[1567]: 2025-08-13 01:29:02.718 [INFO][6589] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.10/26] handle="k8s-pod-network.e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" host="172-234-25-215" Aug 13 01:29:02.739301 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.742632 containerd[1567]: 2025-08-13 01:29:02.718 [INFO][6589] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:02.742632 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.742632 containerd[1567]: 2025-08-13 01:29:02.718 [INFO][6589] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.10/26] IPv6=[] ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" HandleID="k8s-pod-network.e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:02.742632 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.742632 containerd[1567]: 2025-08-13 01:29:02.721 [INFO][6576] cni-plugin/k8s.go 418: Populated endpoint ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11748", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"8335b6b85dd55e150821d81dca90173c90d1e5650fc43686ece711c7a3b77658", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.10/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:02.742632 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.742632 containerd[1567]: 2025-08-13 01:29:02.721 [INFO][6576] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.10/32] ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:02.742632 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.742632 containerd[1567]: 2025-08-13 01:29:02.721 [INFO][6576] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:02.742632 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.742632 containerd[1567]: 2025-08-13 01:29:02.727 [INFO][6576] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:02.742632 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.742632 containerd[1567]: 2025-08-13 01:29:02.728 [INFO][6576] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11748", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.10/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"62:29:25:55:9f:56", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:02.742632 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.746042 containerd[1567]: 2025-08-13 01:29:02.736 [INFO][6576] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:02.746042 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.800152 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:02.801135 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:02.934330 containerd[1567]: 2025-08-13 01:29:02.798 [INFO][6611] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Aug 13 01:29:02.934330 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.934330 containerd[1567]: 2025-08-13 01:29:02.798 [INFO][6611] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" iface="eth0" netns="/var/run/netns/cni-c89d7e47-5cfe-10db-6d08-ce8ca354aa05" Aug 13 01:29:02.934330 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.934330 containerd[1567]: 2025-08-13 01:29:02.799 [INFO][6611] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" iface="eth0" netns="/var/run/netns/cni-c89d7e47-5cfe-10db-6d08-ce8ca354aa05" Aug 13 01:29:02.934330 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.934330 containerd[1567]: 2025-08-13 01:29:02.809 [INFO][6611] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" after=10.196229ms iface="eth0" netns="/var/run/netns/cni-c89d7e47-5cfe-10db-6d08-ce8ca354aa05" Aug 13 01:29:02.934330 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.934330 containerd[1567]: 2025-08-13 01:29:02.809 [INFO][6611] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Aug 13 01:29:02.934330 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.934330 containerd[1567]: 2025-08-13 01:29:02.809 [INFO][6611] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Aug 13 01:29:02.934330 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.934330 containerd[1567]: 2025-08-13 01:29:02.874 [INFO][6621] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" HandleID="k8s-pod-network.e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:02.934330 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.934330 containerd[1567]: 2025-08-13 01:29:02.874 [INFO][6621] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:02.934330 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.934330 containerd[1567]: 2025-08-13 01:29:02.874 [INFO][6621] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:02.934330 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.934330 containerd[1567]: 2025-08-13 01:29:02.927 [INFO][6621] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" HandleID="k8s-pod-network.e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:02.934330 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.934330 containerd[1567]: 2025-08-13 01:29:02.927 [INFO][6621] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" HandleID="k8s-pod-network.e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:02.934330 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.934330 containerd[1567]: 2025-08-13 01:29:02.929 [INFO][6621] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:02.934330 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.934330 containerd[1567]: 2025-08-13 01:29:02.931 [INFO][6611] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006" Aug 13 01:29:02.934330 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:02.939352 containerd[1567]: time="2025-08-13T01:29:02.938345727Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006-lo: no space left on device" Aug 13 01:29:02.939127 systemd[1]: run-netns-cni\x2dc89d7e47\x2d5cfe\x2d10db\x2d6d08\x2dce8ca354aa05.mount: Deactivated successfully. Aug 13 01:29:02.940218 kubelet[2720]: E0813 01:29:02.940087 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006-lo: no space left on device" Aug 13 01:29:02.941618 kubelet[2720]: E0813 01:29:02.940773 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:02.941618 kubelet[2720]: E0813 01:29:02.940804 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:02.941618 kubelet[2720]: E0813 01:29:02.940867 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:03.349649 containerd[1567]: time="2025-08-13T01:29:03.349575120Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:03.443836 containerd[1567]: time="2025-08-13T01:29:03.443785873Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:29:03.616275 containerd[1567]: time="2025-08-13T01:29:03.615953401Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:03.734127 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:03.735488 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.654 [INFO][6630] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 11818 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.655 [INFO][6630] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.682 [INFO][6642] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" HandleID="k8s-pod-network.3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.682 [INFO][6642] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" HandleID="k8s-pod-network.3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cd930), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:03.6820974 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.682 [INFO][6642] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.682 [INFO][6642] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.682 [INFO][6642] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.689 [INFO][6642] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" host="172-234-25-215" Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.694 [INFO][6642] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.700 [INFO][6642] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.703 [INFO][6642] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.708 [INFO][6642] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.708 [INFO][6642] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" host="172-234-25-215" Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.711 [INFO][6642] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.718 [INFO][6642] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" host="172-234-25-215" Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.727 [INFO][6642] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.11/26] block=192.168.62.0/26 handle="k8s-pod-network.3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" host="172-234-25-215" Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.760758 containerd[1567]: 2025-08-13 01:29:03.727 [INFO][6642] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.11/26] handle="k8s-pod-network.3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" host="172-234-25-215" Aug 13 01:29:03.760758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.763740 containerd[1567]: 2025-08-13 01:29:03.727 [INFO][6642] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:03.763740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.763740 containerd[1567]: 2025-08-13 01:29:03.727 [INFO][6642] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.11/26] IPv6=[] ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" HandleID="k8s-pod-network.3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:03.763740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.763740 containerd[1567]: 2025-08-13 01:29:03.730 [INFO][6630] cni-plugin/k8s.go 418: Populated endpoint ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11818", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e80c56b03d8bfc4e02e7d13f9b01cf9f38329c7810f253287d66bfabe619a006", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.11/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:03.763740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.763740 containerd[1567]: 2025-08-13 01:29:03.730 [INFO][6630] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.11/32] ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:03.763740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.763740 containerd[1567]: 2025-08-13 01:29:03.731 [INFO][6630] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:03.763740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.763740 containerd[1567]: 2025-08-13 01:29:03.734 [INFO][6630] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:03.763740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.763740 containerd[1567]: 2025-08-13 01:29:03.736 [INFO][6630] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11818", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.11/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ee:c3:d8:2e:12:1f", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:03.763740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.764081 containerd[1567]: 2025-08-13 01:29:03.746 [INFO][6630] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:03.764081 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.832019 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:03.833102 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:03.923856 containerd[1567]: 2025-08-13 01:29:03.828 [INFO][6666] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Aug 13 01:29:03.923856 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.923856 containerd[1567]: 2025-08-13 01:29:03.829 [INFO][6666] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" iface="eth0" netns="/var/run/netns/cni-05a6313f-0efa-778b-40ed-a30280e68b8b" Aug 13 01:29:03.923856 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.923856 containerd[1567]: 2025-08-13 01:29:03.830 [INFO][6666] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" iface="eth0" netns="/var/run/netns/cni-05a6313f-0efa-778b-40ed-a30280e68b8b" Aug 13 01:29:03.923856 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.923856 containerd[1567]: 2025-08-13 01:29:03.836 [INFO][6666] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" after=6.664226ms iface="eth0" netns="/var/run/netns/cni-05a6313f-0efa-778b-40ed-a30280e68b8b" Aug 13 01:29:03.923856 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.923856 containerd[1567]: 2025-08-13 01:29:03.836 [INFO][6666] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Aug 13 01:29:03.923856 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.923856 containerd[1567]: 2025-08-13 01:29:03.836 [INFO][6666] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Aug 13 01:29:03.923856 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.923856 containerd[1567]: 2025-08-13 01:29:03.878 [INFO][6675] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" HandleID="k8s-pod-network.3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:03.923856 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.923856 containerd[1567]: 2025-08-13 01:29:03.878 [INFO][6675] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:03.923856 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.923856 containerd[1567]: 2025-08-13 01:29:03.878 [INFO][6675] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:03.923856 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.923856 containerd[1567]: 2025-08-13 01:29:03.916 [INFO][6675] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" HandleID="k8s-pod-network.3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:03.923856 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.923856 containerd[1567]: 2025-08-13 01:29:03.916 [INFO][6675] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" HandleID="k8s-pod-network.3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:03.923856 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.923856 containerd[1567]: 2025-08-13 01:29:03.918 [INFO][6675] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:03.923856 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.923856 containerd[1567]: 2025-08-13 01:29:03.920 [INFO][6666] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e" Aug 13 01:29:03.923856 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:03.928293 containerd[1567]: time="2025-08-13T01:29:03.928160822Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e-lo: no space left on device" Aug 13 01:29:03.928229 systemd[1]: run-netns-cni\x2d05a6313f\x2d0efa\x2d778b\x2d40ed\x2da30280e68b8b.mount: Deactivated successfully. Aug 13 01:29:03.928740 kubelet[2720]: E0813 01:29:03.928412 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e-lo: no space left on device" Aug 13 01:29:03.928740 kubelet[2720]: E0813 01:29:03.928463 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:03.928740 kubelet[2720]: E0813 01:29:03.928482 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:03.928740 kubelet[2720]: E0813 01:29:03.928524 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:04.617687 containerd[1567]: time="2025-08-13T01:29:04.617622087Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:04.739039 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:04.741417 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.658 [INFO][6684] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 11893 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.658 [INFO][6684] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.691 [INFO][6696] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" HandleID="k8s-pod-network.8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.691 [INFO][6696] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" HandleID="k8s-pod-network.8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002c5690), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:04.691523571 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.691 [INFO][6696] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.691 [INFO][6696] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.691 [INFO][6696] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.700 [INFO][6696] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" host="172-234-25-215" Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.707 [INFO][6696] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.713 [INFO][6696] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.716 [INFO][6696] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.718 [INFO][6696] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.719 [INFO][6696] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" host="172-234-25-215" Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.720 [INFO][6696] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.724 [INFO][6696] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" host="172-234-25-215" Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.729 [INFO][6696] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.12/26] block=192.168.62.0/26 handle="k8s-pod-network.8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" host="172-234-25-215" Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.766855 containerd[1567]: 2025-08-13 01:29:04.729 [INFO][6696] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.12/26] handle="k8s-pod-network.8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" host="172-234-25-215" Aug 13 01:29:04.766855 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.768071 containerd[1567]: 2025-08-13 01:29:04.729 [INFO][6696] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:04.768071 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.768071 containerd[1567]: 2025-08-13 01:29:04.730 [INFO][6696] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.12/26] IPv6=[] ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" HandleID="k8s-pod-network.8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:04.768071 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.768071 containerd[1567]: 2025-08-13 01:29:04.733 [INFO][6684] cni-plugin/k8s.go 418: Populated endpoint ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11893", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"3e520f5ca7ee60116a9494a6e608ae507082df9382b6b9027bc1ef764576d22e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.12/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:04.768071 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.768071 containerd[1567]: 2025-08-13 01:29:04.733 [INFO][6684] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.12/32] ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:04.768071 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.768071 containerd[1567]: 2025-08-13 01:29:04.733 [INFO][6684] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:04.768071 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.768071 containerd[1567]: 2025-08-13 01:29:04.741 [INFO][6684] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:04.768071 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.768071 containerd[1567]: 2025-08-13 01:29:04.744 [INFO][6684] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11893", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.12/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"62:23:80:c6:88:1e", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:04.768071 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.768378 containerd[1567]: 2025-08-13 01:29:04.761 [INFO][6684] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:04.768378 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.837113 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:04.837124 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:04.952300 containerd[1567]: 2025-08-13 01:29:04.834 [INFO][6720] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Aug 13 01:29:04.952300 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.952300 containerd[1567]: 2025-08-13 01:29:04.835 [INFO][6720] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" iface="eth0" netns="/var/run/netns/cni-0320ae32-c952-ba86-a038-94361dcc1371" Aug 13 01:29:04.952300 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.952300 containerd[1567]: 2025-08-13 01:29:04.836 [INFO][6720] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" iface="eth0" netns="/var/run/netns/cni-0320ae32-c952-ba86-a038-94361dcc1371" Aug 13 01:29:04.952300 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.952300 containerd[1567]: 2025-08-13 01:29:04.842 [INFO][6720] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" after=7.443925ms iface="eth0" netns="/var/run/netns/cni-0320ae32-c952-ba86-a038-94361dcc1371" Aug 13 01:29:04.952300 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.952300 containerd[1567]: 2025-08-13 01:29:04.842 [INFO][6720] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Aug 13 01:29:04.952300 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.952300 containerd[1567]: 2025-08-13 01:29:04.843 [INFO][6720] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Aug 13 01:29:04.952300 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.952300 containerd[1567]: 2025-08-13 01:29:04.899 [INFO][6730] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" HandleID="k8s-pod-network.8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:04.952300 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.952300 containerd[1567]: 2025-08-13 01:29:04.902 [INFO][6730] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:04.952300 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.952300 containerd[1567]: 2025-08-13 01:29:04.902 [INFO][6730] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:04.952300 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.952300 containerd[1567]: 2025-08-13 01:29:04.946 [INFO][6730] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" HandleID="k8s-pod-network.8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:04.952300 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.952300 containerd[1567]: 2025-08-13 01:29:04.946 [INFO][6730] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" HandleID="k8s-pod-network.8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:04.952300 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.952300 containerd[1567]: 2025-08-13 01:29:04.947 [INFO][6730] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:04.952300 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.952300 containerd[1567]: 2025-08-13 01:29:04.949 [INFO][6720] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b" Aug 13 01:29:04.952300 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:04.956133 containerd[1567]: time="2025-08-13T01:29:04.956080747Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b-lo: no space left on device" Aug 13 01:29:04.956458 kubelet[2720]: E0813 01:29:04.956370 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b-lo: no space left on device" Aug 13 01:29:04.956779 kubelet[2720]: E0813 01:29:04.956529 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:04.956779 kubelet[2720]: E0813 01:29:04.956656 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:04.957394 kubelet[2720]: E0813 01:29:04.956941 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:04.957753 systemd[1]: run-netns-cni\x2d0320ae32\x2dc952\x2dba86\x2da038\x2d94361dcc1371.mount: Deactivated successfully. Aug 13 01:29:05.620052 containerd[1567]: time="2025-08-13T01:29:05.620005866Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:05.730733 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:05.732790 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.665 [INFO][6740] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 11965 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.666 [INFO][6740] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.687 [INFO][6752] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" HandleID="k8s-pod-network.34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.687 [INFO][6752] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" HandleID="k8s-pod-network.34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f640), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:05.687317743 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.687 [INFO][6752] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.687 [INFO][6752] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.687 [INFO][6752] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.693 [INFO][6752] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" host="172-234-25-215" Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.700 [INFO][6752] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.707 [INFO][6752] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.708 [INFO][6752] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.710 [INFO][6752] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.710 [INFO][6752] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" host="172-234-25-215" Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.711 [INFO][6752] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.716 [INFO][6752] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" host="172-234-25-215" Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.721 [INFO][6752] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.13/26] block=192.168.62.0/26 handle="k8s-pod-network.34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" host="172-234-25-215" Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.750790 containerd[1567]: 2025-08-13 01:29:05.721 [INFO][6752] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.13/26] handle="k8s-pod-network.34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" host="172-234-25-215" Aug 13 01:29:05.750790 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.752517 containerd[1567]: 2025-08-13 01:29:05.721 [INFO][6752] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:05.752517 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.752517 containerd[1567]: 2025-08-13 01:29:05.721 [INFO][6752] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.13/26] IPv6=[] ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" HandleID="k8s-pod-network.34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:05.752517 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.752517 containerd[1567]: 2025-08-13 01:29:05.724 [INFO][6740] cni-plugin/k8s.go 418: Populated endpoint ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11965", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"8e658aac80fd77159a4d4c8e9fa5454837f239398a72c678f5ccef79d5a7158b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.13/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:05.752517 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.752517 containerd[1567]: 2025-08-13 01:29:05.724 [INFO][6740] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.13/32] ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:05.752517 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.752517 containerd[1567]: 2025-08-13 01:29:05.724 [INFO][6740] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:05.752517 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.752517 containerd[1567]: 2025-08-13 01:29:05.732 [INFO][6740] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:05.752517 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.752517 containerd[1567]: 2025-08-13 01:29:05.732 [INFO][6740] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"11965", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.13/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ca:ad:ba:16:32:60", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:05.752517 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.753163 containerd[1567]: 2025-08-13 01:29:05.743 [INFO][6740] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:05.753163 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.810841 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:05.810850 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:05.886590 containerd[1567]: 2025-08-13 01:29:05.806 [INFO][6777] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Aug 13 01:29:05.886590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.886590 containerd[1567]: 2025-08-13 01:29:05.806 [INFO][6777] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" iface="eth0" netns="/var/run/netns/cni-3f2fddba-fd41-d25b-186d-43d0b9661dfd" Aug 13 01:29:05.886590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.886590 containerd[1567]: 2025-08-13 01:29:05.807 [INFO][6777] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" iface="eth0" netns="/var/run/netns/cni-3f2fddba-fd41-d25b-186d-43d0b9661dfd" Aug 13 01:29:05.886590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.886590 containerd[1567]: 2025-08-13 01:29:05.815 [INFO][6777] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" after=8.594213ms iface="eth0" netns="/var/run/netns/cni-3f2fddba-fd41-d25b-186d-43d0b9661dfd" Aug 13 01:29:05.886590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.886590 containerd[1567]: 2025-08-13 01:29:05.815 [INFO][6777] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Aug 13 01:29:05.886590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.886590 containerd[1567]: 2025-08-13 01:29:05.815 [INFO][6777] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Aug 13 01:29:05.886590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.886590 containerd[1567]: 2025-08-13 01:29:05.854 [INFO][6787] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" HandleID="k8s-pod-network.34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:05.886590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.886590 containerd[1567]: 2025-08-13 01:29:05.854 [INFO][6787] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:05.886590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.886590 containerd[1567]: 2025-08-13 01:29:05.854 [INFO][6787] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:05.886590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.886590 containerd[1567]: 2025-08-13 01:29:05.880 [INFO][6787] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" HandleID="k8s-pod-network.34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:05.886590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.886590 containerd[1567]: 2025-08-13 01:29:05.880 [INFO][6787] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" HandleID="k8s-pod-network.34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:05.886590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.886590 containerd[1567]: 2025-08-13 01:29:05.881 [INFO][6787] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:05.886590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.886590 containerd[1567]: 2025-08-13 01:29:05.883 [INFO][6777] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c" Aug 13 01:29:05.886590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:05.888525 systemd[1]: run-netns-cni\x2d3f2fddba\x2dfd41\x2dd25b\x2d186d\x2d43d0b9661dfd.mount: Deactivated successfully. Aug 13 01:29:05.888663 containerd[1567]: time="2025-08-13T01:29:05.888622856Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c-lo: no space left on device" Aug 13 01:29:05.890464 kubelet[2720]: E0813 01:29:05.890410 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c-lo: no space left on device" Aug 13 01:29:05.890525 kubelet[2720]: E0813 01:29:05.890469 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:05.890525 kubelet[2720]: E0813 01:29:05.890489 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:05.890617 kubelet[2720]: E0813 01:29:05.890589 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:06.349636 containerd[1567]: time="2025-08-13T01:29:06.349568186Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:06.621896 containerd[1567]: time="2025-08-13T01:29:06.621773640Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:06.718314 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:06.719450 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.655 [INFO][6796] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 12021 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.656 [INFO][6796] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.681 [INFO][6810] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" HandleID="k8s-pod-network.bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.681 [INFO][6810] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" HandleID="k8s-pod-network.bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f0a0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:06.681015433 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.681 [INFO][6810] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.681 [INFO][6810] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.681 [INFO][6810] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.688 [INFO][6810] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" host="172-234-25-215" Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.691 [INFO][6810] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.696 [INFO][6810] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.697 [INFO][6810] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.700 [INFO][6810] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.700 [INFO][6810] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" host="172-234-25-215" Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.702 [INFO][6810] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.705 [INFO][6810] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" host="172-234-25-215" Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.712 [INFO][6810] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.14/26] block=192.168.62.0/26 handle="k8s-pod-network.bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" host="172-234-25-215" Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.738348 containerd[1567]: 2025-08-13 01:29:06.712 [INFO][6810] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.14/26] handle="k8s-pod-network.bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" host="172-234-25-215" Aug 13 01:29:06.738348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.740093 containerd[1567]: 2025-08-13 01:29:06.712 [INFO][6810] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:06.740093 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.740093 containerd[1567]: 2025-08-13 01:29:06.712 [INFO][6810] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.14/26] IPv6=[] ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" HandleID="k8s-pod-network.bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:06.740093 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.740093 containerd[1567]: 2025-08-13 01:29:06.715 [INFO][6796] cni-plugin/k8s.go 418: Populated endpoint ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12021", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"34569a04fa08bfdbf956ced3c03a2a893520930434cf65fa2d8d88e3c8d0c34c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.14/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:06.740093 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.740093 containerd[1567]: 2025-08-13 01:29:06.715 [INFO][6796] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.14/32] ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:06.740093 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.740093 containerd[1567]: 2025-08-13 01:29:06.715 [INFO][6796] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:06.740093 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.740093 containerd[1567]: 2025-08-13 01:29:06.721 [INFO][6796] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:06.740093 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.740093 containerd[1567]: 2025-08-13 01:29:06.722 [INFO][6796] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12021", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.14/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"02:98:34:f4:44:a2", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:06.740093 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.740428 containerd[1567]: 2025-08-13 01:29:06.730 [INFO][6796] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:06.740428 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.806050 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:06.806096 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:06.892533 systemd[1]: Started sshd@27-172.234.25.215:22-147.75.109.163:53582.service - OpenSSH per-connection server daemon (147.75.109.163:53582). Aug 13 01:29:06.933033 containerd[1567]: 2025-08-13 01:29:06.803 [INFO][6832] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Aug 13 01:29:06.933033 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.933033 containerd[1567]: 2025-08-13 01:29:06.803 [INFO][6832] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" iface="eth0" netns="/var/run/netns/cni-62cd9b50-6aa7-a857-795c-347bd2e8d3d7" Aug 13 01:29:06.933033 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.933033 containerd[1567]: 2025-08-13 01:29:06.803 [INFO][6832] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" iface="eth0" netns="/var/run/netns/cni-62cd9b50-6aa7-a857-795c-347bd2e8d3d7" Aug 13 01:29:06.933033 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.933033 containerd[1567]: 2025-08-13 01:29:06.812 [INFO][6832] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" after=8.577723ms iface="eth0" netns="/var/run/netns/cni-62cd9b50-6aa7-a857-795c-347bd2e8d3d7" Aug 13 01:29:06.933033 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.933033 containerd[1567]: 2025-08-13 01:29:06.812 [INFO][6832] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Aug 13 01:29:06.933033 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.933033 containerd[1567]: 2025-08-13 01:29:06.812 [INFO][6832] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Aug 13 01:29:06.933033 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.933033 containerd[1567]: 2025-08-13 01:29:06.870 [INFO][6840] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" HandleID="k8s-pod-network.bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:06.933033 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.933033 containerd[1567]: 2025-08-13 01:29:06.872 [INFO][6840] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:06.933033 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.933033 containerd[1567]: 2025-08-13 01:29:06.872 [INFO][6840] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:06.933033 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.933033 containerd[1567]: 2025-08-13 01:29:06.927 [INFO][6840] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" HandleID="k8s-pod-network.bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:06.933033 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.933033 containerd[1567]: 2025-08-13 01:29:06.927 [INFO][6840] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" HandleID="k8s-pod-network.bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:06.933033 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.933033 containerd[1567]: 2025-08-13 01:29:06.928 [INFO][6840] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:06.933033 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.933033 containerd[1567]: 2025-08-13 01:29:06.931 [INFO][6832] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed" Aug 13 01:29:06.933033 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:06.935786 containerd[1567]: time="2025-08-13T01:29:06.934221824Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed-lo: no space left on device" Aug 13 01:29:06.936058 kubelet[2720]: E0813 01:29:06.936016 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed-lo: no space left on device" Aug 13 01:29:06.936364 kubelet[2720]: E0813 01:29:06.936080 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:06.936364 kubelet[2720]: E0813 01:29:06.936100 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:06.936364 kubelet[2720]: E0813 01:29:06.936159 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:06.937018 systemd[1]: run-netns-cni\x2d62cd9b50\x2d6aa7\x2da857\x2d795c\x2d347bd2e8d3d7.mount: Deactivated successfully. Aug 13 01:29:07.243759 sshd[6851]: Accepted publickey for core from 147.75.109.163 port 53582 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:29:07.244944 sshd-session[6851]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:29:07.251739 systemd-logind[1543]: New session 28 of user core. Aug 13 01:29:07.256835 systemd[1]: Started session-28.scope - Session 28 of User core. Aug 13 01:29:07.553182 sshd[6853]: Connection closed by 147.75.109.163 port 53582 Aug 13 01:29:07.553840 sshd-session[6851]: pam_unix(sshd:session): session closed for user core Aug 13 01:29:07.557801 systemd-logind[1543]: Session 28 logged out. Waiting for processes to exit. Aug 13 01:29:07.558665 systemd[1]: sshd@27-172.234.25.215:22-147.75.109.163:53582.service: Deactivated successfully. Aug 13 01:29:07.560649 systemd[1]: session-28.scope: Deactivated successfully. Aug 13 01:29:07.562526 systemd-logind[1543]: Removed session 28. Aug 13 01:29:07.624015 containerd[1567]: time="2025-08-13T01:29:07.623750308Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:07.720116 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:07.721506 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.658 [INFO][6865] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 12095 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.658 [INFO][6865] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.679 [INFO][6878] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" HandleID="k8s-pod-network.6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.680 [INFO][6878] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" HandleID="k8s-pod-network.6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5600), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:07.679829227 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.680 [INFO][6878] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.680 [INFO][6878] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.680 [INFO][6878] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.685 [INFO][6878] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" host="172-234-25-215" Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.689 [INFO][6878] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.693 [INFO][6878] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.694 [INFO][6878] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.697 [INFO][6878] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.697 [INFO][6878] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" host="172-234-25-215" Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.699 [INFO][6878] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.704 [INFO][6878] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" host="172-234-25-215" Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.712 [INFO][6878] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.15/26] block=192.168.62.0/26 handle="k8s-pod-network.6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" host="172-234-25-215" Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.740621 containerd[1567]: 2025-08-13 01:29:07.712 [INFO][6878] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.15/26] handle="k8s-pod-network.6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" host="172-234-25-215" Aug 13 01:29:07.740621 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.741696 containerd[1567]: 2025-08-13 01:29:07.712 [INFO][6878] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:07.741696 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.741696 containerd[1567]: 2025-08-13 01:29:07.713 [INFO][6878] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.15/26] IPv6=[] ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" HandleID="k8s-pod-network.6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:07.741696 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.741696 containerd[1567]: 2025-08-13 01:29:07.715 [INFO][6865] cni-plugin/k8s.go 418: Populated endpoint ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12095", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"bc7ab2aefd244dd11abfcb7abba657343bacb233ac33608b82f83bac2d2b72ed", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.15/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:07.741696 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.741696 containerd[1567]: 2025-08-13 01:29:07.716 [INFO][6865] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.15/32] ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:07.741696 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.741696 containerd[1567]: 2025-08-13 01:29:07.716 [INFO][6865] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:07.741696 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.741696 containerd[1567]: 2025-08-13 01:29:07.721 [INFO][6865] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:07.741696 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.741696 containerd[1567]: 2025-08-13 01:29:07.722 [INFO][6865] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12095", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.15/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"2a:ec:5f:bb:30:36", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:07.741696 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.743955 containerd[1567]: 2025-08-13 01:29:07.734 [INFO][6865] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:07.743955 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.807685 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:07.807694 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:07.887756 containerd[1567]: 2025-08-13 01:29:07.805 [INFO][6901] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Aug 13 01:29:07.887756 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.887756 containerd[1567]: 2025-08-13 01:29:07.806 [INFO][6901] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" iface="eth0" netns="/var/run/netns/cni-dcb476e6-3de6-3bd7-bce8-7620551dc1cf" Aug 13 01:29:07.887756 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.887756 containerd[1567]: 2025-08-13 01:29:07.806 [INFO][6901] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" iface="eth0" netns="/var/run/netns/cni-dcb476e6-3de6-3bd7-bce8-7620551dc1cf" Aug 13 01:29:07.887756 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.887756 containerd[1567]: 2025-08-13 01:29:07.813 [INFO][6901] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" after=7.482136ms iface="eth0" netns="/var/run/netns/cni-dcb476e6-3de6-3bd7-bce8-7620551dc1cf" Aug 13 01:29:07.887756 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.887756 containerd[1567]: 2025-08-13 01:29:07.815 [INFO][6901] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Aug 13 01:29:07.887756 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.887756 containerd[1567]: 2025-08-13 01:29:07.815 [INFO][6901] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Aug 13 01:29:07.887756 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.887756 containerd[1567]: 2025-08-13 01:29:07.848 [INFO][6911] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" HandleID="k8s-pod-network.6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:07.887756 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.887756 containerd[1567]: 2025-08-13 01:29:07.848 [INFO][6911] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:07.887756 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.887756 containerd[1567]: 2025-08-13 01:29:07.848 [INFO][6911] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:07.887756 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.887756 containerd[1567]: 2025-08-13 01:29:07.880 [INFO][6911] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" HandleID="k8s-pod-network.6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:07.887756 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.887756 containerd[1567]: 2025-08-13 01:29:07.880 [INFO][6911] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" HandleID="k8s-pod-network.6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:07.887756 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.887756 containerd[1567]: 2025-08-13 01:29:07.881 [INFO][6911] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:07.887756 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.887756 containerd[1567]: 2025-08-13 01:29:07.883 [INFO][6901] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f" Aug 13 01:29:07.887756 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:07.889400 systemd[1]: run-netns-cni\x2ddcb476e6\x2d3de6\x2d3bd7\x2dbce8\x2d7620551dc1cf.mount: Deactivated successfully. Aug 13 01:29:07.889537 containerd[1567]: time="2025-08-13T01:29:07.889376125Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f-lo: no space left on device" Aug 13 01:29:07.889671 kubelet[2720]: E0813 01:29:07.889635 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f-lo: no space left on device" Aug 13 01:29:07.889762 kubelet[2720]: E0813 01:29:07.889692 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:07.889762 kubelet[2720]: E0813 01:29:07.889750 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:07.890641 kubelet[2720]: E0813 01:29:07.889819 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:08.067338 kubelet[2720]: I0813 01:29:08.067244 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:29:08.067338 kubelet[2720]: I0813 01:29:08.067277 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:29:08.070815 kubelet[2720]: I0813 01:29:08.070800 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:29:08.083064 kubelet[2720]: I0813 01:29:08.083043 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:29:08.083166 kubelet[2720]: I0813 01:29:08.083150 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:29:08.083235 kubelet[2720]: E0813 01:29:08.083176 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:08.083235 kubelet[2720]: E0813 01:29:08.083184 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:29:08.083235 kubelet[2720]: E0813 01:29:08.083190 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:29:08.083235 kubelet[2720]: E0813 01:29:08.083199 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:29:08.083235 kubelet[2720]: E0813 01:29:08.083207 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:29:08.083235 kubelet[2720]: E0813 01:29:08.083217 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:29:08.083235 kubelet[2720]: E0813 01:29:08.083229 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:29:08.083378 kubelet[2720]: E0813 01:29:08.083246 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:29:08.083378 kubelet[2720]: E0813 01:29:08.083259 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:29:08.083378 kubelet[2720]: E0813 01:29:08.083270 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:29:08.083378 kubelet[2720]: I0813 01:29:08.083279 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:29:08.410772 containerd[1567]: time="2025-08-13T01:29:08.410629832Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:29:08.626428 containerd[1567]: time="2025-08-13T01:29:08.626137169Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:08.720495 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:08.721389 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.663 [INFO][6926] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 12156 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.663 [INFO][6926] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.684 [INFO][6937] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" HandleID="k8s-pod-network.c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.685 [INFO][6937] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" HandleID="k8s-pod-network.c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f150), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:08.684836503 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.685 [INFO][6937] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.685 [INFO][6937] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.685 [INFO][6937] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.691 [INFO][6937] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" host="172-234-25-215" Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.694 [INFO][6937] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.699 [INFO][6937] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.701 [INFO][6937] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.703 [INFO][6937] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.703 [INFO][6937] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" host="172-234-25-215" Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.704 [INFO][6937] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.708 [INFO][6937] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" host="172-234-25-215" Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.713 [INFO][6937] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.16/26] block=192.168.62.0/26 handle="k8s-pod-network.c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" host="172-234-25-215" Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.738943 containerd[1567]: 2025-08-13 01:29:08.714 [INFO][6937] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.16/26] handle="k8s-pod-network.c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" host="172-234-25-215" Aug 13 01:29:08.738943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.739644 containerd[1567]: 2025-08-13 01:29:08.714 [INFO][6937] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:08.739644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.739644 containerd[1567]: 2025-08-13 01:29:08.714 [INFO][6937] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.16/26] IPv6=[] ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" HandleID="k8s-pod-network.c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:08.739644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.739644 containerd[1567]: 2025-08-13 01:29:08.717 [INFO][6926] cni-plugin/k8s.go 418: Populated endpoint ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12156", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"6dc7df5a7e120e854b0d15af5e37836d6e53a831661304c68005c94645f4067f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.16/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:08.739644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.739644 containerd[1567]: 2025-08-13 01:29:08.717 [INFO][6926] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.16/32] ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:08.739644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.739644 containerd[1567]: 2025-08-13 01:29:08.717 [INFO][6926] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:08.739644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.739644 containerd[1567]: 2025-08-13 01:29:08.720 [INFO][6926] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:08.739644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.739644 containerd[1567]: 2025-08-13 01:29:08.721 [INFO][6926] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12156", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.16/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"4a:aa:1d:28:29:1c", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:08.739644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.740007 containerd[1567]: 2025-08-13 01:29:08.730 [INFO][6926] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:08.740007 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.814503 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:08.814512 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:08.910826 containerd[1567]: 2025-08-13 01:29:08.809 [INFO][6961] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Aug 13 01:29:08.910826 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.910826 containerd[1567]: 2025-08-13 01:29:08.809 [INFO][6961] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" iface="eth0" netns="/var/run/netns/cni-d723b86e-00a3-e9d2-e065-587fd8d5baa3" Aug 13 01:29:08.910826 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.910826 containerd[1567]: 2025-08-13 01:29:08.812 [INFO][6961] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" iface="eth0" netns="/var/run/netns/cni-d723b86e-00a3-e9d2-e065-587fd8d5baa3" Aug 13 01:29:08.910826 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.910826 containerd[1567]: 2025-08-13 01:29:08.824 [INFO][6961] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" after=14.123192ms iface="eth0" netns="/var/run/netns/cni-d723b86e-00a3-e9d2-e065-587fd8d5baa3" Aug 13 01:29:08.910826 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.910826 containerd[1567]: 2025-08-13 01:29:08.824 [INFO][6961] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Aug 13 01:29:08.910826 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.910826 containerd[1567]: 2025-08-13 01:29:08.824 [INFO][6961] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Aug 13 01:29:08.910826 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.910826 containerd[1567]: 2025-08-13 01:29:08.869 [INFO][6972] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" HandleID="k8s-pod-network.c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:08.910826 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.910826 containerd[1567]: 2025-08-13 01:29:08.869 [INFO][6972] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:08.910826 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.910826 containerd[1567]: 2025-08-13 01:29:08.869 [INFO][6972] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:08.910826 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.910826 containerd[1567]: 2025-08-13 01:29:08.904 [INFO][6972] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" HandleID="k8s-pod-network.c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:08.910826 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.910826 containerd[1567]: 2025-08-13 01:29:08.904 [INFO][6972] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" HandleID="k8s-pod-network.c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:08.910826 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.910826 containerd[1567]: 2025-08-13 01:29:08.906 [INFO][6972] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:08.910826 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.910826 containerd[1567]: 2025-08-13 01:29:08.908 [INFO][6961] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b" Aug 13 01:29:08.910826 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:08.913482 systemd[1]: run-netns-cni\x2dd723b86e\x2d00a3\x2de9d2\x2de065\x2d587fd8d5baa3.mount: Deactivated successfully. Aug 13 01:29:08.914049 containerd[1567]: time="2025-08-13T01:29:08.913546314Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b-lo: no space left on device" Aug 13 01:29:08.915319 kubelet[2720]: E0813 01:29:08.914256 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b-lo: no space left on device" Aug 13 01:29:08.915319 kubelet[2720]: E0813 01:29:08.914311 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:08.915319 kubelet[2720]: E0813 01:29:08.914906 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:08.915319 kubelet[2720]: E0813 01:29:08.914966 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:09.074161 kubelet[2720]: E0813 01:29:09.074055 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:29:09.074535 kubelet[2720]: E0813 01:29:09.074391 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:29:09.075863 containerd[1567]: time="2025-08-13T01:29:09.075631577Z" level=info msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\"" Aug 13 01:29:09.305207 containerd[1567]: time="2025-08-13T01:29:09.305163327Z" level=error msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\" failed" error="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-304131310-cnVI sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-927608295: no space left on device" Aug 13 01:29:09.305468 containerd[1567]: time="2025-08-13T01:29:09.305187787Z" level=info msg="stop pulling image registry.k8s.io/coredns/coredns:v1.12.0: active requests=0, bytes read=0" Aug 13 01:29:09.305625 kubelet[2720]: E0813 01:29:09.305583 2720 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-304131310-cnVI sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-927608295: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:29:09.305687 kubelet[2720]: E0813 01:29:09.305629 2720 kuberuntime_image.go:42] "Failed to pull image" err="failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-304131310-cnVI sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-927608295: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:29:09.305905 kubelet[2720]: E0813 01:29:09.305857 2720 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:coredns,Image:registry.k8s.io/coredns/coredns:v1.12.0,Command:[],Args:[-conf /etc/coredns/Corefile],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:dns,HostPort:0,ContainerPort:53,Protocol:UDP,HostIP:,},ContainerPort{Name:dns-tcp,HostPort:0,ContainerPort:53,Protocol:TCP,HostIP:,},ContainerPort{Name:metrics,HostPort:0,ContainerPort:9153,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{memory: {{178257920 0} {} 170Mi BinarySI},},Requests:ResourceList{cpu: {{100 -3} {} 100m DecimalSI},memory: {{73400320 0} {} 70Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:config-volume,ReadOnly:true,MountPath:/etc/coredns,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-lncr6,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{0 8080 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:60,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:5,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/ready,Port:{0 8181 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[NET_BIND_SERVICE],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b): ErrImagePull: failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-304131310-cnVI sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-927608295: no space left on device" logger="UnhandledError" Aug 13 01:29:09.306249 containerd[1567]: time="2025-08-13T01:29:09.306215526Z" level=info msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\"" Aug 13 01:29:09.307402 kubelet[2720]: E0813 01:29:09.307284 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ErrImagePull: \"failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-304131310-cnVI sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-927608295: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:29:09.350231 containerd[1567]: time="2025-08-13T01:29:09.349871109Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:09.521031 containerd[1567]: time="2025-08-13T01:29:09.520967025Z" level=error msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\" failed" error="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-519958916-KETu sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4205936624: no space left on device" Aug 13 01:29:09.521227 containerd[1567]: time="2025-08-13T01:29:09.521047135Z" level=info msg="stop pulling image registry.k8s.io/coredns/coredns:v1.12.0: active requests=0, bytes read=0" Aug 13 01:29:09.521301 kubelet[2720]: E0813 01:29:09.521256 2720 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-519958916-KETu sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4205936624: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:29:09.521381 kubelet[2720]: E0813 01:29:09.521310 2720 kuberuntime_image.go:42] "Failed to pull image" err="failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-519958916-KETu sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4205936624: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:29:09.521490 kubelet[2720]: E0813 01:29:09.521440 2720 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:coredns,Image:registry.k8s.io/coredns/coredns:v1.12.0,Command:[],Args:[-conf /etc/coredns/Corefile],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:dns,HostPort:0,ContainerPort:53,Protocol:UDP,HostIP:,},ContainerPort{Name:dns-tcp,HostPort:0,ContainerPort:53,Protocol:TCP,HostIP:,},ContainerPort{Name:metrics,HostPort:0,ContainerPort:9153,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{memory: {{178257920 0} {} 170Mi BinarySI},},Requests:ResourceList{cpu: {{100 -3} {} 100m DecimalSI},memory: {{73400320 0} {} 70Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:config-volume,ReadOnly:true,MountPath:/etc/coredns,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-jdpj9,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{0 8080 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:60,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:5,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/ready,Port:{0 8181 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[NET_BIND_SERVICE],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8): ErrImagePull: failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-519958916-KETu sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4205936624: no space left on device" logger="UnhandledError" Aug 13 01:29:09.522749 kubelet[2720]: E0813 01:29:09.522702 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ErrImagePull: \"failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-519958916-KETu sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4205936624: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:29:09.629014 containerd[1567]: time="2025-08-13T01:29:09.628513504Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:09.739816 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:09.741786 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.668 [INFO][6980] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 12230 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.669 [INFO][6980] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.696 [INFO][6992] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" HandleID="k8s-pod-network.bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.696 [INFO][6992] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" HandleID="k8s-pod-network.bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:09.695991621 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.697 [INFO][6992] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.697 [INFO][6992] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.697 [INFO][6992] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.706 [INFO][6992] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" host="172-234-25-215" Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.710 [INFO][6992] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.713 [INFO][6992] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.714 [INFO][6992] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.716 [INFO][6992] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.716 [INFO][6992] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" host="172-234-25-215" Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.718 [INFO][6992] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.726 [INFO][6992] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" host="172-234-25-215" Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.732 [INFO][6992] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.17/26] block=192.168.62.0/26 handle="k8s-pod-network.bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" host="172-234-25-215" Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.767966 containerd[1567]: 2025-08-13 01:29:09.732 [INFO][6992] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.17/26] handle="k8s-pod-network.bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" host="172-234-25-215" Aug 13 01:29:09.767966 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.768829 containerd[1567]: 2025-08-13 01:29:09.732 [INFO][6992] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:09.768829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.768829 containerd[1567]: 2025-08-13 01:29:09.732 [INFO][6992] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.17/26] IPv6=[] ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" HandleID="k8s-pod-network.bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:09.768829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.768829 containerd[1567]: 2025-08-13 01:29:09.735 [INFO][6980] cni-plugin/k8s.go 418: Populated endpoint ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12230", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c208b537ff69f13ba6ae92b45b9a3aef6d80dcd5229802e0f6823770650c969b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.17/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:09.768829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.768829 containerd[1567]: 2025-08-13 01:29:09.735 [INFO][6980] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.17/32] ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:09.768829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.768829 containerd[1567]: 2025-08-13 01:29:09.735 [INFO][6980] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:09.768829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.768829 containerd[1567]: 2025-08-13 01:29:09.737 [INFO][6980] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:09.768829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.768829 containerd[1567]: 2025-08-13 01:29:09.738 [INFO][6980] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12230", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.17/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"7a:29:e8:f4:28:4c", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:09.768829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.769119 containerd[1567]: 2025-08-13 01:29:09.748 [INFO][6980] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:09.769119 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.825497 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:09.825507 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:09.917937 containerd[1567]: 2025-08-13 01:29:09.818 [INFO][7016] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Aug 13 01:29:09.917937 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.917937 containerd[1567]: 2025-08-13 01:29:09.819 [INFO][7016] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" iface="eth0" netns="/var/run/netns/cni-49434fe4-28a9-643b-c41d-cde1fff15796" Aug 13 01:29:09.917937 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.917937 containerd[1567]: 2025-08-13 01:29:09.821 [INFO][7016] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" iface="eth0" netns="/var/run/netns/cni-49434fe4-28a9-643b-c41d-cde1fff15796" Aug 13 01:29:09.917937 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.917937 containerd[1567]: 2025-08-13 01:29:09.828 [INFO][7016] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" after=9.374942ms iface="eth0" netns="/var/run/netns/cni-49434fe4-28a9-643b-c41d-cde1fff15796" Aug 13 01:29:09.917937 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.917937 containerd[1567]: 2025-08-13 01:29:09.828 [INFO][7016] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Aug 13 01:29:09.917937 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.917937 containerd[1567]: 2025-08-13 01:29:09.828 [INFO][7016] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Aug 13 01:29:09.917937 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.917937 containerd[1567]: 2025-08-13 01:29:09.859 [INFO][7026] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" HandleID="k8s-pod-network.bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:09.917937 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.917937 containerd[1567]: 2025-08-13 01:29:09.859 [INFO][7026] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:09.917937 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.917937 containerd[1567]: 2025-08-13 01:29:09.859 [INFO][7026] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:09.917937 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.917937 containerd[1567]: 2025-08-13 01:29:09.910 [INFO][7026] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" HandleID="k8s-pod-network.bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:09.917937 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.917937 containerd[1567]: 2025-08-13 01:29:09.910 [INFO][7026] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" HandleID="k8s-pod-network.bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:09.917937 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.917937 containerd[1567]: 2025-08-13 01:29:09.911 [INFO][7026] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:09.917937 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.917937 containerd[1567]: 2025-08-13 01:29:09.915 [INFO][7016] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be" Aug 13 01:29:09.917937 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:09.922433 containerd[1567]: time="2025-08-13T01:29:09.919627654Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be-lo: no space left on device" Aug 13 01:29:09.921431 systemd[1]: run-netns-cni\x2d49434fe4\x2d28a9\x2d643b\x2dc41d\x2dcde1fff15796.mount: Deactivated successfully. Aug 13 01:29:09.922883 kubelet[2720]: E0813 01:29:09.920503 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be-lo: no space left on device" Aug 13 01:29:09.922883 kubelet[2720]: E0813 01:29:09.920570 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:09.922883 kubelet[2720]: E0813 01:29:09.920597 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:09.922883 kubelet[2720]: E0813 01:29:09.920665 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:10.631298 containerd[1567]: time="2025-08-13T01:29:10.630846093Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:10.741310 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:10.743629 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.671 [INFO][7034] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 12304 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.671 [INFO][7034] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.697 [INFO][7047] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" HandleID="k8s-pod-network.8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.697 [INFO][7047] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" HandleID="k8s-pod-network.8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f200), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:10.697042653 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.697 [INFO][7047] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.698 [INFO][7047] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.698 [INFO][7047] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.706 [INFO][7047] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" host="172-234-25-215" Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.713 [INFO][7047] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.718 [INFO][7047] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.720 [INFO][7047] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.722 [INFO][7047] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.722 [INFO][7047] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" host="172-234-25-215" Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.724 [INFO][7047] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.728 [INFO][7047] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" host="172-234-25-215" Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.734 [INFO][7047] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.18/26] block=192.168.62.0/26 handle="k8s-pod-network.8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" host="172-234-25-215" Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.764169 containerd[1567]: 2025-08-13 01:29:10.734 [INFO][7047] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.18/26] handle="k8s-pod-network.8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" host="172-234-25-215" Aug 13 01:29:10.764169 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.767202 containerd[1567]: 2025-08-13 01:29:10.734 [INFO][7047] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:10.767202 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.767202 containerd[1567]: 2025-08-13 01:29:10.734 [INFO][7047] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.18/26] IPv6=[] ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" HandleID="k8s-pod-network.8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:10.767202 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.767202 containerd[1567]: 2025-08-13 01:29:10.737 [INFO][7034] cni-plugin/k8s.go 418: Populated endpoint ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12304", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"bbdf81f6d0ceb40bb59a898fca9289f32d23ff0809f4ea2057b13f5526e8a5be", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.18/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:10.767202 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.767202 containerd[1567]: 2025-08-13 01:29:10.737 [INFO][7034] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.18/32] ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:10.767202 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.767202 containerd[1567]: 2025-08-13 01:29:10.738 [INFO][7034] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:10.767202 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.767202 containerd[1567]: 2025-08-13 01:29:10.740 [INFO][7034] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:10.767202 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.767202 containerd[1567]: 2025-08-13 01:29:10.743 [INFO][7034] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12304", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.18/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"7a:8d:b6:c3:41:74", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:10.767202 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.767507 containerd[1567]: 2025-08-13 01:29:10.757 [INFO][7034] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:10.767507 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.831694 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:10.831704 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:10.952644 containerd[1567]: 2025-08-13 01:29:10.828 [INFO][7071] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Aug 13 01:29:10.952644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.952644 containerd[1567]: 2025-08-13 01:29:10.830 [INFO][7071] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" iface="eth0" netns="/var/run/netns/cni-22e8c28e-bf2f-1654-5ee8-6cfe6fc51a27" Aug 13 01:29:10.952644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.952644 containerd[1567]: 2025-08-13 01:29:10.831 [INFO][7071] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" iface="eth0" netns="/var/run/netns/cni-22e8c28e-bf2f-1654-5ee8-6cfe6fc51a27" Aug 13 01:29:10.952644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.952644 containerd[1567]: 2025-08-13 01:29:10.838 [INFO][7071] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" after=7.643505ms iface="eth0" netns="/var/run/netns/cni-22e8c28e-bf2f-1654-5ee8-6cfe6fc51a27" Aug 13 01:29:10.952644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.952644 containerd[1567]: 2025-08-13 01:29:10.838 [INFO][7071] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Aug 13 01:29:10.952644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.952644 containerd[1567]: 2025-08-13 01:29:10.838 [INFO][7071] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Aug 13 01:29:10.952644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.952644 containerd[1567]: 2025-08-13 01:29:10.878 [INFO][7080] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" HandleID="k8s-pod-network.8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:10.952644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.952644 containerd[1567]: 2025-08-13 01:29:10.878 [INFO][7080] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:10.952644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.952644 containerd[1567]: 2025-08-13 01:29:10.878 [INFO][7080] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:10.952644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.952644 containerd[1567]: 2025-08-13 01:29:10.940 [INFO][7080] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" HandleID="k8s-pod-network.8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:10.952644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.952644 containerd[1567]: 2025-08-13 01:29:10.940 [INFO][7080] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" HandleID="k8s-pod-network.8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:10.952644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.952644 containerd[1567]: 2025-08-13 01:29:10.945 [INFO][7080] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:10.952644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.952644 containerd[1567]: 2025-08-13 01:29:10.949 [INFO][7071] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b" Aug 13 01:29:10.952644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:10.957239 containerd[1567]: time="2025-08-13T01:29:10.957164225Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b-lo: no space left on device" Aug 13 01:29:10.958738 kubelet[2720]: E0813 01:29:10.957487 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b-lo: no space left on device" Aug 13 01:29:10.958738 kubelet[2720]: E0813 01:29:10.957556 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:10.958738 kubelet[2720]: E0813 01:29:10.957579 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:10.958738 kubelet[2720]: E0813 01:29:10.957634 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:10.958874 systemd[1]: run-netns-cni\x2d22e8c28e\x2dbf2f\x2d1654\x2d5ee8\x2d6cfe6fc51a27.mount: Deactivated successfully. Aug 13 01:29:11.632573 containerd[1567]: time="2025-08-13T01:29:11.632527167Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:11.740626 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:11.741440 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.673 [INFO][7092] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 12376 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.673 [INFO][7092] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.698 [INFO][7101] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" HandleID="k8s-pod-network.44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.699 [INFO][7101] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" HandleID="k8s-pod-network.44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:11.698638348 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.699 [INFO][7101] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.699 [INFO][7101] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.699 [INFO][7101] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.706 [INFO][7101] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" host="172-234-25-215" Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.711 [INFO][7101] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.715 [INFO][7101] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.717 [INFO][7101] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.720 [INFO][7101] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.720 [INFO][7101] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" host="172-234-25-215" Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.723 [INFO][7101] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6 Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.726 [INFO][7101] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" host="172-234-25-215" Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.735 [INFO][7101] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.19/26] block=192.168.62.0/26 handle="k8s-pod-network.44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" host="172-234-25-215" Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.760978 containerd[1567]: 2025-08-13 01:29:11.735 [INFO][7101] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.19/26] handle="k8s-pod-network.44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" host="172-234-25-215" Aug 13 01:29:11.760978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.762787 containerd[1567]: 2025-08-13 01:29:11.735 [INFO][7101] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:11.762787 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.762787 containerd[1567]: 2025-08-13 01:29:11.735 [INFO][7101] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.19/26] IPv6=[] ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" HandleID="k8s-pod-network.44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:11.762787 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.762787 containerd[1567]: 2025-08-13 01:29:11.737 [INFO][7092] cni-plugin/k8s.go 418: Populated endpoint ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12376", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"8a09373f43a29866fabd9d324a9ca8e18a5a9395f806d6aa11fda9b8d10da97b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.19/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:11.762787 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.762787 containerd[1567]: 2025-08-13 01:29:11.738 [INFO][7092] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.19/32] ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:11.762787 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.762787 containerd[1567]: 2025-08-13 01:29:11.738 [INFO][7092] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:11.762787 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.762787 containerd[1567]: 2025-08-13 01:29:11.742 [INFO][7092] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:11.762787 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.762787 containerd[1567]: 2025-08-13 01:29:11.743 [INFO][7092] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12376", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.19/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ae:7e:00:5a:d5:71", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:11.762787 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.763338 containerd[1567]: 2025-08-13 01:29:11.753 [INFO][7092] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:11.763338 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.825450 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:11.825461 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:11.941663 containerd[1567]: 2025-08-13 01:29:11.822 [INFO][7125] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Aug 13 01:29:11.941663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.941663 containerd[1567]: 2025-08-13 01:29:11.823 [INFO][7125] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" iface="eth0" netns="/var/run/netns/cni-9af8cafa-1606-3da9-6ab5-0773bbda65fd" Aug 13 01:29:11.941663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.941663 containerd[1567]: 2025-08-13 01:29:11.823 [INFO][7125] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" iface="eth0" netns="/var/run/netns/cni-9af8cafa-1606-3da9-6ab5-0773bbda65fd" Aug 13 01:29:11.941663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.941663 containerd[1567]: 2025-08-13 01:29:11.834 [INFO][7125] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" after=10.368641ms iface="eth0" netns="/var/run/netns/cni-9af8cafa-1606-3da9-6ab5-0773bbda65fd" Aug 13 01:29:11.941663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.941663 containerd[1567]: 2025-08-13 01:29:11.834 [INFO][7125] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Aug 13 01:29:11.941663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.941663 containerd[1567]: 2025-08-13 01:29:11.834 [INFO][7125] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Aug 13 01:29:11.941663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.941663 containerd[1567]: 2025-08-13 01:29:11.900 [INFO][7133] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" HandleID="k8s-pod-network.44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:11.941663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.941663 containerd[1567]: 2025-08-13 01:29:11.900 [INFO][7133] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:11.941663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.941663 containerd[1567]: 2025-08-13 01:29:11.900 [INFO][7133] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:11.941663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.941663 containerd[1567]: 2025-08-13 01:29:11.931 [INFO][7133] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" HandleID="k8s-pod-network.44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:11.941663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.941663 containerd[1567]: 2025-08-13 01:29:11.931 [INFO][7133] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" HandleID="k8s-pod-network.44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:11.941663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.941663 containerd[1567]: 2025-08-13 01:29:11.932 [INFO][7133] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:11.941663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.941663 containerd[1567]: 2025-08-13 01:29:11.934 [INFO][7125] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6" Aug 13 01:29:11.941663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:11.941663 containerd[1567]: time="2025-08-13T01:29:11.940707525Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6-lo: no space left on device" Aug 13 01:29:11.939491 systemd[1]: run-netns-cni\x2d9af8cafa\x2d1606\x2d3da9\x2d6ab5\x2d0773bbda65fd.mount: Deactivated successfully. Aug 13 01:29:11.943292 kubelet[2720]: E0813 01:29:11.943222 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6-lo: no space left on device" Aug 13 01:29:11.943813 kubelet[2720]: E0813 01:29:11.943352 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:11.943813 kubelet[2720]: E0813 01:29:11.943410 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:11.943813 kubelet[2720]: E0813 01:29:11.943505 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:12.350128 containerd[1567]: time="2025-08-13T01:29:12.349938788Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:12.616369 systemd[1]: Started sshd@28-172.234.25.215:22-147.75.109.163:49526.service - OpenSSH per-connection server daemon (147.75.109.163:49526). Aug 13 01:29:12.634606 containerd[1567]: time="2025-08-13T01:29:12.634557424Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:12.743176 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:12.744757 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.674 [INFO][7148] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 12456 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.674 [INFO][7148] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.700 [INFO][7160] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" HandleID="k8s-pod-network.e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.701 [INFO][7160] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" HandleID="k8s-pod-network.e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f100), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:12.700153145 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.701 [INFO][7160] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.701 [INFO][7160] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.701 [INFO][7160] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.711 [INFO][7160] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" host="172-234-25-215" Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.715 [INFO][7160] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.719 [INFO][7160] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.721 [INFO][7160] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.723 [INFO][7160] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.723 [INFO][7160] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" host="172-234-25-215" Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.726 [INFO][7160] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5 Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.729 [INFO][7160] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" host="172-234-25-215" Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.735 [INFO][7160] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.20/26] block=192.168.62.0/26 handle="k8s-pod-network.e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" host="172-234-25-215" Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.766951 containerd[1567]: 2025-08-13 01:29:12.735 [INFO][7160] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.20/26] handle="k8s-pod-network.e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" host="172-234-25-215" Aug 13 01:29:12.766951 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.768061 containerd[1567]: 2025-08-13 01:29:12.735 [INFO][7160] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:12.768061 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.768061 containerd[1567]: 2025-08-13 01:29:12.735 [INFO][7160] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.20/26] IPv6=[] ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" HandleID="k8s-pod-network.e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:12.768061 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.768061 containerd[1567]: 2025-08-13 01:29:12.737 [INFO][7148] cni-plugin/k8s.go 418: Populated endpoint ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12456", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"44e5a66458c115c7d8aff104b9b04d3faedbb60782f84838ca44907f3d645bb6", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.20/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:12.768061 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.768061 containerd[1567]: 2025-08-13 01:29:12.738 [INFO][7148] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.20/32] ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:12.768061 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.768061 containerd[1567]: 2025-08-13 01:29:12.738 [INFO][7148] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:12.768061 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.768061 containerd[1567]: 2025-08-13 01:29:12.746 [INFO][7148] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:12.768061 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.768061 containerd[1567]: 2025-08-13 01:29:12.746 [INFO][7148] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12456", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.20/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"1a:eb:fe:e8:ae:d2", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:12.768061 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.768366 containerd[1567]: 2025-08-13 01:29:12.758 [INFO][7148] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:12.768366 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.839323 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:12.839332 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:12.934987 containerd[1567]: 2025-08-13 01:29:12.833 [INFO][7184] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Aug 13 01:29:12.934987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.934987 containerd[1567]: 2025-08-13 01:29:12.834 [INFO][7184] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" iface="eth0" netns="/var/run/netns/cni-f9b0b596-d19e-6be5-2e42-eba1a3704df3" Aug 13 01:29:12.934987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.934987 containerd[1567]: 2025-08-13 01:29:12.837 [INFO][7184] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" iface="eth0" netns="/var/run/netns/cni-f9b0b596-d19e-6be5-2e42-eba1a3704df3" Aug 13 01:29:12.934987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.934987 containerd[1567]: 2025-08-13 01:29:12.850 [INFO][7184] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" after=13.977623ms iface="eth0" netns="/var/run/netns/cni-f9b0b596-d19e-6be5-2e42-eba1a3704df3" Aug 13 01:29:12.934987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.934987 containerd[1567]: 2025-08-13 01:29:12.850 [INFO][7184] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Aug 13 01:29:12.934987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.934987 containerd[1567]: 2025-08-13 01:29:12.850 [INFO][7184] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Aug 13 01:29:12.934987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.934987 containerd[1567]: 2025-08-13 01:29:12.888 [INFO][7193] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" HandleID="k8s-pod-network.e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:12.934987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.934987 containerd[1567]: 2025-08-13 01:29:12.888 [INFO][7193] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:12.934987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.934987 containerd[1567]: 2025-08-13 01:29:12.888 [INFO][7193] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:12.934987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.934987 containerd[1567]: 2025-08-13 01:29:12.928 [INFO][7193] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" HandleID="k8s-pod-network.e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:12.934987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.934987 containerd[1567]: 2025-08-13 01:29:12.928 [INFO][7193] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" HandleID="k8s-pod-network.e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:12.934987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.934987 containerd[1567]: 2025-08-13 01:29:12.929 [INFO][7193] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:12.934987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.934987 containerd[1567]: 2025-08-13 01:29:12.932 [INFO][7184] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5" Aug 13 01:29:12.934987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:12.936341 containerd[1567]: time="2025-08-13T01:29:12.936235795Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5-lo: no space left on device" Aug 13 01:29:12.936776 kubelet[2720]: E0813 01:29:12.936569 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5-lo: no space left on device" Aug 13 01:29:12.936776 kubelet[2720]: E0813 01:29:12.936624 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:12.936776 kubelet[2720]: E0813 01:29:12.936644 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:12.936776 kubelet[2720]: E0813 01:29:12.936696 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:12.940795 systemd[1]: run-netns-cni\x2df9b0b596\x2dd19e\x2d6be5\x2d2e42\x2deba1a3704df3.mount: Deactivated successfully. Aug 13 01:29:12.963894 sshd[7144]: Accepted publickey for core from 147.75.109.163 port 49526 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:29:12.965552 sshd-session[7144]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:29:12.971459 systemd-logind[1543]: New session 29 of user core. Aug 13 01:29:12.977854 systemd[1]: Started session-29.scope - Session 29 of User core. Aug 13 01:29:13.073937 kubelet[2720]: E0813 01:29:13.073904 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:29:13.271439 sshd[7202]: Connection closed by 147.75.109.163 port 49526 Aug 13 01:29:13.272358 sshd-session[7144]: pam_unix(sshd:session): session closed for user core Aug 13 01:29:13.277487 systemd-logind[1543]: Session 29 logged out. Waiting for processes to exit. Aug 13 01:29:13.279414 systemd[1]: sshd@28-172.234.25.215:22-147.75.109.163:49526.service: Deactivated successfully. Aug 13 01:29:13.282636 systemd[1]: session-29.scope: Deactivated successfully. Aug 13 01:29:13.286122 systemd-logind[1543]: Removed session 29. Aug 13 01:29:13.638488 containerd[1567]: time="2025-08-13T01:29:13.638185900Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:13.761373 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:13.763762 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.683 [INFO][7215] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 12521 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.684 [INFO][7215] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.716 [INFO][7228] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" HandleID="k8s-pod-network.2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.717 [INFO][7228] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" HandleID="k8s-pod-network.2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cf020), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:13.716886017 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.717 [INFO][7228] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.717 [INFO][7228] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.717 [INFO][7228] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.724 [INFO][7228] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" host="172-234-25-215" Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.728 [INFO][7228] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.731 [INFO][7228] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.733 [INFO][7228] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.736 [INFO][7228] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.736 [INFO][7228] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" host="172-234-25-215" Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.738 [INFO][7228] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.742 [INFO][7228] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" host="172-234-25-215" Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.748 [INFO][7228] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.21/26] block=192.168.62.0/26 handle="k8s-pod-network.2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" host="172-234-25-215" Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.784296 containerd[1567]: 2025-08-13 01:29:13.748 [INFO][7228] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.21/26] handle="k8s-pod-network.2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" host="172-234-25-215" Aug 13 01:29:13.784296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.785087 containerd[1567]: 2025-08-13 01:29:13.748 [INFO][7228] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:13.785087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.785087 containerd[1567]: 2025-08-13 01:29:13.748 [INFO][7228] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.21/26] IPv6=[] ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" HandleID="k8s-pod-network.2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:13.785087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.785087 containerd[1567]: 2025-08-13 01:29:13.752 [INFO][7215] cni-plugin/k8s.go 418: Populated endpoint ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12521", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e6d1ae6fc82d90487ec6c6e11c2b87eaa989857880a1b32457a8bc6a6eac86b5", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.21/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:13.785087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.785087 containerd[1567]: 2025-08-13 01:29:13.752 [INFO][7215] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.21/32] ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:13.785087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.785087 containerd[1567]: 2025-08-13 01:29:13.752 [INFO][7215] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:13.785087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.785087 containerd[1567]: 2025-08-13 01:29:13.764 [INFO][7215] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:13.785087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.785087 containerd[1567]: 2025-08-13 01:29:13.764 [INFO][7215] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12521", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.21/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"26:a5:f6:d6:c6:8e", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:13.785087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.787290 containerd[1567]: 2025-08-13 01:29:13.776 [INFO][7215] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:13.787290 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.855404 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:13.855414 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:13.985703 containerd[1567]: 2025-08-13 01:29:13.852 [INFO][7251] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Aug 13 01:29:13.985703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.985703 containerd[1567]: 2025-08-13 01:29:13.852 [INFO][7251] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" iface="eth0" netns="/var/run/netns/cni-d3648083-29b2-0c65-33f6-44ab149b1f1b" Aug 13 01:29:13.985703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.985703 containerd[1567]: 2025-08-13 01:29:13.853 [INFO][7251] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" iface="eth0" netns="/var/run/netns/cni-d3648083-29b2-0c65-33f6-44ab149b1f1b" Aug 13 01:29:13.985703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.985703 containerd[1567]: 2025-08-13 01:29:13.880 [INFO][7251] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" after=27.634507ms iface="eth0" netns="/var/run/netns/cni-d3648083-29b2-0c65-33f6-44ab149b1f1b" Aug 13 01:29:13.985703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.985703 containerd[1567]: 2025-08-13 01:29:13.880 [INFO][7251] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Aug 13 01:29:13.985703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.985703 containerd[1567]: 2025-08-13 01:29:13.880 [INFO][7251] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Aug 13 01:29:13.985703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.985703 containerd[1567]: 2025-08-13 01:29:13.923 [INFO][7262] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" HandleID="k8s-pod-network.2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:13.985703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.985703 containerd[1567]: 2025-08-13 01:29:13.924 [INFO][7262] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:13.985703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.985703 containerd[1567]: 2025-08-13 01:29:13.924 [INFO][7262] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:13.985703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.985703 containerd[1567]: 2025-08-13 01:29:13.979 [INFO][7262] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" HandleID="k8s-pod-network.2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:13.985703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.985703 containerd[1567]: 2025-08-13 01:29:13.979 [INFO][7262] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" HandleID="k8s-pod-network.2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:13.985703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.985703 containerd[1567]: 2025-08-13 01:29:13.980 [INFO][7262] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:13.985703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.985703 containerd[1567]: 2025-08-13 01:29:13.983 [INFO][7251] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd" Aug 13 01:29:13.985703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:13.988568 systemd[1]: run-netns-cni\x2dd3648083\x2d29b2\x2d0c65\x2d33f6\x2d44ab149b1f1b.mount: Deactivated successfully. Aug 13 01:29:13.989326 containerd[1567]: time="2025-08-13T01:29:13.988739428Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd-lo: no space left on device" Aug 13 01:29:13.989389 kubelet[2720]: E0813 01:29:13.989151 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd-lo: no space left on device" Aug 13 01:29:13.989389 kubelet[2720]: E0813 01:29:13.989208 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:13.989389 kubelet[2720]: E0813 01:29:13.989232 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:13.989389 kubelet[2720]: E0813 01:29:13.989282 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:14.377395 containerd[1567]: time="2025-08-13T01:29:14.377230394Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:29:14.639700 containerd[1567]: time="2025-08-13T01:29:14.639590734Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:14.752344 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:14.754346 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.678 [INFO][7271] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 12600 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.678 [INFO][7271] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.707 [INFO][7284] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" HandleID="k8s-pod-network.3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.710 [INFO][7284] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" HandleID="k8s-pod-network.3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002c4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:14.707229883 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.710 [INFO][7284] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.710 [INFO][7284] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.710 [INFO][7284] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.717 [INFO][7284] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" host="172-234-25-215" Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.721 [INFO][7284] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.725 [INFO][7284] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.727 [INFO][7284] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.729 [INFO][7284] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.729 [INFO][7284] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" host="172-234-25-215" Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.732 [INFO][7284] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.736 [INFO][7284] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" host="172-234-25-215" Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.741 [INFO][7284] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.22/26] block=192.168.62.0/26 handle="k8s-pod-network.3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" host="172-234-25-215" Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.773987 containerd[1567]: 2025-08-13 01:29:14.741 [INFO][7284] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.22/26] handle="k8s-pod-network.3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" host="172-234-25-215" Aug 13 01:29:14.773987 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.776003 containerd[1567]: 2025-08-13 01:29:14.742 [INFO][7284] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:14.776003 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.776003 containerd[1567]: 2025-08-13 01:29:14.742 [INFO][7284] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.22/26] IPv6=[] ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" HandleID="k8s-pod-network.3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:14.776003 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.776003 containerd[1567]: 2025-08-13 01:29:14.744 [INFO][7271] cni-plugin/k8s.go 418: Populated endpoint ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12600", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"2ff644d8f4485aaf09e1d148d67b5edc7e3c53416b09423ef331a9a2ef958ebd", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.22/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:14.776003 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.776003 containerd[1567]: 2025-08-13 01:29:14.745 [INFO][7271] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.22/32] ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:14.776003 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.776003 containerd[1567]: 2025-08-13 01:29:14.745 [INFO][7271] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:14.776003 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.776003 containerd[1567]: 2025-08-13 01:29:14.754 [INFO][7271] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:14.776003 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.776003 containerd[1567]: 2025-08-13 01:29:14.755 [INFO][7271] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12600", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.22/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"66:06:89:02:90:46", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:14.776003 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.776435 containerd[1567]: 2025-08-13 01:29:14.769 [INFO][7271] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:14.776435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.843768 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:14.843777 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:14.926817 containerd[1567]: 2025-08-13 01:29:14.840 [INFO][7306] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Aug 13 01:29:14.926817 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.926817 containerd[1567]: 2025-08-13 01:29:14.841 [INFO][7306] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" iface="eth0" netns="/var/run/netns/cni-a049746c-6da4-d7bf-04f7-ccacb095373c" Aug 13 01:29:14.926817 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.926817 containerd[1567]: 2025-08-13 01:29:14.841 [INFO][7306] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" iface="eth0" netns="/var/run/netns/cni-a049746c-6da4-d7bf-04f7-ccacb095373c" Aug 13 01:29:14.926817 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.926817 containerd[1567]: 2025-08-13 01:29:14.850 [INFO][7306] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" after=9.164972ms iface="eth0" netns="/var/run/netns/cni-a049746c-6da4-d7bf-04f7-ccacb095373c" Aug 13 01:29:14.926817 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.926817 containerd[1567]: 2025-08-13 01:29:14.850 [INFO][7306] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Aug 13 01:29:14.926817 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.926817 containerd[1567]: 2025-08-13 01:29:14.850 [INFO][7306] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Aug 13 01:29:14.926817 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.926817 containerd[1567]: 2025-08-13 01:29:14.885 [INFO][7314] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" HandleID="k8s-pod-network.3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:14.926817 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.926817 containerd[1567]: 2025-08-13 01:29:14.886 [INFO][7314] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:14.926817 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.926817 containerd[1567]: 2025-08-13 01:29:14.886 [INFO][7314] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:14.926817 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.926817 containerd[1567]: 2025-08-13 01:29:14.920 [INFO][7314] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" HandleID="k8s-pod-network.3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:14.926817 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.926817 containerd[1567]: 2025-08-13 01:29:14.920 [INFO][7314] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" HandleID="k8s-pod-network.3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:14.926817 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.926817 containerd[1567]: 2025-08-13 01:29:14.922 [INFO][7314] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:14.926817 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.926817 containerd[1567]: 2025-08-13 01:29:14.924 [INFO][7306] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b" Aug 13 01:29:14.926817 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:14.931707 containerd[1567]: time="2025-08-13T01:29:14.931593307Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b-lo: no space left on device" Aug 13 01:29:14.931843 kubelet[2720]: E0813 01:29:14.931820 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b-lo: no space left on device" Aug 13 01:29:14.931911 kubelet[2720]: E0813 01:29:14.931865 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:14.931911 kubelet[2720]: E0813 01:29:14.931885 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:14.931961 kubelet[2720]: E0813 01:29:14.931929 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:14.932166 systemd[1]: run-netns-cni\x2da049746c\x2d6da4\x2dd7bf\x2d04f7\x2dccacb095373c.mount: Deactivated successfully. Aug 13 01:29:15.350240 containerd[1567]: time="2025-08-13T01:29:15.350193826Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:15.641551 containerd[1567]: time="2025-08-13T01:29:15.641431752Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:15.741887 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:15.743041 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.681 [INFO][7325] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 12679 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.681 [INFO][7325] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.706 [INFO][7337] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" HandleID="k8s-pod-network.61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.707 [INFO][7337] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" HandleID="k8s-pod-network.61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f2e0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:15.706661776 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.707 [INFO][7337] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.707 [INFO][7337] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.707 [INFO][7337] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.713 [INFO][7337] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" host="172-234-25-215" Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.717 [INFO][7337] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.721 [INFO][7337] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.722 [INFO][7337] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.724 [INFO][7337] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.724 [INFO][7337] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" host="172-234-25-215" Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.726 [INFO][7337] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.729 [INFO][7337] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" host="172-234-25-215" Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.733 [INFO][7337] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.23/26] block=192.168.62.0/26 handle="k8s-pod-network.61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" host="172-234-25-215" Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.764752 containerd[1567]: 2025-08-13 01:29:15.734 [INFO][7337] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.23/26] handle="k8s-pod-network.61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" host="172-234-25-215" Aug 13 01:29:15.764752 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.766781 containerd[1567]: 2025-08-13 01:29:15.734 [INFO][7337] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:15.766781 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.766781 containerd[1567]: 2025-08-13 01:29:15.734 [INFO][7337] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.23/26] IPv6=[] ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" HandleID="k8s-pod-network.61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:15.766781 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.766781 containerd[1567]: 2025-08-13 01:29:15.737 [INFO][7325] cni-plugin/k8s.go 418: Populated endpoint ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12679", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"3a76441a5459cf0ad5a40ddbccbe04d78ccfefc0fadde5c4d5e1848e07005b5b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.23/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:15.766781 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.766781 containerd[1567]: 2025-08-13 01:29:15.737 [INFO][7325] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.23/32] ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:15.766781 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.766781 containerd[1567]: 2025-08-13 01:29:15.737 [INFO][7325] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:15.766781 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.766781 containerd[1567]: 2025-08-13 01:29:15.743 [INFO][7325] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:15.766781 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.766781 containerd[1567]: 2025-08-13 01:29:15.743 [INFO][7325] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12679", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.23/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"46:96:e3:83:0d:d4", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:15.766781 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.767088 containerd[1567]: 2025-08-13 01:29:15.758 [INFO][7325] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:15.767088 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.829654 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:15.829666 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:15.926015 containerd[1567]: 2025-08-13 01:29:15.823 [INFO][7362] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Aug 13 01:29:15.926015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.926015 containerd[1567]: 2025-08-13 01:29:15.825 [INFO][7362] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" iface="eth0" netns="/var/run/netns/cni-baf1bf60-c2ca-93d2-6969-b0088aff565d" Aug 13 01:29:15.926015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.926015 containerd[1567]: 2025-08-13 01:29:15.825 [INFO][7362] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" iface="eth0" netns="/var/run/netns/cni-baf1bf60-c2ca-93d2-6969-b0088aff565d" Aug 13 01:29:15.926015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.926015 containerd[1567]: 2025-08-13 01:29:15.836 [INFO][7362] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" after=11.545758ms iface="eth0" netns="/var/run/netns/cni-baf1bf60-c2ca-93d2-6969-b0088aff565d" Aug 13 01:29:15.926015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.926015 containerd[1567]: 2025-08-13 01:29:15.837 [INFO][7362] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Aug 13 01:29:15.926015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.926015 containerd[1567]: 2025-08-13 01:29:15.837 [INFO][7362] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Aug 13 01:29:15.926015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.926015 containerd[1567]: 2025-08-13 01:29:15.868 [INFO][7370] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" HandleID="k8s-pod-network.61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:15.926015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.926015 containerd[1567]: 2025-08-13 01:29:15.868 [INFO][7370] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:15.926015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.926015 containerd[1567]: 2025-08-13 01:29:15.869 [INFO][7370] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:15.926015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.926015 containerd[1567]: 2025-08-13 01:29:15.915 [INFO][7370] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" HandleID="k8s-pod-network.61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:15.926015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.926015 containerd[1567]: 2025-08-13 01:29:15.916 [INFO][7370] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" HandleID="k8s-pod-network.61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:15.926015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.926015 containerd[1567]: 2025-08-13 01:29:15.918 [INFO][7370] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:15.926015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.926015 containerd[1567]: 2025-08-13 01:29:15.922 [INFO][7362] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f" Aug 13 01:29:15.926015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:15.929428 systemd[1]: run-netns-cni\x2dbaf1bf60\x2dc2ca\x2d93d2\x2d6969\x2db0088aff565d.mount: Deactivated successfully. Aug 13 01:29:15.932443 containerd[1567]: time="2025-08-13T01:29:15.932343757Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f-lo: no space left on device" Aug 13 01:29:15.932691 kubelet[2720]: E0813 01:29:15.932652 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f-lo: no space left on device" Aug 13 01:29:15.934126 kubelet[2720]: E0813 01:29:15.933485 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:15.934126 kubelet[2720]: E0813 01:29:15.933513 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:15.934516 kubelet[2720]: E0813 01:29:15.934228 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:16.073880 kubelet[2720]: E0813 01:29:16.073841 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:29:16.642852 containerd[1567]: time="2025-08-13T01:29:16.642752212Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:16.748596 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:16.748873 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.684 [INFO][7379] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 12747 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.684 [INFO][7379] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.709 [INFO][7393] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" HandleID="k8s-pod-network.e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.709 [INFO][7393] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" HandleID="k8s-pod-network.e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f870), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:16.709312104 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.709 [INFO][7393] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.709 [INFO][7393] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.709 [INFO][7393] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.717 [INFO][7393] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" host="172-234-25-215" Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.723 [INFO][7393] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.727 [INFO][7393] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.728 [INFO][7393] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.730 [INFO][7393] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.730 [INFO][7393] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" host="172-234-25-215" Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.732 [INFO][7393] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86 Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.736 [INFO][7393] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" host="172-234-25-215" Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.740 [INFO][7393] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.24/26] block=192.168.62.0/26 handle="k8s-pod-network.e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" host="172-234-25-215" Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.771747 containerd[1567]: 2025-08-13 01:29:16.740 [INFO][7393] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.24/26] handle="k8s-pod-network.e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" host="172-234-25-215" Aug 13 01:29:16.771747 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.773801 containerd[1567]: 2025-08-13 01:29:16.740 [INFO][7393] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:16.773801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.773801 containerd[1567]: 2025-08-13 01:29:16.740 [INFO][7393] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.24/26] IPv6=[] ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" HandleID="k8s-pod-network.e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:16.773801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.773801 containerd[1567]: 2025-08-13 01:29:16.743 [INFO][7379] cni-plugin/k8s.go 418: Populated endpoint ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12747", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"61f1c1167852d6085723bd3ca6d8528c8da62d96289d87386b0416f5b717fa6f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.24/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:16.773801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.773801 containerd[1567]: 2025-08-13 01:29:16.743 [INFO][7379] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.24/32] ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:16.773801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.773801 containerd[1567]: 2025-08-13 01:29:16.743 [INFO][7379] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:16.773801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.773801 containerd[1567]: 2025-08-13 01:29:16.748 [INFO][7379] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:16.773801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.773801 containerd[1567]: 2025-08-13 01:29:16.749 [INFO][7379] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12747", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.24/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"12:69:9f:b3:28:c3", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:16.773801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.774099 containerd[1567]: 2025-08-13 01:29:16.761 [INFO][7379] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:16.774099 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.827939 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:16.827948 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:16.907217 containerd[1567]: 2025-08-13 01:29:16.824 [INFO][7416] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Aug 13 01:29:16.907217 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.907217 containerd[1567]: 2025-08-13 01:29:16.825 [INFO][7416] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" iface="eth0" netns="/var/run/netns/cni-f77d1fe2-7d10-a0b1-9ddf-95fcea1b76f9" Aug 13 01:29:16.907217 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.907217 containerd[1567]: 2025-08-13 01:29:16.826 [INFO][7416] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" iface="eth0" netns="/var/run/netns/cni-f77d1fe2-7d10-a0b1-9ddf-95fcea1b76f9" Aug 13 01:29:16.907217 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.907217 containerd[1567]: 2025-08-13 01:29:16.839 [INFO][7416] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" after=13.020914ms iface="eth0" netns="/var/run/netns/cni-f77d1fe2-7d10-a0b1-9ddf-95fcea1b76f9" Aug 13 01:29:16.907217 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.907217 containerd[1567]: 2025-08-13 01:29:16.839 [INFO][7416] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Aug 13 01:29:16.907217 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.907217 containerd[1567]: 2025-08-13 01:29:16.839 [INFO][7416] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Aug 13 01:29:16.907217 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.907217 containerd[1567]: 2025-08-13 01:29:16.868 [INFO][7426] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" HandleID="k8s-pod-network.e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:16.907217 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.907217 containerd[1567]: 2025-08-13 01:29:16.869 [INFO][7426] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:16.907217 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.907217 containerd[1567]: 2025-08-13 01:29:16.869 [INFO][7426] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:16.907217 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.907217 containerd[1567]: 2025-08-13 01:29:16.898 [INFO][7426] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" HandleID="k8s-pod-network.e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:16.907217 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.907217 containerd[1567]: 2025-08-13 01:29:16.898 [INFO][7426] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" HandleID="k8s-pod-network.e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:16.907217 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.907217 containerd[1567]: 2025-08-13 01:29:16.900 [INFO][7426] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:16.907217 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.907217 containerd[1567]: 2025-08-13 01:29:16.905 [INFO][7416] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86" Aug 13 01:29:16.907217 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:16.909331 containerd[1567]: time="2025-08-13T01:29:16.909163287Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86-lo: no space left on device" Aug 13 01:29:16.910976 kubelet[2720]: E0813 01:29:16.910870 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86-lo: no space left on device" Aug 13 01:29:16.910976 kubelet[2720]: E0813 01:29:16.910926 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:16.910976 kubelet[2720]: E0813 01:29:16.910946 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:16.911301 kubelet[2720]: E0813 01:29:16.911006 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:16.911500 systemd[1]: run-netns-cni\x2df77d1fe2\x2d7d10\x2da0b1\x2d9ddf\x2d95fcea1b76f9.mount: Deactivated successfully. Aug 13 01:29:17.394840 containerd[1567]: time="2025-08-13T01:29:17.394772749Z" level=error msg="Fail to write \"stdout\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:17.641334 containerd[1567]: time="2025-08-13T01:29:17.641287442Z" level=info msg="TaskExit event in podsandbox handler container_id:\"05a7e6943f56eff710d015cc4d373aa0238d7506c1cf61f23b36bc5ebcb7ce84\" id:\"36a6f027664073f6ff10ab4d9424c0c15eb7b3f4064dc665d692154b5f8b7efc\" pid:7447 exited_at:{seconds:1755048557 nanos:641019093}" Aug 13 01:29:17.649354 containerd[1567]: time="2025-08-13T01:29:17.648679358Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:17.834948 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:17.836311 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.733 [INFO][7458] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 12828 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.733 [INFO][7458] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.773 [INFO][7470] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" HandleID="k8s-pod-network.0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.773 [INFO][7470] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" HandleID="k8s-pod-network.0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d58d0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:17.773254507 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.773 [INFO][7470] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.773 [INFO][7470] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.773 [INFO][7470] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.789 [INFO][7470] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" host="172-234-25-215" Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.799 [INFO][7470] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.804 [INFO][7470] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.808 [INFO][7470] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.811 [INFO][7470] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.811 [INFO][7470] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" host="172-234-25-215" Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.812 [INFO][7470] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.818 [INFO][7470] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" host="172-234-25-215" Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.823 [INFO][7470] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.25/26] block=192.168.62.0/26 handle="k8s-pod-network.0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" host="172-234-25-215" Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.862923 containerd[1567]: 2025-08-13 01:29:17.824 [INFO][7470] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.25/26] handle="k8s-pod-network.0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" host="172-234-25-215" Aug 13 01:29:17.862923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.865347 containerd[1567]: 2025-08-13 01:29:17.824 [INFO][7470] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:17.865347 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.865347 containerd[1567]: 2025-08-13 01:29:17.824 [INFO][7470] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.25/26] IPv6=[] ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" HandleID="k8s-pod-network.0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:17.865347 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.865347 containerd[1567]: 2025-08-13 01:29:17.828 [INFO][7458] cni-plugin/k8s.go 418: Populated endpoint ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12828", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e00a0e4c4348e997c30bf5d453492bf0324b94fdad0fd1fa200236e468975c86", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.25/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:17.865347 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.865347 containerd[1567]: 2025-08-13 01:29:17.829 [INFO][7458] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.25/32] ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:17.865347 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.865347 containerd[1567]: 2025-08-13 01:29:17.829 [INFO][7458] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:17.865347 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.865347 containerd[1567]: 2025-08-13 01:29:17.835 [INFO][7458] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:17.865347 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.865347 containerd[1567]: 2025-08-13 01:29:17.838 [INFO][7458] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12828", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.25/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"7e:6c:c3:cf:38:42", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:17.865347 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.865645 containerd[1567]: 2025-08-13 01:29:17.856 [INFO][7458] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:17.865645 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:17.954066 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:17.954098 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:18.051471 containerd[1567]: 2025-08-13 01:29:17.944 [INFO][7492] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Aug 13 01:29:18.051471 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.051471 containerd[1567]: 2025-08-13 01:29:17.944 [INFO][7492] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" iface="eth0" netns="/var/run/netns/cni-a88cd0f9-9789-9a6d-b039-4cd9bfdd62ab" Aug 13 01:29:18.051471 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.051471 containerd[1567]: 2025-08-13 01:29:17.945 [INFO][7492] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" iface="eth0" netns="/var/run/netns/cni-a88cd0f9-9789-9a6d-b039-4cd9bfdd62ab" Aug 13 01:29:18.051471 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.051471 containerd[1567]: 2025-08-13 01:29:17.964 [INFO][7492] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" after=20.048191ms iface="eth0" netns="/var/run/netns/cni-a88cd0f9-9789-9a6d-b039-4cd9bfdd62ab" Aug 13 01:29:18.051471 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.051471 containerd[1567]: 2025-08-13 01:29:17.965 [INFO][7492] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Aug 13 01:29:18.051471 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.051471 containerd[1567]: 2025-08-13 01:29:17.965 [INFO][7492] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Aug 13 01:29:18.051471 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.051471 containerd[1567]: 2025-08-13 01:29:18.011 [INFO][7502] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" HandleID="k8s-pod-network.0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:18.051471 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.051471 containerd[1567]: 2025-08-13 01:29:18.013 [INFO][7502] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:18.051471 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.051471 containerd[1567]: 2025-08-13 01:29:18.014 [INFO][7502] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:18.051471 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.051471 containerd[1567]: 2025-08-13 01:29:18.044 [INFO][7502] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" HandleID="k8s-pod-network.0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:18.051471 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.051471 containerd[1567]: 2025-08-13 01:29:18.044 [INFO][7502] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" HandleID="k8s-pod-network.0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:18.051471 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.051471 containerd[1567]: 2025-08-13 01:29:18.046 [INFO][7502] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:18.051471 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.051471 containerd[1567]: 2025-08-13 01:29:18.048 [INFO][7492] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c" Aug 13 01:29:18.051471 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.055316 containerd[1567]: time="2025-08-13T01:29:18.055204383Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c-lo: no space left on device" Aug 13 01:29:18.055683 systemd[1]: run-netns-cni\x2da88cd0f9\x2d9789\x2d9a6d\x2db039\x2d4cd9bfdd62ab.mount: Deactivated successfully. Aug 13 01:29:18.056596 kubelet[2720]: E0813 01:29:18.056253 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c-lo: no space left on device" Aug 13 01:29:18.056596 kubelet[2720]: E0813 01:29:18.056309 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:18.056596 kubelet[2720]: E0813 01:29:18.056334 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:18.058333 kubelet[2720]: E0813 01:29:18.057027 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:18.103561 kubelet[2720]: I0813 01:29:18.103464 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:29:18.103561 kubelet[2720]: I0813 01:29:18.103504 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:29:18.105901 kubelet[2720]: I0813 01:29:18.105601 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:29:18.121984 kubelet[2720]: I0813 01:29:18.121946 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:29:18.122254 kubelet[2720]: I0813 01:29:18.122058 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:29:18.122254 kubelet[2720]: E0813 01:29:18.122085 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:18.122254 kubelet[2720]: E0813 01:29:18.122094 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:29:18.122254 kubelet[2720]: E0813 01:29:18.122100 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:29:18.122254 kubelet[2720]: E0813 01:29:18.122110 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:29:18.122254 kubelet[2720]: E0813 01:29:18.122119 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:29:18.122254 kubelet[2720]: E0813 01:29:18.122126 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:29:18.122254 kubelet[2720]: E0813 01:29:18.122133 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:29:18.122254 kubelet[2720]: E0813 01:29:18.122142 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:29:18.122254 kubelet[2720]: E0813 01:29:18.122148 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:29:18.122254 kubelet[2720]: E0813 01:29:18.122155 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:29:18.122254 kubelet[2720]: I0813 01:29:18.122163 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:29:18.329477 systemd[1]: Started sshd@29-172.234.25.215:22-147.75.109.163:40648.service - OpenSSH per-connection server daemon (147.75.109.163:40648). Aug 13 01:29:18.350313 containerd[1567]: time="2025-08-13T01:29:18.350255443Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:18.648770 containerd[1567]: time="2025-08-13T01:29:18.648474407Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:18.664874 sshd[7514]: Accepted publickey for core from 147.75.109.163 port 40648 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:29:18.666359 sshd-session[7514]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:29:18.675107 systemd-logind[1543]: New session 30 of user core. Aug 13 01:29:18.679942 systemd[1]: Started session-30.scope - Session 30 of User core. Aug 13 01:29:18.776594 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:18.777168 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.693 [INFO][7517] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 12911 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.694 [INFO][7517] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.727 [INFO][7532] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" HandleID="k8s-pod-network.84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.727 [INFO][7532] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" HandleID="k8s-pod-network.84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d56e0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:18.727469946 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.728 [INFO][7532] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.728 [INFO][7532] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.728 [INFO][7532] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.734 [INFO][7532] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" host="172-234-25-215" Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.738 [INFO][7532] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.745 [INFO][7532] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.748 [INFO][7532] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.753 [INFO][7532] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.753 [INFO][7532] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" host="172-234-25-215" Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.755 [INFO][7532] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402 Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.762 [INFO][7532] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" host="172-234-25-215" Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.768 [INFO][7532] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.26/26] block=192.168.62.0/26 handle="k8s-pod-network.84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" host="172-234-25-215" Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.791285 containerd[1567]: 2025-08-13 01:29:18.768 [INFO][7532] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.26/26] handle="k8s-pod-network.84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" host="172-234-25-215" Aug 13 01:29:18.791285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.792487 containerd[1567]: 2025-08-13 01:29:18.768 [INFO][7532] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:18.792487 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.792487 containerd[1567]: 2025-08-13 01:29:18.768 [INFO][7532] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.26/26] IPv6=[] ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" HandleID="k8s-pod-network.84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:18.792487 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.792487 containerd[1567]: 2025-08-13 01:29:18.770 [INFO][7517] cni-plugin/k8s.go 418: Populated endpoint ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12911", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0c9c85a5c24c8014a1ce9e5e944c79f949f4524795ac537a605eaff99351096c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.26/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:18.792487 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.792487 containerd[1567]: 2025-08-13 01:29:18.770 [INFO][7517] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.26/32] ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:18.792487 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.792487 containerd[1567]: 2025-08-13 01:29:18.770 [INFO][7517] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:18.792487 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.792487 containerd[1567]: 2025-08-13 01:29:18.775 [INFO][7517] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:18.792487 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.792487 containerd[1567]: 2025-08-13 01:29:18.778 [INFO][7517] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12911", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.26/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"5e:1b:5c:63:6f:94", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:18.792487 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.794067 containerd[1567]: 2025-08-13 01:29:18.788 [INFO][7517] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:18.794067 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:18.887138 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:18.887147 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:19.001625 containerd[1567]: 2025-08-13 01:29:18.876 [INFO][7555] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Aug 13 01:29:19.001625 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.001625 containerd[1567]: 2025-08-13 01:29:18.876 [INFO][7555] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" iface="eth0" netns="/var/run/netns/cni-4dcdeb58-87fa-a6cc-26af-b442fbc567ba" Aug 13 01:29:19.001625 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.001625 containerd[1567]: 2025-08-13 01:29:18.882 [INFO][7555] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" iface="eth0" netns="/var/run/netns/cni-4dcdeb58-87fa-a6cc-26af-b442fbc567ba" Aug 13 01:29:19.001625 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.001625 containerd[1567]: 2025-08-13 01:29:18.894 [INFO][7555] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" after=14.693701ms iface="eth0" netns="/var/run/netns/cni-4dcdeb58-87fa-a6cc-26af-b442fbc567ba" Aug 13 01:29:19.001625 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.001625 containerd[1567]: 2025-08-13 01:29:18.896 [INFO][7555] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Aug 13 01:29:19.001625 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.001625 containerd[1567]: 2025-08-13 01:29:18.896 [INFO][7555] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Aug 13 01:29:19.001625 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.001625 containerd[1567]: 2025-08-13 01:29:18.963 [INFO][7571] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" HandleID="k8s-pod-network.84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:19.001625 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.001625 containerd[1567]: 2025-08-13 01:29:18.964 [INFO][7571] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:19.001625 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.001625 containerd[1567]: 2025-08-13 01:29:18.964 [INFO][7571] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:19.001625 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.001625 containerd[1567]: 2025-08-13 01:29:18.994 [INFO][7571] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" HandleID="k8s-pod-network.84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:19.001625 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.001625 containerd[1567]: 2025-08-13 01:29:18.994 [INFO][7571] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" HandleID="k8s-pod-network.84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:19.001625 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.001625 containerd[1567]: 2025-08-13 01:29:18.996 [INFO][7571] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:19.001625 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.001625 containerd[1567]: 2025-08-13 01:29:18.999 [INFO][7555] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402" Aug 13 01:29:19.001625 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.004444 systemd[1]: run-netns-cni\x2d4dcdeb58\x2d87fa\x2da6cc\x2d26af\x2db442fbc567ba.mount: Deactivated successfully. Aug 13 01:29:19.004824 containerd[1567]: time="2025-08-13T01:29:19.004774470Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402-lo: no space left on device" Aug 13 01:29:19.005292 kubelet[2720]: E0813 01:29:19.005247 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402-lo: no space left on device" Aug 13 01:29:19.005440 kubelet[2720]: E0813 01:29:19.005306 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:19.005440 kubelet[2720]: E0813 01:29:19.005327 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:19.005440 kubelet[2720]: E0813 01:29:19.005373 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:19.008510 sshd[7529]: Connection closed by 147.75.109.163 port 40648 Aug 13 01:29:19.009007 sshd-session[7514]: pam_unix(sshd:session): session closed for user core Aug 13 01:29:19.012570 systemd[1]: sshd@29-172.234.25.215:22-147.75.109.163:40648.service: Deactivated successfully. Aug 13 01:29:19.015053 systemd[1]: session-30.scope: Deactivated successfully. Aug 13 01:29:19.017380 systemd-logind[1543]: Session 30 logged out. Waiting for processes to exit. Aug 13 01:29:19.019244 systemd-logind[1543]: Removed session 30. Aug 13 01:29:19.653570 containerd[1567]: time="2025-08-13T01:29:19.653448091Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:19.766900 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:19.770001 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.691 [INFO][7586] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 12977 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.691 [INFO][7586] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.721 [INFO][7600] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" HandleID="k8s-pod-network.65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.722 [INFO][7600] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" HandleID="k8s-pod-network.65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5600), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:19.72158893 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.722 [INFO][7600] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.722 [INFO][7600] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.722 [INFO][7600] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.728 [INFO][7600] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" host="172-234-25-215" Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.733 [INFO][7600] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.736 [INFO][7600] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.738 [INFO][7600] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.740 [INFO][7600] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.740 [INFO][7600] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" host="172-234-25-215" Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.743 [INFO][7600] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52 Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.748 [INFO][7600] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" host="172-234-25-215" Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.755 [INFO][7600] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.27/26] block=192.168.62.0/26 handle="k8s-pod-network.65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" host="172-234-25-215" Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.787921 containerd[1567]: 2025-08-13 01:29:19.755 [INFO][7600] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.27/26] handle="k8s-pod-network.65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" host="172-234-25-215" Aug 13 01:29:19.787921 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.788624 containerd[1567]: 2025-08-13 01:29:19.756 [INFO][7600] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:19.788624 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.788624 containerd[1567]: 2025-08-13 01:29:19.756 [INFO][7600] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.27/26] IPv6=[] ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" HandleID="k8s-pod-network.65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:19.788624 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.788624 containerd[1567]: 2025-08-13 01:29:19.761 [INFO][7586] cni-plugin/k8s.go 418: Populated endpoint ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12977", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"84df1129e9aab72a079fe75136e34188b31f74a12851fbc846eb8f18fd954402", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.27/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:19.788624 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.788624 containerd[1567]: 2025-08-13 01:29:19.761 [INFO][7586] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.27/32] ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:19.788624 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.788624 containerd[1567]: 2025-08-13 01:29:19.761 [INFO][7586] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:19.788624 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.788624 containerd[1567]: 2025-08-13 01:29:19.770 [INFO][7586] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:19.788624 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.788624 containerd[1567]: 2025-08-13 01:29:19.771 [INFO][7586] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"12977", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.27/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ce:e7:17:5c:36:15", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:19.788624 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.791844 containerd[1567]: 2025-08-13 01:29:19.782 [INFO][7586] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:19.791844 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.864763 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:19.864773 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:19.947864 containerd[1567]: 2025-08-13 01:29:19.860 [INFO][7623] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Aug 13 01:29:19.947864 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.947864 containerd[1567]: 2025-08-13 01:29:19.861 [INFO][7623] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" iface="eth0" netns="/var/run/netns/cni-d8d1b534-0ae2-3811-fd62-48f2223ebc2c" Aug 13 01:29:19.947864 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.947864 containerd[1567]: 2025-08-13 01:29:19.862 [INFO][7623] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" iface="eth0" netns="/var/run/netns/cni-d8d1b534-0ae2-3811-fd62-48f2223ebc2c" Aug 13 01:29:19.947864 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.947864 containerd[1567]: 2025-08-13 01:29:19.871 [INFO][7623] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" after=10.73365ms iface="eth0" netns="/var/run/netns/cni-d8d1b534-0ae2-3811-fd62-48f2223ebc2c" Aug 13 01:29:19.947864 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.947864 containerd[1567]: 2025-08-13 01:29:19.872 [INFO][7623] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Aug 13 01:29:19.947864 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.947864 containerd[1567]: 2025-08-13 01:29:19.872 [INFO][7623] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Aug 13 01:29:19.947864 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.947864 containerd[1567]: 2025-08-13 01:29:19.906 [INFO][7633] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" HandleID="k8s-pod-network.65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:19.947864 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.947864 containerd[1567]: 2025-08-13 01:29:19.907 [INFO][7633] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:19.947864 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.947864 containerd[1567]: 2025-08-13 01:29:19.907 [INFO][7633] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:19.947864 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.947864 containerd[1567]: 2025-08-13 01:29:19.940 [INFO][7633] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" HandleID="k8s-pod-network.65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:19.947864 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.947864 containerd[1567]: 2025-08-13 01:29:19.941 [INFO][7633] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" HandleID="k8s-pod-network.65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:19.947864 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.947864 containerd[1567]: 2025-08-13 01:29:19.942 [INFO][7633] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:19.947864 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.947864 containerd[1567]: 2025-08-13 01:29:19.944 [INFO][7623] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52" Aug 13 01:29:19.947864 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:19.955172 containerd[1567]: time="2025-08-13T01:29:19.953899722Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52-lo: no space left on device" Aug 13 01:29:19.955282 kubelet[2720]: E0813 01:29:19.954181 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52-lo: no space left on device" Aug 13 01:29:19.955282 kubelet[2720]: E0813 01:29:19.954233 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:19.955282 kubelet[2720]: E0813 01:29:19.954255 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:19.955282 kubelet[2720]: E0813 01:29:19.955239 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:19.954120 systemd[1]: run-netns-cni\x2dd8d1b534\x2d0ae2\x2d3811\x2dfd62\x2d48f2223ebc2c.mount: Deactivated successfully. Aug 13 01:29:20.656894 containerd[1567]: time="2025-08-13T01:29:20.656850420Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:20.769645 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:20.771577 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.697 [INFO][7642] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 13032 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.697 [INFO][7642] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.723 [INFO][7656] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" HandleID="k8s-pod-network.dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.723 [INFO][7656] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" HandleID="k8s-pod-network.dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f1e0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:20.723195292 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.723 [INFO][7656] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.723 [INFO][7656] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.723 [INFO][7656] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.730 [INFO][7656] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" host="172-234-25-215" Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.735 [INFO][7656] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.739 [INFO][7656] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.741 [INFO][7656] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.744 [INFO][7656] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.744 [INFO][7656] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" host="172-234-25-215" Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.746 [INFO][7656] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43 Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.750 [INFO][7656] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" host="172-234-25-215" Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.760 [INFO][7656] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.28/26] block=192.168.62.0/26 handle="k8s-pod-network.dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" host="172-234-25-215" Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.785185 containerd[1567]: 2025-08-13 01:29:20.760 [INFO][7656] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.28/26] handle="k8s-pod-network.dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" host="172-234-25-215" Aug 13 01:29:20.785185 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.787299 containerd[1567]: 2025-08-13 01:29:20.760 [INFO][7656] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:20.787299 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.787299 containerd[1567]: 2025-08-13 01:29:20.760 [INFO][7656] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.28/26] IPv6=[] ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" HandleID="k8s-pod-network.dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:20.787299 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.787299 containerd[1567]: 2025-08-13 01:29:20.763 [INFO][7642] cni-plugin/k8s.go 418: Populated endpoint ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13032", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"65915f5844c5fb656e4ea537ee31eabd2bb1069cbb026e9aa8ec9bc35b677b52", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.28/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:20.787299 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.787299 containerd[1567]: 2025-08-13 01:29:20.763 [INFO][7642] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.28/32] ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:20.787299 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.787299 containerd[1567]: 2025-08-13 01:29:20.763 [INFO][7642] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:20.787299 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.787299 containerd[1567]: 2025-08-13 01:29:20.771 [INFO][7642] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:20.787299 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.787299 containerd[1567]: 2025-08-13 01:29:20.772 [INFO][7642] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13032", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.28/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"46:2d:ab:da:5c:c6", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:20.787299 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.788027 containerd[1567]: 2025-08-13 01:29:20.781 [INFO][7642] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:20.788027 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.851195 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:20.851205 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:20.941088 containerd[1567]: 2025-08-13 01:29:20.846 [INFO][7680] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Aug 13 01:29:20.941088 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.941088 containerd[1567]: 2025-08-13 01:29:20.847 [INFO][7680] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" iface="eth0" netns="/var/run/netns/cni-6b962e61-5ebe-d2a2-f86d-016b3d7e8943" Aug 13 01:29:20.941088 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.941088 containerd[1567]: 2025-08-13 01:29:20.849 [INFO][7680] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" iface="eth0" netns="/var/run/netns/cni-6b962e61-5ebe-d2a2-f86d-016b3d7e8943" Aug 13 01:29:20.941088 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.941088 containerd[1567]: 2025-08-13 01:29:20.860 [INFO][7680] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" after=13.028265ms iface="eth0" netns="/var/run/netns/cni-6b962e61-5ebe-d2a2-f86d-016b3d7e8943" Aug 13 01:29:20.941088 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.941088 containerd[1567]: 2025-08-13 01:29:20.861 [INFO][7680] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Aug 13 01:29:20.941088 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.941088 containerd[1567]: 2025-08-13 01:29:20.861 [INFO][7680] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Aug 13 01:29:20.941088 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.941088 containerd[1567]: 2025-08-13 01:29:20.897 [INFO][7689] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" HandleID="k8s-pod-network.dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:20.941088 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.941088 containerd[1567]: 2025-08-13 01:29:20.897 [INFO][7689] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:20.941088 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.941088 containerd[1567]: 2025-08-13 01:29:20.897 [INFO][7689] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:20.941088 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.941088 containerd[1567]: 2025-08-13 01:29:20.934 [INFO][7689] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" HandleID="k8s-pod-network.dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:20.941088 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.941088 containerd[1567]: 2025-08-13 01:29:20.934 [INFO][7689] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" HandleID="k8s-pod-network.dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:20.941088 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.941088 containerd[1567]: 2025-08-13 01:29:20.936 [INFO][7689] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:20.941088 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.941088 containerd[1567]: 2025-08-13 01:29:20.938 [INFO][7680] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43" Aug 13 01:29:20.941088 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:20.945199 containerd[1567]: time="2025-08-13T01:29:20.942494331Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43-lo: no space left on device" Aug 13 01:29:20.945268 kubelet[2720]: E0813 01:29:20.942885 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43-lo: no space left on device" Aug 13 01:29:20.945268 kubelet[2720]: E0813 01:29:20.942947 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:20.945268 kubelet[2720]: E0813 01:29:20.942968 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:20.945268 kubelet[2720]: E0813 01:29:20.943166 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:20.947217 systemd[1]: run-netns-cni\x2d6b962e61\x2d5ebe\x2dd2a2\x2df86d\x2d016b3d7e8943.mount: Deactivated successfully. Aug 13 01:29:21.343885 containerd[1567]: time="2025-08-13T01:29:21.343838539Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:29:21.351072 containerd[1567]: time="2025-08-13T01:29:21.351010425Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:21.659183 containerd[1567]: time="2025-08-13T01:29:21.659060353Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:21.770939 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:21.773546 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.697 [INFO][7698] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 13099 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.697 [INFO][7698] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.728 [INFO][7711] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" HandleID="k8s-pod-network.8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.728 [INFO][7711] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" HandleID="k8s-pod-network.8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:21.727960071 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.728 [INFO][7711] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.728 [INFO][7711] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.728 [INFO][7711] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.734 [INFO][7711] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" host="172-234-25-215" Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.739 [INFO][7711] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.742 [INFO][7711] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.745 [INFO][7711] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.749 [INFO][7711] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.749 [INFO][7711] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" host="172-234-25-215" Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.750 [INFO][7711] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.754 [INFO][7711] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" host="172-234-25-215" Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.762 [INFO][7711] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.29/26] block=192.168.62.0/26 handle="k8s-pod-network.8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" host="172-234-25-215" Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.794599 containerd[1567]: 2025-08-13 01:29:21.762 [INFO][7711] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.29/26] handle="k8s-pod-network.8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" host="172-234-25-215" Aug 13 01:29:21.794599 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.797789 containerd[1567]: 2025-08-13 01:29:21.762 [INFO][7711] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:21.797789 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.797789 containerd[1567]: 2025-08-13 01:29:21.763 [INFO][7711] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.29/26] IPv6=[] ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" HandleID="k8s-pod-network.8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:21.797789 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.797789 containerd[1567]: 2025-08-13 01:29:21.766 [INFO][7698] cni-plugin/k8s.go 418: Populated endpoint ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13099", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"dc8ee123b5f9d8fcf51c03f50d791bc9e2974f834abcdc372d825f545aadcc43", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.29/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:21.797789 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.797789 containerd[1567]: 2025-08-13 01:29:21.766 [INFO][7698] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.29/32] ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:21.797789 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.797789 containerd[1567]: 2025-08-13 01:29:21.767 [INFO][7698] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:21.797789 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.797789 containerd[1567]: 2025-08-13 01:29:21.774 [INFO][7698] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:21.797789 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.797789 containerd[1567]: 2025-08-13 01:29:21.776 [INFO][7698] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13099", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.29/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"1e:aa:f3:44:a0:f2", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:21.797789 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.798525 containerd[1567]: 2025-08-13 01:29:21.787 [INFO][7698] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:21.798525 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.889704 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:21.890752 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:21.981328 containerd[1567]: 2025-08-13 01:29:21.887 [INFO][7734] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Aug 13 01:29:21.981328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.981328 containerd[1567]: 2025-08-13 01:29:21.887 [INFO][7734] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" iface="eth0" netns="/var/run/netns/cni-d550ed94-41ba-16e8-1ecc-664464caeece" Aug 13 01:29:21.981328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.981328 containerd[1567]: 2025-08-13 01:29:21.888 [INFO][7734] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" iface="eth0" netns="/var/run/netns/cni-d550ed94-41ba-16e8-1ecc-664464caeece" Aug 13 01:29:21.981328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.981328 containerd[1567]: 2025-08-13 01:29:21.898 [INFO][7734] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" after=10.812909ms iface="eth0" netns="/var/run/netns/cni-d550ed94-41ba-16e8-1ecc-664464caeece" Aug 13 01:29:21.981328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.981328 containerd[1567]: 2025-08-13 01:29:21.898 [INFO][7734] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Aug 13 01:29:21.981328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.981328 containerd[1567]: 2025-08-13 01:29:21.898 [INFO][7734] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Aug 13 01:29:21.981328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.981328 containerd[1567]: 2025-08-13 01:29:21.940 [INFO][7745] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" HandleID="k8s-pod-network.8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:21.981328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.981328 containerd[1567]: 2025-08-13 01:29:21.940 [INFO][7745] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:21.981328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.981328 containerd[1567]: 2025-08-13 01:29:21.940 [INFO][7745] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:21.981328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.981328 containerd[1567]: 2025-08-13 01:29:21.975 [INFO][7745] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" HandleID="k8s-pod-network.8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:21.981328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.981328 containerd[1567]: 2025-08-13 01:29:21.975 [INFO][7745] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" HandleID="k8s-pod-network.8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:21.981328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.981328 containerd[1567]: 2025-08-13 01:29:21.976 [INFO][7745] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:21.981328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.981328 containerd[1567]: 2025-08-13 01:29:21.978 [INFO][7734] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e" Aug 13 01:29:21.981328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:21.983325 systemd[1]: run-netns-cni\x2dd550ed94\x2d41ba\x2d16e8\x2d1ecc\x2d664464caeece.mount: Deactivated successfully. Aug 13 01:29:21.984320 containerd[1567]: time="2025-08-13T01:29:21.984286028Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e-lo: no space left on device" Aug 13 01:29:21.985383 kubelet[2720]: E0813 01:29:21.984992 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e-lo: no space left on device" Aug 13 01:29:21.986478 kubelet[2720]: E0813 01:29:21.985486 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:21.986478 kubelet[2720]: E0813 01:29:21.985511 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:21.986478 kubelet[2720]: E0813 01:29:21.986355 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:22.080742 kubelet[2720]: E0813 01:29:22.079971 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:29:22.082018 kubelet[2720]: E0813 01:29:22.081405 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-304131310-cnVI sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-927608295: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:29:22.661511 containerd[1567]: time="2025-08-13T01:29:22.661451510Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:22.767456 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:22.768875 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.697 [INFO][7758] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 13173 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.698 [INFO][7758] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.727 [INFO][7770] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" HandleID="k8s-pod-network.ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.727 [INFO][7770] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" HandleID="k8s-pod-network.ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:22.727344392 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.727 [INFO][7770] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.727 [INFO][7770] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.727 [INFO][7770] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.734 [INFO][7770] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" host="172-234-25-215" Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.739 [INFO][7770] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.744 [INFO][7770] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.746 [INFO][7770] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.748 [INFO][7770] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.748 [INFO][7770] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" host="172-234-25-215" Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.751 [INFO][7770] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581 Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.755 [INFO][7770] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" host="172-234-25-215" Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.760 [INFO][7770] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.30/26] block=192.168.62.0/26 handle="k8s-pod-network.ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" host="172-234-25-215" Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.780491 containerd[1567]: 2025-08-13 01:29:22.760 [INFO][7770] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.30/26] handle="k8s-pod-network.ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" host="172-234-25-215" Aug 13 01:29:22.780491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.781236 containerd[1567]: 2025-08-13 01:29:22.760 [INFO][7770] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:22.781236 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.781236 containerd[1567]: 2025-08-13 01:29:22.760 [INFO][7770] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.30/26] IPv6=[] ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" HandleID="k8s-pod-network.ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:22.781236 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.781236 containerd[1567]: 2025-08-13 01:29:22.764 [INFO][7758] cni-plugin/k8s.go 418: Populated endpoint ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13173", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"8e097c13ae3c92436d6ae098ec61b4a7fc0f2df1a7fc99c045bb0ae736a8e21e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.30/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:22.781236 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.781236 containerd[1567]: 2025-08-13 01:29:22.764 [INFO][7758] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.30/32] ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:22.781236 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.781236 containerd[1567]: 2025-08-13 01:29:22.764 [INFO][7758] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:22.781236 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.781236 containerd[1567]: 2025-08-13 01:29:22.769 [INFO][7758] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:22.781236 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.781236 containerd[1567]: 2025-08-13 01:29:22.770 [INFO][7758] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13173", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.30/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"92:44:38:45:dd:82", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:22.781236 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.781555 containerd[1567]: 2025-08-13 01:29:22.777 [INFO][7758] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:22.781555 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.847511 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:22.847522 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:22.972164 containerd[1567]: 2025-08-13 01:29:22.845 [INFO][7795] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Aug 13 01:29:22.972164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.972164 containerd[1567]: 2025-08-13 01:29:22.845 [INFO][7795] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" iface="eth0" netns="/var/run/netns/cni-8f723827-90d6-4576-029b-bd10fb6622db" Aug 13 01:29:22.972164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.972164 containerd[1567]: 2025-08-13 01:29:22.846 [INFO][7795] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" iface="eth0" netns="/var/run/netns/cni-8f723827-90d6-4576-029b-bd10fb6622db" Aug 13 01:29:22.972164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.972164 containerd[1567]: 2025-08-13 01:29:22.853 [INFO][7795] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" after=7.051986ms iface="eth0" netns="/var/run/netns/cni-8f723827-90d6-4576-029b-bd10fb6622db" Aug 13 01:29:22.972164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.972164 containerd[1567]: 2025-08-13 01:29:22.853 [INFO][7795] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Aug 13 01:29:22.972164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.972164 containerd[1567]: 2025-08-13 01:29:22.853 [INFO][7795] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Aug 13 01:29:22.972164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.972164 containerd[1567]: 2025-08-13 01:29:22.909 [INFO][7803] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" HandleID="k8s-pod-network.ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:22.972164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.972164 containerd[1567]: 2025-08-13 01:29:22.910 [INFO][7803] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:22.972164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.972164 containerd[1567]: 2025-08-13 01:29:22.910 [INFO][7803] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:22.972164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.972164 containerd[1567]: 2025-08-13 01:29:22.964 [INFO][7803] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" HandleID="k8s-pod-network.ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:22.972164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.972164 containerd[1567]: 2025-08-13 01:29:22.964 [INFO][7803] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" HandleID="k8s-pod-network.ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:22.972164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.972164 containerd[1567]: 2025-08-13 01:29:22.966 [INFO][7803] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:22.972164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.972164 containerd[1567]: 2025-08-13 01:29:22.969 [INFO][7795] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581" Aug 13 01:29:22.972164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:22.974275 containerd[1567]: time="2025-08-13T01:29:22.973631061Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581-lo: no space left on device" Aug 13 01:29:22.974350 kubelet[2720]: E0813 01:29:22.973990 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581-lo: no space left on device" Aug 13 01:29:22.974350 kubelet[2720]: E0813 01:29:22.974046 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:22.974350 kubelet[2720]: E0813 01:29:22.974065 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:22.974350 kubelet[2720]: E0813 01:29:22.974109 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:22.976981 systemd[1]: run-netns-cni\x2d8f723827\x2d90d6\x2d4576\x2d029b\x2dbd10fb6622db.mount: Deactivated successfully. Aug 13 01:29:23.662928 containerd[1567]: time="2025-08-13T01:29:23.662889649Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:23.773063 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:23.774812 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.700 [INFO][7814] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 13244 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.701 [INFO][7814] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.730 [INFO][7826] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" HandleID="k8s-pod-network.1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.730 [INFO][7826] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" HandleID="k8s-pod-network.1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:23.730761169 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.731 [INFO][7826] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.731 [INFO][7826] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.731 [INFO][7826] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.736 [INFO][7826] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" host="172-234-25-215" Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.741 [INFO][7826] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.746 [INFO][7826] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.747 [INFO][7826] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.751 [INFO][7826] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.751 [INFO][7826] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" host="172-234-25-215" Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.754 [INFO][7826] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321 Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.759 [INFO][7826] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" host="172-234-25-215" Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.765 [INFO][7826] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.31/26] block=192.168.62.0/26 handle="k8s-pod-network.1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" host="172-234-25-215" Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.794802 containerd[1567]: 2025-08-13 01:29:23.766 [INFO][7826] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.31/26] handle="k8s-pod-network.1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" host="172-234-25-215" Aug 13 01:29:23.794802 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.795645 containerd[1567]: 2025-08-13 01:29:23.766 [INFO][7826] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:23.795645 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.795645 containerd[1567]: 2025-08-13 01:29:23.766 [INFO][7826] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.31/26] IPv6=[] ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" HandleID="k8s-pod-network.1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:23.795645 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.795645 containerd[1567]: 2025-08-13 01:29:23.769 [INFO][7814] cni-plugin/k8s.go 418: Populated endpoint ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13244", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ac46ac0558c24560318ab6d14a588a7098d14eaafb1848aebaca625fa8afb581", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.31/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:23.795645 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.795645 containerd[1567]: 2025-08-13 01:29:23.769 [INFO][7814] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.31/32] ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:23.795645 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.795645 containerd[1567]: 2025-08-13 01:29:23.769 [INFO][7814] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:23.795645 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.795645 containerd[1567]: 2025-08-13 01:29:23.773 [INFO][7814] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:23.795645 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.795645 containerd[1567]: 2025-08-13 01:29:23.775 [INFO][7814] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13244", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.31/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"f2:54:7d:97:19:4c", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:23.795645 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.797387 containerd[1567]: 2025-08-13 01:29:23.787 [INFO][7814] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:23.797387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.867016 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:23.867027 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:23.961194 containerd[1567]: 2025-08-13 01:29:23.863 [INFO][7850] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Aug 13 01:29:23.961194 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.961194 containerd[1567]: 2025-08-13 01:29:23.864 [INFO][7850] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" iface="eth0" netns="/var/run/netns/cni-132697f2-4832-ffdf-a98d-18cd8c381a08" Aug 13 01:29:23.961194 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.961194 containerd[1567]: 2025-08-13 01:29:23.865 [INFO][7850] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" iface="eth0" netns="/var/run/netns/cni-132697f2-4832-ffdf-a98d-18cd8c381a08" Aug 13 01:29:23.961194 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.961194 containerd[1567]: 2025-08-13 01:29:23.872 [INFO][7850] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" after=7.648045ms iface="eth0" netns="/var/run/netns/cni-132697f2-4832-ffdf-a98d-18cd8c381a08" Aug 13 01:29:23.961194 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.961194 containerd[1567]: 2025-08-13 01:29:23.872 [INFO][7850] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Aug 13 01:29:23.961194 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.961194 containerd[1567]: 2025-08-13 01:29:23.872 [INFO][7850] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Aug 13 01:29:23.961194 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.961194 containerd[1567]: 2025-08-13 01:29:23.926 [INFO][7858] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" HandleID="k8s-pod-network.1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:23.961194 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.961194 containerd[1567]: 2025-08-13 01:29:23.926 [INFO][7858] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:23.961194 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.961194 containerd[1567]: 2025-08-13 01:29:23.927 [INFO][7858] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:23.961194 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.961194 containerd[1567]: 2025-08-13 01:29:23.955 [INFO][7858] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" HandleID="k8s-pod-network.1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:23.961194 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.961194 containerd[1567]: 2025-08-13 01:29:23.955 [INFO][7858] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" HandleID="k8s-pod-network.1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:23.961194 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.961194 containerd[1567]: 2025-08-13 01:29:23.956 [INFO][7858] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:23.961194 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.961194 containerd[1567]: 2025-08-13 01:29:23.958 [INFO][7850] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321" Aug 13 01:29:23.961194 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:23.965311 containerd[1567]: time="2025-08-13T01:29:23.965258440Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321-lo: no space left on device" Aug 13 01:29:23.965760 kubelet[2720]: E0813 01:29:23.965641 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321-lo: no space left on device" Aug 13 01:29:23.965760 kubelet[2720]: E0813 01:29:23.965754 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:23.967786 kubelet[2720]: E0813 01:29:23.965800 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:23.967786 kubelet[2720]: E0813 01:29:23.965876 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:23.966345 systemd[1]: run-netns-cni\x2d132697f2\x2d4832\x2dffdf\x2da98d\x2d18cd8c381a08.mount: Deactivated successfully. Aug 13 01:29:24.073736 kubelet[2720]: E0813 01:29:24.073476 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:29:24.076542 kubelet[2720]: E0813 01:29:24.076506 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-519958916-KETu sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4205936624: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:29:24.078849 systemd[1]: Started sshd@30-172.234.25.215:22-147.75.109.163:40664.service - OpenSSH per-connection server daemon (147.75.109.163:40664). Aug 13 01:29:24.351264 containerd[1567]: time="2025-08-13T01:29:24.351186421Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:24.425476 sshd[7869]: Accepted publickey for core from 147.75.109.163 port 40664 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:29:24.426883 sshd-session[7869]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:29:24.435219 systemd-logind[1543]: New session 31 of user core. Aug 13 01:29:24.444872 systemd[1]: Started session-31.scope - Session 31 of User core. Aug 13 01:29:24.664649 containerd[1567]: time="2025-08-13T01:29:24.664475311Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:24.741308 sshd[7872]: Connection closed by 147.75.109.163 port 40664 Aug 13 01:29:24.741913 sshd-session[7869]: pam_unix(sshd:session): session closed for user core Aug 13 01:29:24.746630 systemd[1]: sshd@30-172.234.25.215:22-147.75.109.163:40664.service: Deactivated successfully. Aug 13 01:29:24.747236 systemd-logind[1543]: Session 31 logged out. Waiting for processes to exit. Aug 13 01:29:24.751513 systemd[1]: session-31.scope: Deactivated successfully. Aug 13 01:29:24.757749 systemd-logind[1543]: Removed session 31. Aug 13 01:29:24.803176 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:24.803364 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.708 [INFO][7883] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 13315 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.709 [INFO][7883] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.737 [INFO][7896] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" HandleID="k8s-pod-network.c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.738 [INFO][7896] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" HandleID="k8s-pod-network.c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5600), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:24.737933021 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.738 [INFO][7896] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.738 [INFO][7896] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.738 [INFO][7896] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.747 [INFO][7896] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" host="172-234-25-215" Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.757 [INFO][7896] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.762 [INFO][7896] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.765 [INFO][7896] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.767 [INFO][7896] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.767 [INFO][7896] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" host="172-234-25-215" Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.769 [INFO][7896] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.775 [INFO][7896] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" host="172-234-25-215" Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.791 [INFO][7896] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.32/26] block=192.168.62.0/26 handle="k8s-pod-network.c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" host="172-234-25-215" Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.830435 containerd[1567]: 2025-08-13 01:29:24.792 [INFO][7896] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.32/26] handle="k8s-pod-network.c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" host="172-234-25-215" Aug 13 01:29:24.830435 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.831323 containerd[1567]: 2025-08-13 01:29:24.792 [INFO][7896] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:24.831323 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.831323 containerd[1567]: 2025-08-13 01:29:24.793 [INFO][7896] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.32/26] IPv6=[] ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" HandleID="k8s-pod-network.c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:24.831323 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.831323 containerd[1567]: 2025-08-13 01:29:24.797 [INFO][7883] cni-plugin/k8s.go 418: Populated endpoint ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13315", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"1ed2708bbea3dede08f81aa79815c3a887dfe722e5faa903d33af1656ab83321", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.32/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:24.831323 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.831323 containerd[1567]: 2025-08-13 01:29:24.797 [INFO][7883] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.32/32] ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:24.831323 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.831323 containerd[1567]: 2025-08-13 01:29:24.797 [INFO][7883] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:24.831323 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.831323 containerd[1567]: 2025-08-13 01:29:24.803 [INFO][7883] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:24.831323 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.831323 containerd[1567]: 2025-08-13 01:29:24.804 [INFO][7883] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13315", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.32/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ba:cc:f3:3f:05:df", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:24.831323 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.831703 containerd[1567]: 2025-08-13 01:29:24.825 [INFO][7883] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:24.831703 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:24.906225 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:24.906236 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:25.024497 containerd[1567]: 2025-08-13 01:29:24.904 [INFO][7921] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Aug 13 01:29:25.024497 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.024497 containerd[1567]: 2025-08-13 01:29:24.904 [INFO][7921] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" iface="eth0" netns="/var/run/netns/cni-610c8cce-7cc3-3085-1b59-72ccc0f5f6b9" Aug 13 01:29:25.024497 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.024497 containerd[1567]: 2025-08-13 01:29:24.905 [INFO][7921] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" iface="eth0" netns="/var/run/netns/cni-610c8cce-7cc3-3085-1b59-72ccc0f5f6b9" Aug 13 01:29:25.024497 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.024497 containerd[1567]: 2025-08-13 01:29:24.913 [INFO][7921] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" after=9.073822ms iface="eth0" netns="/var/run/netns/cni-610c8cce-7cc3-3085-1b59-72ccc0f5f6b9" Aug 13 01:29:25.024497 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.024497 containerd[1567]: 2025-08-13 01:29:24.914 [INFO][7921] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Aug 13 01:29:25.024497 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.024497 containerd[1567]: 2025-08-13 01:29:24.914 [INFO][7921] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Aug 13 01:29:25.024497 containerd[1567]: 2025-08-13 01:29:24.914 [INFO][7921] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Aug 13 01:29:25.024497 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.024497 containerd[1567]: 2025-08-13 01:29:24.952 [INFO][7928] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" HandleID="k8s-pod-network.c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:25.024497 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.024497 containerd[1567]: 2025-08-13 01:29:24.952 [INFO][7928] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:25.024497 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.024497 containerd[1567]: 2025-08-13 01:29:24.953 [INFO][7928] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:25.024497 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.024497 containerd[1567]: 2025-08-13 01:29:25.015 [INFO][7928] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" HandleID="k8s-pod-network.c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:25.024497 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.024497 containerd[1567]: 2025-08-13 01:29:25.015 [INFO][7928] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" HandleID="k8s-pod-network.c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:25.024497 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.024497 containerd[1567]: 2025-08-13 01:29:25.017 [INFO][7928] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:25.024497 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.024497 containerd[1567]: 2025-08-13 01:29:25.020 [INFO][7921] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba" Aug 13 01:29:25.024497 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.083688 systemd-journald[1206]: Failed to create new system journal: No space left on device Aug 13 01:29:25.083747 systemd-journald[1206]: Failed to open system journal: No space left on device Aug 13 01:29:25.031555 systemd[1]: run-netns-cni\x2d610c8cce\x2d7cc3\x2d3085\x2d1b59\x2d72ccc0f5f6b9.mount: Deactivated successfully. Aug 13 01:29:25.084019 containerd[1567]: time="2025-08-13T01:29:25.033796175Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba-lo: no space left on device" Aug 13 01:29:25.084089 kubelet[2720]: E0813 01:29:25.034060 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba-lo: no space left on device" Aug 13 01:29:25.084089 kubelet[2720]: E0813 01:29:25.034120 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:25.084089 kubelet[2720]: E0813 01:29:25.034140 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:25.084089 kubelet[2720]: E0813 01:29:25.034192 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:25.666322 containerd[1567]: time="2025-08-13T01:29:25.666281836Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:25.767503 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:25.768996 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.702 [INFO][7935] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 13391 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.702 [INFO][7935] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.726 [INFO][7948] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" HandleID="k8s-pod-network.f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.726 [INFO][7948] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" HandleID="k8s-pod-network.f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f020), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:25.726453741 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.726 [INFO][7948] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.726 [INFO][7948] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.726 [INFO][7948] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.732 [INFO][7948] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" host="172-234-25-215" Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.736 [INFO][7948] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.740 [INFO][7948] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.741 [INFO][7948] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.743 [INFO][7948] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.743 [INFO][7948] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" host="172-234-25-215" Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.745 [INFO][7948] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.750 [INFO][7948] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" host="172-234-25-215" Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.759 [INFO][7948] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.33/26] block=192.168.62.0/26 handle="k8s-pod-network.f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" host="172-234-25-215" Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.784389 containerd[1567]: 2025-08-13 01:29:25.759 [INFO][7948] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.33/26] handle="k8s-pod-network.f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" host="172-234-25-215" Aug 13 01:29:25.784389 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.785580 containerd[1567]: 2025-08-13 01:29:25.759 [INFO][7948] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:25.785580 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.785580 containerd[1567]: 2025-08-13 01:29:25.759 [INFO][7948] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.33/26] IPv6=[] ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" HandleID="k8s-pod-network.f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:25.785580 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.785580 containerd[1567]: 2025-08-13 01:29:25.763 [INFO][7935] cni-plugin/k8s.go 418: Populated endpoint ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13391", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c92cedda4dc2b2ca43f60f630a4d0dde59935bb87268b5ea21b55d23b4792eba", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.33/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:25.785580 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.785580 containerd[1567]: 2025-08-13 01:29:25.763 [INFO][7935] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.33/32] ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:25.785580 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.785580 containerd[1567]: 2025-08-13 01:29:25.763 [INFO][7935] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:25.785580 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.785580 containerd[1567]: 2025-08-13 01:29:25.768 [INFO][7935] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:25.785580 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.785580 containerd[1567]: 2025-08-13 01:29:25.769 [INFO][7935] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13391", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.33/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"a2:94:1f:fd:b6:86", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:25.785580 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.786442 containerd[1567]: 2025-08-13 01:29:25.779 [INFO][7935] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:25.786442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.840043 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:25.840470 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:25.928486 containerd[1567]: 2025-08-13 01:29:25.836 [INFO][7973] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Aug 13 01:29:25.928486 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.928486 containerd[1567]: 2025-08-13 01:29:25.837 [INFO][7973] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" iface="eth0" netns="/var/run/netns/cni-069936ba-8b16-c18a-96d4-f2bc715e7ab9" Aug 13 01:29:25.928486 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.928486 containerd[1567]: 2025-08-13 01:29:25.838 [INFO][7973] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" iface="eth0" netns="/var/run/netns/cni-069936ba-8b16-c18a-96d4-f2bc715e7ab9" Aug 13 01:29:25.928486 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.928486 containerd[1567]: 2025-08-13 01:29:25.845 [INFO][7973] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" after=8.049134ms iface="eth0" netns="/var/run/netns/cni-069936ba-8b16-c18a-96d4-f2bc715e7ab9" Aug 13 01:29:25.928486 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.928486 containerd[1567]: 2025-08-13 01:29:25.846 [INFO][7973] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Aug 13 01:29:25.928486 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.928486 containerd[1567]: 2025-08-13 01:29:25.846 [INFO][7973] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Aug 13 01:29:25.928486 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.928486 containerd[1567]: 2025-08-13 01:29:25.889 [INFO][7983] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" HandleID="k8s-pod-network.f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:25.928486 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.928486 containerd[1567]: 2025-08-13 01:29:25.889 [INFO][7983] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:25.928486 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.928486 containerd[1567]: 2025-08-13 01:29:25.889 [INFO][7983] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:25.928486 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.928486 containerd[1567]: 2025-08-13 01:29:25.922 [INFO][7983] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" HandleID="k8s-pod-network.f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:25.928486 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.928486 containerd[1567]: 2025-08-13 01:29:25.923 [INFO][7983] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" HandleID="k8s-pod-network.f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:25.928486 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.928486 containerd[1567]: 2025-08-13 01:29:25.924 [INFO][7983] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:25.928486 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.928486 containerd[1567]: 2025-08-13 01:29:25.926 [INFO][7973] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e" Aug 13 01:29:25.928486 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:25.931477 containerd[1567]: time="2025-08-13T01:29:25.931424519Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e-lo: no space left on device" Aug 13 01:29:25.932152 kubelet[2720]: E0813 01:29:25.931855 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e-lo: no space left on device" Aug 13 01:29:25.932152 kubelet[2720]: E0813 01:29:25.931908 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:25.932152 kubelet[2720]: E0813 01:29:25.931931 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:25.932152 kubelet[2720]: E0813 01:29:25.931976 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:25.933044 systemd[1]: run-netns-cni\x2d069936ba\x2d8b16\x2dc18a\x2d96d4\x2df2bc715e7ab9.mount: Deactivated successfully. Aug 13 01:29:26.668849 containerd[1567]: time="2025-08-13T01:29:26.668708012Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:26.775320 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:26.777102 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.704 [INFO][7992] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 13456 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.704 [INFO][7992] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.731 [INFO][8005] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" HandleID="k8s-pod-network.0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.731 [INFO][8005] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" HandleID="k8s-pod-network.0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5640), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:26.731568021 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.731 [INFO][8005] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.731 [INFO][8005] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.731 [INFO][8005] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.737 [INFO][8005] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" host="172-234-25-215" Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.741 [INFO][8005] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.744 [INFO][8005] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.746 [INFO][8005] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.750 [INFO][8005] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.751 [INFO][8005] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" host="172-234-25-215" Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.754 [INFO][8005] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991 Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.758 [INFO][8005] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" host="172-234-25-215" Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.765 [INFO][8005] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.34/26] block=192.168.62.0/26 handle="k8s-pod-network.0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" host="172-234-25-215" Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.790732 containerd[1567]: 2025-08-13 01:29:26.765 [INFO][8005] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.34/26] handle="k8s-pod-network.0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" host="172-234-25-215" Aug 13 01:29:26.790732 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.792550 containerd[1567]: 2025-08-13 01:29:26.765 [INFO][8005] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:26.792550 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.792550 containerd[1567]: 2025-08-13 01:29:26.765 [INFO][8005] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.34/26] IPv6=[] ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" HandleID="k8s-pod-network.0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:26.792550 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.792550 containerd[1567]: 2025-08-13 01:29:26.768 [INFO][7992] cni-plugin/k8s.go 418: Populated endpoint ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13456", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"f5fd01c40bd2b6cd948771de6889d049d486890ed27434c29b437ba47521632e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.34/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:26.792550 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.792550 containerd[1567]: 2025-08-13 01:29:26.769 [INFO][7992] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.34/32] ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:26.792550 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.792550 containerd[1567]: 2025-08-13 01:29:26.769 [INFO][7992] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:26.792550 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.792550 containerd[1567]: 2025-08-13 01:29:26.776 [INFO][7992] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:26.792550 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.792550 containerd[1567]: 2025-08-13 01:29:26.777 [INFO][7992] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13456", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.34/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"de:22:28:33:fb:bf", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:26.792550 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.793242 containerd[1567]: 2025-08-13 01:29:26.786 [INFO][7992] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:26.793242 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.848334 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:26.848445 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:26.959593 containerd[1567]: 2025-08-13 01:29:26.846 [INFO][8030] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Aug 13 01:29:26.959593 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.959593 containerd[1567]: 2025-08-13 01:29:26.847 [INFO][8030] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" iface="eth0" netns="/var/run/netns/cni-4dfac525-2c25-5f90-103c-87bdd32cf0e6" Aug 13 01:29:26.959593 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.959593 containerd[1567]: 2025-08-13 01:29:26.847 [INFO][8030] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" iface="eth0" netns="/var/run/netns/cni-4dfac525-2c25-5f90-103c-87bdd32cf0e6" Aug 13 01:29:26.959593 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.959593 containerd[1567]: 2025-08-13 01:29:26.853 [INFO][8030] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" after=6.171008ms iface="eth0" netns="/var/run/netns/cni-4dfac525-2c25-5f90-103c-87bdd32cf0e6" Aug 13 01:29:26.959593 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.959593 containerd[1567]: 2025-08-13 01:29:26.853 [INFO][8030] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Aug 13 01:29:26.959593 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.959593 containerd[1567]: 2025-08-13 01:29:26.854 [INFO][8030] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Aug 13 01:29:26.959593 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.959593 containerd[1567]: 2025-08-13 01:29:26.882 [INFO][8038] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" HandleID="k8s-pod-network.0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:26.959593 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.959593 containerd[1567]: 2025-08-13 01:29:26.883 [INFO][8038] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:26.959593 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.959593 containerd[1567]: 2025-08-13 01:29:26.883 [INFO][8038] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:26.959593 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.959593 containerd[1567]: 2025-08-13 01:29:26.949 [INFO][8038] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" HandleID="k8s-pod-network.0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:26.959593 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.959593 containerd[1567]: 2025-08-13 01:29:26.950 [INFO][8038] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" HandleID="k8s-pod-network.0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:26.959593 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.959593 containerd[1567]: 2025-08-13 01:29:26.952 [INFO][8038] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:26.959593 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.959593 containerd[1567]: 2025-08-13 01:29:26.955 [INFO][8030] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991" Aug 13 01:29:26.959593 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:26.964030 containerd[1567]: time="2025-08-13T01:29:26.963468029Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991-lo: no space left on device" Aug 13 01:29:26.965366 kubelet[2720]: E0813 01:29:26.964245 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991-lo: no space left on device" Aug 13 01:29:26.965366 kubelet[2720]: E0813 01:29:26.964384 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:26.965366 kubelet[2720]: E0813 01:29:26.964402 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:26.965708 systemd[1]: run-netns-cni\x2d4dfac525\x2d2c25\x2d5f90\x2d103c\x2d87bdd32cf0e6.mount: Deactivated successfully. Aug 13 01:29:26.967447 kubelet[2720]: E0813 01:29:26.965958 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:27.073853 kubelet[2720]: E0813 01:29:27.073822 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:29:27.351671 containerd[1567]: time="2025-08-13T01:29:27.351619608Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:27.670620 containerd[1567]: time="2025-08-13T01:29:27.670515580Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:27.774586 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:27.775649 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.710 [INFO][8047] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 13537 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.710 [INFO][8047] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.736 [INFO][8060] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" HandleID="k8s-pod-network.5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.737 [INFO][8060] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" HandleID="k8s-pod-network.5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002ad4a0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:27.736835685 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.737 [INFO][8060] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.737 [INFO][8060] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.737 [INFO][8060] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.743 [INFO][8060] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" host="172-234-25-215" Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.747 [INFO][8060] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.753 [INFO][8060] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.754 [INFO][8060] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.757 [INFO][8060] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.757 [INFO][8060] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" host="172-234-25-215" Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.758 [INFO][8060] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4 Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.763 [INFO][8060] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" host="172-234-25-215" Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.769 [INFO][8060] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.35/26] block=192.168.62.0/26 handle="k8s-pod-network.5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" host="172-234-25-215" Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.790326 containerd[1567]: 2025-08-13 01:29:27.769 [INFO][8060] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.35/26] handle="k8s-pod-network.5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" host="172-234-25-215" Aug 13 01:29:27.790326 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.793143 containerd[1567]: 2025-08-13 01:29:27.769 [INFO][8060] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:27.793143 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.793143 containerd[1567]: 2025-08-13 01:29:27.769 [INFO][8060] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.35/26] IPv6=[] ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" HandleID="k8s-pod-network.5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:27.793143 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.793143 containerd[1567]: 2025-08-13 01:29:27.771 [INFO][8047] cni-plugin/k8s.go 418: Populated endpoint ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13537", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0809a50eeb6544c3e9138138ff85b804a24e3ed86232a2a35260856956b2c991", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.35/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:27.793143 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.793143 containerd[1567]: 2025-08-13 01:29:27.771 [INFO][8047] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.35/32] ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:27.793143 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.793143 containerd[1567]: 2025-08-13 01:29:27.771 [INFO][8047] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:27.793143 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.793143 containerd[1567]: 2025-08-13 01:29:27.774 [INFO][8047] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:27.793143 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.793143 containerd[1567]: 2025-08-13 01:29:27.776 [INFO][8047] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13537", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.35/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"36:12:ea:e5:28:8b", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:27.793143 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.793431 containerd[1567]: 2025-08-13 01:29:27.784 [INFO][8047] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:27.793431 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.856585 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:27.856699 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:27.940740 containerd[1567]: 2025-08-13 01:29:27.853 [INFO][8083] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Aug 13 01:29:27.940740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.940740 containerd[1567]: 2025-08-13 01:29:27.854 [INFO][8083] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" iface="eth0" netns="/var/run/netns/cni-e23550a9-87d3-edbb-14f0-3ae29681a347" Aug 13 01:29:27.940740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.940740 containerd[1567]: 2025-08-13 01:29:27.854 [INFO][8083] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" iface="eth0" netns="/var/run/netns/cni-e23550a9-87d3-edbb-14f0-3ae29681a347" Aug 13 01:29:27.940740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.940740 containerd[1567]: 2025-08-13 01:29:27.864 [INFO][8083] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" after=10.28106ms iface="eth0" netns="/var/run/netns/cni-e23550a9-87d3-edbb-14f0-3ae29681a347" Aug 13 01:29:27.940740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.940740 containerd[1567]: 2025-08-13 01:29:27.865 [INFO][8083] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Aug 13 01:29:27.940740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.940740 containerd[1567]: 2025-08-13 01:29:27.865 [INFO][8083] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Aug 13 01:29:27.940740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.940740 containerd[1567]: 2025-08-13 01:29:27.895 [INFO][8092] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" HandleID="k8s-pod-network.5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:27.940740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.940740 containerd[1567]: 2025-08-13 01:29:27.896 [INFO][8092] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:27.940740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.940740 containerd[1567]: 2025-08-13 01:29:27.896 [INFO][8092] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:27.940740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.940740 containerd[1567]: 2025-08-13 01:29:27.934 [INFO][8092] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" HandleID="k8s-pod-network.5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:27.940740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.940740 containerd[1567]: 2025-08-13 01:29:27.934 [INFO][8092] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" HandleID="k8s-pod-network.5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:27.940740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.940740 containerd[1567]: 2025-08-13 01:29:27.935 [INFO][8092] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:27.940740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.940740 containerd[1567]: 2025-08-13 01:29:27.938 [INFO][8083] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4" Aug 13 01:29:27.940740 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:27.942688 systemd[1]: run-netns-cni\x2de23550a9\x2d87d3\x2dedbb\x2d14f0\x2d3ae29681a347.mount: Deactivated successfully. Aug 13 01:29:27.945403 containerd[1567]: time="2025-08-13T01:29:27.945300497Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4-lo: no space left on device" Aug 13 01:29:27.945984 kubelet[2720]: E0813 01:29:27.945835 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4-lo: no space left on device" Aug 13 01:29:27.945984 kubelet[2720]: E0813 01:29:27.945895 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:27.945984 kubelet[2720]: E0813 01:29:27.945927 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:27.946197 kubelet[2720]: E0813 01:29:27.946166 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:28.138406 kubelet[2720]: I0813 01:29:28.138373 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:29:28.138406 kubelet[2720]: I0813 01:29:28.138412 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:29:28.140180 kubelet[2720]: I0813 01:29:28.140154 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:29:28.154897 kubelet[2720]: I0813 01:29:28.154855 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:29:28.156666 kubelet[2720]: I0813 01:29:28.156636 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-b2ds8","kube-system/coredns-674b8bbfcf-jlpx4","calico-system/calico-kube-controllers-b57d47f84-lznkx","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:29:28.156760 kubelet[2720]: E0813 01:29:28.156689 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:29:28.156760 kubelet[2720]: E0813 01:29:28.156700 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:29:28.156760 kubelet[2720]: E0813 01:29:28.156706 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:28.156760 kubelet[2720]: E0813 01:29:28.156738 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:29:28.156760 kubelet[2720]: E0813 01:29:28.156748 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:29:28.156760 kubelet[2720]: E0813 01:29:28.156755 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:29:28.156760 kubelet[2720]: E0813 01:29:28.156762 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:29:28.156879 kubelet[2720]: E0813 01:29:28.156772 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:29:28.156879 kubelet[2720]: E0813 01:29:28.156780 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:29:28.156879 kubelet[2720]: E0813 01:29:28.156787 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:29:28.156879 kubelet[2720]: I0813 01:29:28.156813 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:29:28.672808 containerd[1567]: time="2025-08-13T01:29:28.672753901Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:28.768187 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:28.768395 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.707 [INFO][8108] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 13623 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.707 [INFO][8108] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.731 [INFO][8122] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" HandleID="k8s-pod-network.32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.731 [INFO][8122] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" HandleID="k8s-pod-network.32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f290), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:28.731529529 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.731 [INFO][8122] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.731 [INFO][8122] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.731 [INFO][8122] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.738 [INFO][8122] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" host="172-234-25-215" Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.741 [INFO][8122] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.745 [INFO][8122] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.748 [INFO][8122] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.750 [INFO][8122] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.750 [INFO][8122] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" host="172-234-25-215" Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.751 [INFO][8122] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.755 [INFO][8122] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" host="172-234-25-215" Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.760 [INFO][8122] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.36/26] block=192.168.62.0/26 handle="k8s-pod-network.32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" host="172-234-25-215" Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.790806 containerd[1567]: 2025-08-13 01:29:28.760 [INFO][8122] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.36/26] handle="k8s-pod-network.32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" host="172-234-25-215" Aug 13 01:29:28.790806 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.791454 containerd[1567]: 2025-08-13 01:29:28.760 [INFO][8122] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:28.791454 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.791454 containerd[1567]: 2025-08-13 01:29:28.760 [INFO][8122] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.36/26] IPv6=[] ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" HandleID="k8s-pod-network.32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:28.791454 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.791454 containerd[1567]: 2025-08-13 01:29:28.763 [INFO][8108] cni-plugin/k8s.go 418: Populated endpoint ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13623", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"5645a5ff8a6eb8ee610410ca74ec0f78620cab825cc611deda7800d44e20d1f4", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.36/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:28.791454 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.791454 containerd[1567]: 2025-08-13 01:29:28.763 [INFO][8108] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.36/32] ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:28.791454 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.791454 containerd[1567]: 2025-08-13 01:29:28.763 [INFO][8108] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:28.791454 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.791454 containerd[1567]: 2025-08-13 01:29:28.768 [INFO][8108] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:28.791454 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.791454 containerd[1567]: 2025-08-13 01:29:28.770 [INFO][8108] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13623", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.36/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"56:53:b4:be:3a:ce", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:28.791454 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.791770 containerd[1567]: 2025-08-13 01:29:28.784 [INFO][8108] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:28.791770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.854065 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:28.854078 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:28.965769 containerd[1567]: 2025-08-13 01:29:28.851 [INFO][8147] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Aug 13 01:29:28.965769 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.965769 containerd[1567]: 2025-08-13 01:29:28.852 [INFO][8147] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" iface="eth0" netns="/var/run/netns/cni-9e5d4f56-3789-82dd-0257-97c061556fd8" Aug 13 01:29:28.965769 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.965769 containerd[1567]: 2025-08-13 01:29:28.852 [INFO][8147] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" iface="eth0" netns="/var/run/netns/cni-9e5d4f56-3789-82dd-0257-97c061556fd8" Aug 13 01:29:28.965769 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.965769 containerd[1567]: 2025-08-13 01:29:28.861 [INFO][8147] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" after=9.416342ms iface="eth0" netns="/var/run/netns/cni-9e5d4f56-3789-82dd-0257-97c061556fd8" Aug 13 01:29:28.965769 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.965769 containerd[1567]: 2025-08-13 01:29:28.862 [INFO][8147] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Aug 13 01:29:28.965769 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.965769 containerd[1567]: 2025-08-13 01:29:28.863 [INFO][8147] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Aug 13 01:29:28.965769 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.965769 containerd[1567]: 2025-08-13 01:29:28.927 [INFO][8155] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" HandleID="k8s-pod-network.32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:28.965769 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.965769 containerd[1567]: 2025-08-13 01:29:28.927 [INFO][8155] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:28.965769 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.965769 containerd[1567]: 2025-08-13 01:29:28.927 [INFO][8155] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:28.965769 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.965769 containerd[1567]: 2025-08-13 01:29:28.957 [INFO][8155] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" HandleID="k8s-pod-network.32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:28.965769 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.965769 containerd[1567]: 2025-08-13 01:29:28.957 [INFO][8155] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" HandleID="k8s-pod-network.32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:28.965769 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.965769 containerd[1567]: 2025-08-13 01:29:28.960 [INFO][8155] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:28.965769 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.965769 containerd[1567]: 2025-08-13 01:29:28.963 [INFO][8147] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b" Aug 13 01:29:28.965769 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:28.967332 containerd[1567]: time="2025-08-13T01:29:28.967269640Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b-lo: no space left on device" Aug 13 01:29:28.969962 kubelet[2720]: E0813 01:29:28.968888 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b-lo: no space left on device" Aug 13 01:29:28.969962 kubelet[2720]: E0813 01:29:28.968946 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:28.969962 kubelet[2720]: E0813 01:29:28.968969 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:28.969962 kubelet[2720]: E0813 01:29:28.969013 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:28.971000 systemd[1]: run-netns-cni\x2d9e5d4f56\x2d3789\x2d82dd\x2d0257\x2d97c061556fd8.mount: Deactivated successfully. Aug 13 01:29:29.313864 containerd[1567]: time="2025-08-13T01:29:29.313010632Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:29:29.674558 containerd[1567]: time="2025-08-13T01:29:29.674458775Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:29.784362 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:29.785214 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:29.804498 systemd[1]: Started sshd@31-172.234.25.215:22-147.75.109.163:50404.service - OpenSSH per-connection server daemon (147.75.109.163:50404). Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.714 [INFO][8164] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 13684 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.714 [INFO][8164] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.742 [INFO][8178] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" HandleID="k8s-pod-network.0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.742 [INFO][8178] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" HandleID="k8s-pod-network.0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d58d0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:29.742242716 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.742 [INFO][8178] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.742 [INFO][8178] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.742 [INFO][8178] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.748 [INFO][8178] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" host="172-234-25-215" Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.754 [INFO][8178] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.759 [INFO][8178] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.761 [INFO][8178] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.764 [INFO][8178] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.764 [INFO][8178] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" host="172-234-25-215" Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.765 [INFO][8178] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.770 [INFO][8178] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" host="172-234-25-215" Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.775 [INFO][8178] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.37/26] block=192.168.62.0/26 handle="k8s-pod-network.0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" host="172-234-25-215" Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.816351 containerd[1567]: 2025-08-13 01:29:29.776 [INFO][8178] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.37/26] handle="k8s-pod-network.0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" host="172-234-25-215" Aug 13 01:29:29.816351 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.819440 containerd[1567]: 2025-08-13 01:29:29.776 [INFO][8178] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:29.819440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.819440 containerd[1567]: 2025-08-13 01:29:29.776 [INFO][8178] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.37/26] IPv6=[] ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" HandleID="k8s-pod-network.0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:29.819440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.819440 containerd[1567]: 2025-08-13 01:29:29.779 [INFO][8164] cni-plugin/k8s.go 418: Populated endpoint ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13684", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"32950e20fdd9433e2465165a98fc8406f833d50f66f8e46bc9db9442b76b852b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.37/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:29.819440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.819440 containerd[1567]: 2025-08-13 01:29:29.780 [INFO][8164] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.37/32] ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:29.819440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.819440 containerd[1567]: 2025-08-13 01:29:29.780 [INFO][8164] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:29.819440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.819440 containerd[1567]: 2025-08-13 01:29:29.786 [INFO][8164] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:29.819440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.819440 containerd[1567]: 2025-08-13 01:29:29.786 [INFO][8164] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13684", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.37/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"d2:26:1b:db:37:98", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:29.819440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.820393 containerd[1567]: 2025-08-13 01:29:29.805 [INFO][8164] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:29.820393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.894600 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:29.894608 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:29.977253 containerd[1567]: 2025-08-13 01:29:29.891 [INFO][8204] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Aug 13 01:29:29.977253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.977253 containerd[1567]: 2025-08-13 01:29:29.892 [INFO][8204] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" iface="eth0" netns="/var/run/netns/cni-b4f944bf-6fd9-5aac-f1b3-1eccaf3b56d6" Aug 13 01:29:29.977253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.977253 containerd[1567]: 2025-08-13 01:29:29.892 [INFO][8204] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" iface="eth0" netns="/var/run/netns/cni-b4f944bf-6fd9-5aac-f1b3-1eccaf3b56d6" Aug 13 01:29:29.977253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.977253 containerd[1567]: 2025-08-13 01:29:29.898 [INFO][8204] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" after=6.483508ms iface="eth0" netns="/var/run/netns/cni-b4f944bf-6fd9-5aac-f1b3-1eccaf3b56d6" Aug 13 01:29:29.977253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.977253 containerd[1567]: 2025-08-13 01:29:29.899 [INFO][8204] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Aug 13 01:29:29.977253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.977253 containerd[1567]: 2025-08-13 01:29:29.899 [INFO][8204] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Aug 13 01:29:29.977253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.977253 containerd[1567]: 2025-08-13 01:29:29.929 [INFO][8212] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" HandleID="k8s-pod-network.0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:29.977253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.977253 containerd[1567]: 2025-08-13 01:29:29.931 [INFO][8212] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:29.977253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.977253 containerd[1567]: 2025-08-13 01:29:29.931 [INFO][8212] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:29.977253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.977253 containerd[1567]: 2025-08-13 01:29:29.969 [INFO][8212] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" HandleID="k8s-pod-network.0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:29.977253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.977253 containerd[1567]: 2025-08-13 01:29:29.969 [INFO][8212] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" HandleID="k8s-pod-network.0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:29.977253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.977253 containerd[1567]: 2025-08-13 01:29:29.971 [INFO][8212] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:29.977253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.977253 containerd[1567]: 2025-08-13 01:29:29.974 [INFO][8204] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e" Aug 13 01:29:29.977253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:29.979852 containerd[1567]: time="2025-08-13T01:29:29.979811223Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e-lo: no space left on device" Aug 13 01:29:29.981072 kubelet[2720]: E0813 01:29:29.980115 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e-lo: no space left on device" Aug 13 01:29:29.981381 systemd[1]: run-netns-cni\x2db4f944bf\x2d6fd9\x2d5aac\x2df1b3\x2d1eccaf3b56d6.mount: Deactivated successfully. Aug 13 01:29:29.981646 kubelet[2720]: E0813 01:29:29.981368 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:29.981646 kubelet[2720]: E0813 01:29:29.981421 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:29.981646 kubelet[2720]: E0813 01:29:29.981471 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:30.151370 sshd[8187]: Accepted publickey for core from 147.75.109.163 port 50404 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:29:30.154485 sshd-session[8187]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:29:30.160784 systemd-logind[1543]: New session 32 of user core. Aug 13 01:29:30.165847 systemd[1]: Started session-32.scope - Session 32 of User core. Aug 13 01:29:30.352617 containerd[1567]: time="2025-08-13T01:29:30.352536834Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:30.460925 sshd[8224]: Connection closed by 147.75.109.163 port 50404 Aug 13 01:29:30.461468 sshd-session[8187]: pam_unix(sshd:session): session closed for user core Aug 13 01:29:30.465788 systemd[1]: sshd@31-172.234.25.215:22-147.75.109.163:50404.service: Deactivated successfully. Aug 13 01:29:30.468341 systemd[1]: session-32.scope: Deactivated successfully. Aug 13 01:29:30.469798 systemd-logind[1543]: Session 32 logged out. Waiting for processes to exit. Aug 13 01:29:30.471836 systemd-logind[1543]: Removed session 32. Aug 13 01:29:30.676691 containerd[1567]: time="2025-08-13T01:29:30.676593189Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:30.792232 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:30.792466 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.714 [INFO][8236] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 13754 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.715 [INFO][8236] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.740 [INFO][8247] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" HandleID="k8s-pod-network.706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.740 [INFO][8247] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" HandleID="k8s-pod-network.706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f210), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:30.740036948 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.740 [INFO][8247] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.740 [INFO][8247] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.740 [INFO][8247] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.746 [INFO][8247] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" host="172-234-25-215" Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.753 [INFO][8247] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.760 [INFO][8247] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.763 [INFO][8247] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.767 [INFO][8247] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.767 [INFO][8247] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" host="172-234-25-215" Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.769 [INFO][8247] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6 Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.773 [INFO][8247] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" host="172-234-25-215" Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.781 [INFO][8247] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.38/26] block=192.168.62.0/26 handle="k8s-pod-network.706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" host="172-234-25-215" Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.815503 containerd[1567]: 2025-08-13 01:29:30.781 [INFO][8247] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.38/26] handle="k8s-pod-network.706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" host="172-234-25-215" Aug 13 01:29:30.815503 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.816694 containerd[1567]: 2025-08-13 01:29:30.781 [INFO][8247] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:30.816694 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.816694 containerd[1567]: 2025-08-13 01:29:30.781 [INFO][8247] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.38/26] IPv6=[] ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" HandleID="k8s-pod-network.706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:30.816694 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.816694 containerd[1567]: 2025-08-13 01:29:30.784 [INFO][8236] cni-plugin/k8s.go 418: Populated endpoint ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13754", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0be641dfd5b6d34f227a9b3e371aaddaa4d3681eb8759c48092f62768a054a9e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.38/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:30.816694 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.816694 containerd[1567]: 2025-08-13 01:29:30.784 [INFO][8236] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.38/32] ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:30.816694 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.816694 containerd[1567]: 2025-08-13 01:29:30.784 [INFO][8236] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:30.816694 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.816694 containerd[1567]: 2025-08-13 01:29:30.792 [INFO][8236] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:30.816694 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.816694 containerd[1567]: 2025-08-13 01:29:30.795 [INFO][8236] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13754", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.38/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"7e:d2:b1:85:65:3e", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:30.816694 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.817677 containerd[1567]: 2025-08-13 01:29:30.810 [INFO][8236] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:30.817677 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.884693 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:30.885802 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:30.992437 containerd[1567]: 2025-08-13 01:29:30.882 [INFO][8269] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Aug 13 01:29:30.992437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.992437 containerd[1567]: 2025-08-13 01:29:30.883 [INFO][8269] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" iface="eth0" netns="/var/run/netns/cni-6bb6587e-105a-5e6f-ac3f-2c6e2477ac1b" Aug 13 01:29:30.992437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.992437 containerd[1567]: 2025-08-13 01:29:30.883 [INFO][8269] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" iface="eth0" netns="/var/run/netns/cni-6bb6587e-105a-5e6f-ac3f-2c6e2477ac1b" Aug 13 01:29:30.992437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.992437 containerd[1567]: 2025-08-13 01:29:30.892 [INFO][8269] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" after=9.417441ms iface="eth0" netns="/var/run/netns/cni-6bb6587e-105a-5e6f-ac3f-2c6e2477ac1b" Aug 13 01:29:30.992437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.992437 containerd[1567]: 2025-08-13 01:29:30.892 [INFO][8269] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Aug 13 01:29:30.992437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.992437 containerd[1567]: 2025-08-13 01:29:30.893 [INFO][8269] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Aug 13 01:29:30.992437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.992437 containerd[1567]: 2025-08-13 01:29:30.924 [INFO][8279] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" HandleID="k8s-pod-network.706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:30.992437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.992437 containerd[1567]: 2025-08-13 01:29:30.926 [INFO][8279] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:30.992437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.992437 containerd[1567]: 2025-08-13 01:29:30.926 [INFO][8279] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:30.992437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.992437 containerd[1567]: 2025-08-13 01:29:30.983 [INFO][8279] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" HandleID="k8s-pod-network.706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:30.992437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.992437 containerd[1567]: 2025-08-13 01:29:30.983 [INFO][8279] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" HandleID="k8s-pod-network.706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:30.992437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.992437 containerd[1567]: 2025-08-13 01:29:30.987 [INFO][8279] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:30.992437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.992437 containerd[1567]: 2025-08-13 01:29:30.990 [INFO][8269] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6" Aug 13 01:29:30.992437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:30.995781 containerd[1567]: time="2025-08-13T01:29:30.995511663Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6-lo: no space left on device" Aug 13 01:29:30.997794 systemd[1]: run-netns-cni\x2d6bb6587e\x2d105a\x2d5e6f\x2dac3f\x2d2c6e2477ac1b.mount: Deactivated successfully. Aug 13 01:29:30.998094 kubelet[2720]: E0813 01:29:30.998052 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6-lo: no space left on device" Aug 13 01:29:30.998320 kubelet[2720]: E0813 01:29:30.998178 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:30.998320 kubelet[2720]: E0813 01:29:30.998226 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:30.998320 kubelet[2720]: E0813 01:29:30.998273 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:31.074060 kubelet[2720]: E0813 01:29:31.074023 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:29:31.678584 containerd[1567]: time="2025-08-13T01:29:31.678334006Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:31.803506 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:31.805258 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.717 [INFO][8289] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 13837 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.718 [INFO][8289] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.741 [INFO][8302] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" HandleID="k8s-pod-network.0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.741 [INFO][8302] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" HandleID="k8s-pod-network.0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f060), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:31.741224377 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.741 [INFO][8302] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.741 [INFO][8302] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.741 [INFO][8302] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.747 [INFO][8302] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" host="172-234-25-215" Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.754 [INFO][8302] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.760 [INFO][8302] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.761 [INFO][8302] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.764 [INFO][8302] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.764 [INFO][8302] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" host="172-234-25-215" Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.765 [INFO][8302] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1 Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.771 [INFO][8302] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" host="172-234-25-215" Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.789 [INFO][8302] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.39/26] block=192.168.62.0/26 handle="k8s-pod-network.0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" host="172-234-25-215" Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.838889 containerd[1567]: 2025-08-13 01:29:31.789 [INFO][8302] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.39/26] handle="k8s-pod-network.0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" host="172-234-25-215" Aug 13 01:29:31.838889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.839560 containerd[1567]: 2025-08-13 01:29:31.789 [INFO][8302] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:31.839560 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.839560 containerd[1567]: 2025-08-13 01:29:31.789 [INFO][8302] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.39/26] IPv6=[] ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" HandleID="k8s-pod-network.0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:31.839560 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.839560 containerd[1567]: 2025-08-13 01:29:31.793 [INFO][8289] cni-plugin/k8s.go 418: Populated endpoint ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13837", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"706623d5dad8f7c1e893cec7bc840b488adf062593b152668f008f5967cca5d6", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.39/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:31.839560 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.839560 containerd[1567]: 2025-08-13 01:29:31.794 [INFO][8289] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.39/32] ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:31.839560 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.839560 containerd[1567]: 2025-08-13 01:29:31.794 [INFO][8289] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:31.839560 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.839560 containerd[1567]: 2025-08-13 01:29:31.807 [INFO][8289] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:31.839560 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.839560 containerd[1567]: 2025-08-13 01:29:31.811 [INFO][8289] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13837", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.39/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"52:d6:3d:9d:9e:1b", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:31.839560 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.844122 containerd[1567]: 2025-08-13 01:29:31.828 [INFO][8289] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:31.844122 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:31.923912 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:31.924377 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:32.035557 containerd[1567]: 2025-08-13 01:29:31.921 [INFO][8324] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Aug 13 01:29:32.035557 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.035557 containerd[1567]: 2025-08-13 01:29:31.922 [INFO][8324] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" iface="eth0" netns="/var/run/netns/cni-a97d59c2-06e3-3f04-5169-b000429069fb" Aug 13 01:29:32.035557 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.035557 containerd[1567]: 2025-08-13 01:29:31.922 [INFO][8324] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" iface="eth0" netns="/var/run/netns/cni-a97d59c2-06e3-3f04-5169-b000429069fb" Aug 13 01:29:32.035557 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.035557 containerd[1567]: 2025-08-13 01:29:31.934 [INFO][8324] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" after=12.301806ms iface="eth0" netns="/var/run/netns/cni-a97d59c2-06e3-3f04-5169-b000429069fb" Aug 13 01:29:32.035557 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.035557 containerd[1567]: 2025-08-13 01:29:31.934 [INFO][8324] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Aug 13 01:29:32.035557 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.035557 containerd[1567]: 2025-08-13 01:29:31.934 [INFO][8324] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Aug 13 01:29:32.035557 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.035557 containerd[1567]: 2025-08-13 01:29:31.977 [INFO][8332] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" HandleID="k8s-pod-network.0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:32.035557 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.035557 containerd[1567]: 2025-08-13 01:29:31.978 [INFO][8332] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:32.035557 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.035557 containerd[1567]: 2025-08-13 01:29:31.978 [INFO][8332] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:32.035557 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.035557 containerd[1567]: 2025-08-13 01:29:32.024 [INFO][8332] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" HandleID="k8s-pod-network.0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:32.035557 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.035557 containerd[1567]: 2025-08-13 01:29:32.024 [INFO][8332] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" HandleID="k8s-pod-network.0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:32.035557 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.035557 containerd[1567]: 2025-08-13 01:29:32.028 [INFO][8332] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:32.035557 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.035557 containerd[1567]: 2025-08-13 01:29:32.030 [INFO][8324] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1" Aug 13 01:29:32.035557 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.040975 systemd[1]: run-netns-cni\x2da97d59c2\x2d06e3\x2d3f04\x2d5169\x2db000429069fb.mount: Deactivated successfully. Aug 13 01:29:32.042259 containerd[1567]: time="2025-08-13T01:29:32.042214016Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1-lo: no space left on device" Aug 13 01:29:32.042909 kubelet[2720]: E0813 01:29:32.042598 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1-lo: no space left on device" Aug 13 01:29:32.042909 kubelet[2720]: E0813 01:29:32.042648 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:32.042909 kubelet[2720]: E0813 01:29:32.042670 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:32.042909 kubelet[2720]: E0813 01:29:32.042747 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:32.085377 kubelet[2720]: E0813 01:29:32.084901 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:29:32.680321 containerd[1567]: time="2025-08-13T01:29:32.680024176Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:32.804987 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:32.805388 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.719 [INFO][8345] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 13912 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.719 [INFO][8345] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.757 [INFO][8357] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" HandleID="k8s-pod-network.f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.758 [INFO][8357] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" HandleID="k8s-pod-network.f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002deff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:32.75703451 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.758 [INFO][8357] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.758 [INFO][8357] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.758 [INFO][8357] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.766 [INFO][8357] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" host="172-234-25-215" Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.771 [INFO][8357] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.775 [INFO][8357] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.777 [INFO][8357] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.780 [INFO][8357] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.780 [INFO][8357] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" host="172-234-25-215" Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.781 [INFO][8357] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659 Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.789 [INFO][8357] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" host="172-234-25-215" Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.795 [INFO][8357] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.40/26] block=192.168.62.0/26 handle="k8s-pod-network.f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" host="172-234-25-215" Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.821674 containerd[1567]: 2025-08-13 01:29:32.795 [INFO][8357] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.40/26] handle="k8s-pod-network.f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" host="172-234-25-215" Aug 13 01:29:32.821674 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.823644 containerd[1567]: 2025-08-13 01:29:32.795 [INFO][8357] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:32.823644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.823644 containerd[1567]: 2025-08-13 01:29:32.795 [INFO][8357] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.40/26] IPv6=[] ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" HandleID="k8s-pod-network.f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:32.823644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.823644 containerd[1567]: 2025-08-13 01:29:32.800 [INFO][8345] cni-plugin/k8s.go 418: Populated endpoint ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13912", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0c71e5cc94bb96452227224b817e0f739ec197916fa516857a1598d80b0ed7b1", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.40/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:32.823644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.823644 containerd[1567]: 2025-08-13 01:29:32.801 [INFO][8345] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.40/32] ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:32.823644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.823644 containerd[1567]: 2025-08-13 01:29:32.801 [INFO][8345] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:32.823644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.823644 containerd[1567]: 2025-08-13 01:29:32.804 [INFO][8345] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:32.823644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.823644 containerd[1567]: 2025-08-13 01:29:32.804 [INFO][8345] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13912", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.40/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"8a:fc:88:83:a3:f8", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:32.823644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.824848 containerd[1567]: 2025-08-13 01:29:32.815 [INFO][8345] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:32.824848 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:32.898954 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:32.899738 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:33.015226 containerd[1567]: 2025-08-13 01:29:32.894 [INFO][8380] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Aug 13 01:29:33.015226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.015226 containerd[1567]: 2025-08-13 01:29:32.895 [INFO][8380] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" iface="eth0" netns="/var/run/netns/cni-514b872d-614b-9750-ced6-0b46db938d3e" Aug 13 01:29:33.015226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.015226 containerd[1567]: 2025-08-13 01:29:32.896 [INFO][8380] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" iface="eth0" netns="/var/run/netns/cni-514b872d-614b-9750-ced6-0b46db938d3e" Aug 13 01:29:33.015226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.015226 containerd[1567]: 2025-08-13 01:29:32.902 [INFO][8380] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" after=6.223689ms iface="eth0" netns="/var/run/netns/cni-514b872d-614b-9750-ced6-0b46db938d3e" Aug 13 01:29:33.015226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.015226 containerd[1567]: 2025-08-13 01:29:32.902 [INFO][8380] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Aug 13 01:29:33.015226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.015226 containerd[1567]: 2025-08-13 01:29:32.902 [INFO][8380] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Aug 13 01:29:33.015226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.015226 containerd[1567]: 2025-08-13 01:29:32.930 [INFO][8389] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" HandleID="k8s-pod-network.f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:33.015226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.015226 containerd[1567]: 2025-08-13 01:29:32.931 [INFO][8389] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:33.015226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.015226 containerd[1567]: 2025-08-13 01:29:32.931 [INFO][8389] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:33.015226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.015226 containerd[1567]: 2025-08-13 01:29:33.007 [INFO][8389] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" HandleID="k8s-pod-network.f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:33.015226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.015226 containerd[1567]: 2025-08-13 01:29:33.007 [INFO][8389] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" HandleID="k8s-pod-network.f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:33.015226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.015226 containerd[1567]: 2025-08-13 01:29:33.008 [INFO][8389] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:33.015226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.015226 containerd[1567]: 2025-08-13 01:29:33.011 [INFO][8380] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659" Aug 13 01:29:33.015226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.020659 systemd[1]: run-netns-cni\x2d514b872d\x2d614b\x2d9750\x2dced6\x2d0b46db938d3e.mount: Deactivated successfully. Aug 13 01:29:33.021308 containerd[1567]: time="2025-08-13T01:29:33.021200159Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659-lo: no space left on device" Aug 13 01:29:33.022140 kubelet[2720]: E0813 01:29:33.021609 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659-lo: no space left on device" Aug 13 01:29:33.022824 kubelet[2720]: E0813 01:29:33.022799 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:33.022899 kubelet[2720]: E0813 01:29:33.022885 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:33.023520 kubelet[2720]: E0813 01:29:33.023480 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:33.356086 containerd[1567]: time="2025-08-13T01:29:33.355432335Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:33.682065 containerd[1567]: time="2025-08-13T01:29:33.681937256Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:33.797600 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:33.798398 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.725 [INFO][8400] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 13987 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.726 [INFO][8400] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.756 [INFO][8412] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" HandleID="k8s-pod-network.815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.756 [INFO][8412] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" HandleID="k8s-pod-network.815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:33.756009026 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.756 [INFO][8412] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.757 [INFO][8412] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.757 [INFO][8412] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.764 [INFO][8412] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" host="172-234-25-215" Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.769 [INFO][8412] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.774 [INFO][8412] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.777 [INFO][8412] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.779 [INFO][8412] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.779 [INFO][8412] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" host="172-234-25-215" Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.781 [INFO][8412] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.784 [INFO][8412] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" host="172-234-25-215" Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.790 [INFO][8412] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.41/26] block=192.168.62.0/26 handle="k8s-pod-network.815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" host="172-234-25-215" Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.826993 containerd[1567]: 2025-08-13 01:29:33.790 [INFO][8412] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.41/26] handle="k8s-pod-network.815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" host="172-234-25-215" Aug 13 01:29:33.826993 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.830736 containerd[1567]: 2025-08-13 01:29:33.790 [INFO][8412] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:33.830736 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.830736 containerd[1567]: 2025-08-13 01:29:33.790 [INFO][8412] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.41/26] IPv6=[] ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" HandleID="k8s-pod-network.815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:33.830736 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.830736 containerd[1567]: 2025-08-13 01:29:33.793 [INFO][8400] cni-plugin/k8s.go 418: Populated endpoint ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13987", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"f14ae3c54bc7fdb2fd031bb2cbbb2ae7ff907aa72abe48dec6da9698f7201659", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.41/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:33.830736 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.830736 containerd[1567]: 2025-08-13 01:29:33.793 [INFO][8400] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.41/32] ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:33.830736 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.830736 containerd[1567]: 2025-08-13 01:29:33.793 [INFO][8400] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:33.830736 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.830736 containerd[1567]: 2025-08-13 01:29:33.798 [INFO][8400] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:33.830736 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.830736 containerd[1567]: 2025-08-13 01:29:33.804 [INFO][8400] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"13987", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.41/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"4a:61:67:69:dd:6e", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:33.830736 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.831028 containerd[1567]: 2025-08-13 01:29:33.819 [INFO][8400] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:33.831028 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.896783 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:33.896793 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:33.989342 containerd[1567]: 2025-08-13 01:29:33.892 [INFO][8434] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Aug 13 01:29:33.989342 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.989342 containerd[1567]: 2025-08-13 01:29:33.894 [INFO][8434] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" iface="eth0" netns="/var/run/netns/cni-13d4ee2b-4286-dfa7-2245-dc4523c6ab52" Aug 13 01:29:33.989342 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.989342 containerd[1567]: 2025-08-13 01:29:33.894 [INFO][8434] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" iface="eth0" netns="/var/run/netns/cni-13d4ee2b-4286-dfa7-2245-dc4523c6ab52" Aug 13 01:29:33.989342 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.989342 containerd[1567]: 2025-08-13 01:29:33.904 [INFO][8434] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" after=10.848079ms iface="eth0" netns="/var/run/netns/cni-13d4ee2b-4286-dfa7-2245-dc4523c6ab52" Aug 13 01:29:33.989342 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.989342 containerd[1567]: 2025-08-13 01:29:33.905 [INFO][8434] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Aug 13 01:29:33.989342 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.989342 containerd[1567]: 2025-08-13 01:29:33.905 [INFO][8434] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Aug 13 01:29:33.989342 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.989342 containerd[1567]: 2025-08-13 01:29:33.944 [INFO][8444] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" HandleID="k8s-pod-network.815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:33.989342 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.989342 containerd[1567]: 2025-08-13 01:29:33.945 [INFO][8444] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:33.989342 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.989342 containerd[1567]: 2025-08-13 01:29:33.945 [INFO][8444] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:33.989342 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.989342 containerd[1567]: 2025-08-13 01:29:33.983 [INFO][8444] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" HandleID="k8s-pod-network.815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:33.989342 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.989342 containerd[1567]: 2025-08-13 01:29:33.983 [INFO][8444] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" HandleID="k8s-pod-network.815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:33.989342 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.989342 containerd[1567]: 2025-08-13 01:29:33.984 [INFO][8444] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:33.989342 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.989342 containerd[1567]: 2025-08-13 01:29:33.986 [INFO][8434] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd" Aug 13 01:29:33.989342 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:33.991824 containerd[1567]: time="2025-08-13T01:29:33.991580720Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd-lo: no space left on device" Aug 13 01:29:33.993591 systemd[1]: run-netns-cni\x2d13d4ee2b\x2d4286\x2ddfa7\x2d2245\x2ddc4523c6ab52.mount: Deactivated successfully. Aug 13 01:29:33.994789 kubelet[2720]: E0813 01:29:33.994054 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd-lo: no space left on device" Aug 13 01:29:33.994789 kubelet[2720]: E0813 01:29:33.994118 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:33.994789 kubelet[2720]: E0813 01:29:33.994140 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:33.994789 kubelet[2720]: E0813 01:29:33.994199 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:34.683224 containerd[1567]: time="2025-08-13T01:29:34.683139590Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:34.798369 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:34.802032 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.729 [INFO][8454] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 14065 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.729 [INFO][8454] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.758 [INFO][8468] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" HandleID="k8s-pod-network.14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.758 [INFO][8468] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" HandleID="k8s-pod-network.14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5890), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:34.757997849 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.758 [INFO][8468] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.758 [INFO][8468] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.758 [INFO][8468] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.765 [INFO][8468] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" host="172-234-25-215" Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.769 [INFO][8468] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.774 [INFO][8468] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.777 [INFO][8468] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.780 [INFO][8468] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.780 [INFO][8468] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" host="172-234-25-215" Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.782 [INFO][8468] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88 Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.785 [INFO][8468] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" host="172-234-25-215" Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.791 [INFO][8468] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.42/26] block=192.168.62.0/26 handle="k8s-pod-network.14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" host="172-234-25-215" Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.825859 containerd[1567]: 2025-08-13 01:29:34.791 [INFO][8468] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.42/26] handle="k8s-pod-network.14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" host="172-234-25-215" Aug 13 01:29:34.825859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.826542 containerd[1567]: 2025-08-13 01:29:34.791 [INFO][8468] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:34.826542 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.826542 containerd[1567]: 2025-08-13 01:29:34.791 [INFO][8468] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.42/26] IPv6=[] ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" HandleID="k8s-pod-network.14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:34.826542 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.826542 containerd[1567]: 2025-08-13 01:29:34.794 [INFO][8454] cni-plugin/k8s.go 418: Populated endpoint ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14065", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"815954528303aea143e2893a7b98db2e850d1ec277d5f29f6443ddf85206fbfd", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.42/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:34.826542 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.826542 containerd[1567]: 2025-08-13 01:29:34.794 [INFO][8454] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.42/32] ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:34.826542 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.826542 containerd[1567]: 2025-08-13 01:29:34.794 [INFO][8454] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:34.826542 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.826542 containerd[1567]: 2025-08-13 01:29:34.806 [INFO][8454] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:34.826542 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.826542 containerd[1567]: 2025-08-13 01:29:34.806 [INFO][8454] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14065", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.42/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"92:86:a6:8f:30:61", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:34.826542 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.826863 containerd[1567]: 2025-08-13 01:29:34.819 [INFO][8454] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:34.826863 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:34.892022 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:34.892031 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:35.001745 containerd[1567]: 2025-08-13 01:29:34.886 [INFO][8493] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Aug 13 01:29:35.001745 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.001745 containerd[1567]: 2025-08-13 01:29:34.887 [INFO][8493] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" iface="eth0" netns="/var/run/netns/cni-42ab7001-b21d-1afc-4d71-b339d96accf6" Aug 13 01:29:35.001745 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.001745 containerd[1567]: 2025-08-13 01:29:34.888 [INFO][8493] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" iface="eth0" netns="/var/run/netns/cni-42ab7001-b21d-1afc-4d71-b339d96accf6" Aug 13 01:29:35.001745 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.001745 containerd[1567]: 2025-08-13 01:29:34.898 [INFO][8493] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" after=10.911449ms iface="eth0" netns="/var/run/netns/cni-42ab7001-b21d-1afc-4d71-b339d96accf6" Aug 13 01:29:35.001745 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.001745 containerd[1567]: 2025-08-13 01:29:34.898 [INFO][8493] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Aug 13 01:29:35.001745 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.001745 containerd[1567]: 2025-08-13 01:29:34.899 [INFO][8493] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Aug 13 01:29:35.001745 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.001745 containerd[1567]: 2025-08-13 01:29:34.945 [INFO][8502] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" HandleID="k8s-pod-network.14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:35.001745 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.001745 containerd[1567]: 2025-08-13 01:29:34.946 [INFO][8502] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:35.001745 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.001745 containerd[1567]: 2025-08-13 01:29:34.946 [INFO][8502] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:35.001745 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.001745 containerd[1567]: 2025-08-13 01:29:34.993 [INFO][8502] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" HandleID="k8s-pod-network.14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:35.001745 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.001745 containerd[1567]: 2025-08-13 01:29:34.993 [INFO][8502] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" HandleID="k8s-pod-network.14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:35.001745 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.001745 containerd[1567]: 2025-08-13 01:29:34.995 [INFO][8502] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:35.001745 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.001745 containerd[1567]: 2025-08-13 01:29:34.998 [INFO][8493] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88" Aug 13 01:29:35.001745 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.006754 containerd[1567]: time="2025-08-13T01:29:35.005846690Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88-lo: no space left on device" Aug 13 01:29:35.008140 kubelet[2720]: E0813 01:29:35.007360 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88-lo: no space left on device" Aug 13 01:29:35.008140 kubelet[2720]: E0813 01:29:35.007416 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:35.008140 kubelet[2720]: E0813 01:29:35.007435 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:35.008140 kubelet[2720]: E0813 01:29:35.007488 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:35.007863 systemd[1]: run-netns-cni\x2d42ab7001\x2db21d\x2d1afc\x2d4d71\x2db339d96accf6.mount: Deactivated successfully. Aug 13 01:29:35.526852 systemd[1]: Started sshd@32-172.234.25.215:22-147.75.109.163:50408.service - OpenSSH per-connection server daemon (147.75.109.163:50408). Aug 13 01:29:35.685099 containerd[1567]: time="2025-08-13T01:29:35.684817815Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:35.790586 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:35.790841 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.724 [INFO][8514] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 14132 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.724 [INFO][8514] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.746 [INFO][8527] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" HandleID="k8s-pod-network.77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.746 [INFO][8527] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" HandleID="k8s-pod-network.77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc000259610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:35.746420609 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.746 [INFO][8527] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.746 [INFO][8527] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.746 [INFO][8527] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.754 [INFO][8527] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" host="172-234-25-215" Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.759 [INFO][8527] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.764 [INFO][8527] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.766 [INFO][8527] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.769 [INFO][8527] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.770 [INFO][8527] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" host="172-234-25-215" Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.772 [INFO][8527] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9 Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.777 [INFO][8527] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" host="172-234-25-215" Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.783 [INFO][8527] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.43/26] block=192.168.62.0/26 handle="k8s-pod-network.77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" host="172-234-25-215" Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.809392 containerd[1567]: 2025-08-13 01:29:35.783 [INFO][8527] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.43/26] handle="k8s-pod-network.77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" host="172-234-25-215" Aug 13 01:29:35.809392 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.810264 containerd[1567]: 2025-08-13 01:29:35.783 [INFO][8527] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:35.810264 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.810264 containerd[1567]: 2025-08-13 01:29:35.783 [INFO][8527] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.43/26] IPv6=[] ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" HandleID="k8s-pod-network.77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:35.810264 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.810264 containerd[1567]: 2025-08-13 01:29:35.785 [INFO][8514] cni-plugin/k8s.go 418: Populated endpoint ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14132", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"14be257e481e316d17d4e2cca110a7dfd4b01228a06f2acd34b150f1ecd07a88", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.43/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:35.810264 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.810264 containerd[1567]: 2025-08-13 01:29:35.786 [INFO][8514] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.43/32] ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:35.810264 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.810264 containerd[1567]: 2025-08-13 01:29:35.786 [INFO][8514] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:35.810264 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.810264 containerd[1567]: 2025-08-13 01:29:35.791 [INFO][8514] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:35.810264 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.810264 containerd[1567]: 2025-08-13 01:29:35.792 [INFO][8514] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14132", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.43/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"56:77:48:06:cf:b4", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:35.810264 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.813811 containerd[1567]: 2025-08-13 01:29:35.802 [INFO][8514] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:35.813811 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.879579 sshd[8511]: Accepted publickey for core from 147.75.109.163 port 50408 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:29:35.881272 sshd-session[8511]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:29:35.889117 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:35.889126 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:35.893774 systemd-logind[1543]: New session 33 of user core. Aug 13 01:29:35.898164 systemd[1]: Started session-33.scope - Session 33 of User core. Aug 13 01:29:35.996035 containerd[1567]: 2025-08-13 01:29:35.886 [INFO][8550] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Aug 13 01:29:35.996035 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.996035 containerd[1567]: 2025-08-13 01:29:35.887 [INFO][8550] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" iface="eth0" netns="/var/run/netns/cni-dbe64408-dbc9-4f6b-f8c5-8e6d60739ac4" Aug 13 01:29:35.996035 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.996035 containerd[1567]: 2025-08-13 01:29:35.888 [INFO][8550] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" iface="eth0" netns="/var/run/netns/cni-dbe64408-dbc9-4f6b-f8c5-8e6d60739ac4" Aug 13 01:29:35.996035 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.996035 containerd[1567]: 2025-08-13 01:29:35.897 [INFO][8550] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" after=8.894834ms iface="eth0" netns="/var/run/netns/cni-dbe64408-dbc9-4f6b-f8c5-8e6d60739ac4" Aug 13 01:29:35.996035 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.996035 containerd[1567]: 2025-08-13 01:29:35.897 [INFO][8550] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Aug 13 01:29:35.996035 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.996035 containerd[1567]: 2025-08-13 01:29:35.897 [INFO][8550] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Aug 13 01:29:35.996035 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.996035 containerd[1567]: 2025-08-13 01:29:35.957 [INFO][8560] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" HandleID="k8s-pod-network.77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:35.996035 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.996035 containerd[1567]: 2025-08-13 01:29:35.957 [INFO][8560] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:35.996035 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.996035 containerd[1567]: 2025-08-13 01:29:35.958 [INFO][8560] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:35.996035 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.996035 containerd[1567]: 2025-08-13 01:29:35.989 [INFO][8560] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" HandleID="k8s-pod-network.77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:35.996035 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.996035 containerd[1567]: 2025-08-13 01:29:35.989 [INFO][8560] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" HandleID="k8s-pod-network.77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:35.996035 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.996035 containerd[1567]: 2025-08-13 01:29:35.990 [INFO][8560] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:35.996035 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:35.996035 containerd[1567]: 2025-08-13 01:29:35.993 [INFO][8550] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9" Aug 13 01:29:35.996035 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.000482 containerd[1567]: time="2025-08-13T01:29:36.000380747Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9-lo: no space left on device" Aug 13 01:29:36.001256 kubelet[2720]: E0813 01:29:36.000816 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9-lo: no space left on device" Aug 13 01:29:36.001256 kubelet[2720]: E0813 01:29:36.000878 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:36.001256 kubelet[2720]: E0813 01:29:36.000898 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:36.001256 kubelet[2720]: E0813 01:29:36.000948 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:36.000920 systemd[1]: run-netns-cni\x2ddbe64408\x2ddbc9\x2d4f6b\x2df8c5\x2d8e6d60739ac4.mount: Deactivated successfully. Aug 13 01:29:36.191409 sshd[8565]: Connection closed by 147.75.109.163 port 50408 Aug 13 01:29:36.192941 sshd-session[8511]: pam_unix(sshd:session): session closed for user core Aug 13 01:29:36.197555 systemd[1]: sshd@32-172.234.25.215:22-147.75.109.163:50408.service: Deactivated successfully. Aug 13 01:29:36.202267 systemd[1]: session-33.scope: Deactivated successfully. Aug 13 01:29:36.203972 systemd-logind[1543]: Session 33 logged out. Waiting for processes to exit. Aug 13 01:29:36.206028 systemd-logind[1543]: Removed session 33. Aug 13 01:29:36.355633 containerd[1567]: time="2025-08-13T01:29:36.355558557Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:36.687780 containerd[1567]: time="2025-08-13T01:29:36.687626389Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:36.810980 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:36.811831 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.735 [INFO][8581] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 14207 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.736 [INFO][8581] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.767 [INFO][8594] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" HandleID="k8s-pod-network.c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.768 [INFO][8594] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" HandleID="k8s-pod-network.c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cd100), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:36.767846727 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.768 [INFO][8594] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.768 [INFO][8594] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.768 [INFO][8594] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.774 [INFO][8594] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" host="172-234-25-215" Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.778 [INFO][8594] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.782 [INFO][8594] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.784 [INFO][8594] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.786 [INFO][8594] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.786 [INFO][8594] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" host="172-234-25-215" Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.788 [INFO][8594] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7 Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.792 [INFO][8594] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" host="172-234-25-215" Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.797 [INFO][8594] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.44/26] block=192.168.62.0/26 handle="k8s-pod-network.c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" host="172-234-25-215" Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828030 containerd[1567]: 2025-08-13 01:29:36.797 [INFO][8594] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.44/26] handle="k8s-pod-network.c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" host="172-234-25-215" Aug 13 01:29:36.828030 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828917 containerd[1567]: 2025-08-13 01:29:36.797 [INFO][8594] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:36.828917 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828917 containerd[1567]: 2025-08-13 01:29:36.797 [INFO][8594] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.44/26] IPv6=[] ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" HandleID="k8s-pod-network.c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:36.828917 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828917 containerd[1567]: 2025-08-13 01:29:36.801 [INFO][8581] cni-plugin/k8s.go 418: Populated endpoint ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14207", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"77704fc8169cc06a61f4d58aa4fada6a63df1e91cc3188b47190dc8dbf7ed7f9", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.44/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:36.828917 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828917 containerd[1567]: 2025-08-13 01:29:36.801 [INFO][8581] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.44/32] ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:36.828917 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828917 containerd[1567]: 2025-08-13 01:29:36.801 [INFO][8581] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:36.828917 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828917 containerd[1567]: 2025-08-13 01:29:36.812 [INFO][8581] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:36.828917 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.828917 containerd[1567]: 2025-08-13 01:29:36.813 [INFO][8581] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14207", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.44/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"9e:3a:d3:9d:da:7d", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:36.828917 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.829232 containerd[1567]: 2025-08-13 01:29:36.824 [INFO][8581] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:36.829232 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.893089 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:36.893200 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:36.989216 containerd[1567]: 2025-08-13 01:29:36.891 [INFO][8616] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Aug 13 01:29:36.989216 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.989216 containerd[1567]: 2025-08-13 01:29:36.891 [INFO][8616] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" iface="eth0" netns="/var/run/netns/cni-0fa18ee8-98f8-4905-d570-72d61794ebbd" Aug 13 01:29:36.989216 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.989216 containerd[1567]: 2025-08-13 01:29:36.892 [INFO][8616] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" iface="eth0" netns="/var/run/netns/cni-0fa18ee8-98f8-4905-d570-72d61794ebbd" Aug 13 01:29:36.989216 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.989216 containerd[1567]: 2025-08-13 01:29:36.901 [INFO][8616] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" after=10.001501ms iface="eth0" netns="/var/run/netns/cni-0fa18ee8-98f8-4905-d570-72d61794ebbd" Aug 13 01:29:36.989216 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.989216 containerd[1567]: 2025-08-13 01:29:36.902 [INFO][8616] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Aug 13 01:29:36.989216 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.989216 containerd[1567]: 2025-08-13 01:29:36.902 [INFO][8616] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Aug 13 01:29:36.989216 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.989216 containerd[1567]: 2025-08-13 01:29:36.932 [INFO][8624] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" HandleID="k8s-pod-network.c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:36.989216 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.989216 containerd[1567]: 2025-08-13 01:29:36.934 [INFO][8624] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:36.989216 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.989216 containerd[1567]: 2025-08-13 01:29:36.934 [INFO][8624] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:36.989216 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.989216 containerd[1567]: 2025-08-13 01:29:36.980 [INFO][8624] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" HandleID="k8s-pod-network.c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:36.989216 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.989216 containerd[1567]: 2025-08-13 01:29:36.980 [INFO][8624] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" HandleID="k8s-pod-network.c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:36.989216 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.989216 containerd[1567]: 2025-08-13 01:29:36.982 [INFO][8624] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:36.989216 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.989216 containerd[1567]: 2025-08-13 01:29:36.985 [INFO][8616] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7" Aug 13 01:29:36.989216 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:36.994293 systemd[1]: run-netns-cni\x2d0fa18ee8\x2d98f8\x2d4905\x2dd570\x2d72d61794ebbd.mount: Deactivated successfully. Aug 13 01:29:36.996301 containerd[1567]: time="2025-08-13T01:29:36.995400828Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7-lo: no space left on device" Aug 13 01:29:36.996379 kubelet[2720]: E0813 01:29:36.995615 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7-lo: no space left on device" Aug 13 01:29:36.996379 kubelet[2720]: E0813 01:29:36.995670 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:36.996379 kubelet[2720]: E0813 01:29:36.995689 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:36.996379 kubelet[2720]: E0813 01:29:36.995795 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:37.073946 kubelet[2720]: E0813 01:29:37.073917 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:29:37.076362 containerd[1567]: time="2025-08-13T01:29:37.076335844Z" level=info msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\"" Aug 13 01:29:37.309008 containerd[1567]: time="2025-08-13T01:29:37.308788325Z" level=error msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\" failed" error="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-307916557-_eRC sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-3587324838: no space left on device" Aug 13 01:29:37.309008 containerd[1567]: time="2025-08-13T01:29:37.308862725Z" level=info msg="stop pulling image registry.k8s.io/coredns/coredns:v1.12.0: active requests=0, bytes read=0" Aug 13 01:29:37.309389 kubelet[2720]: E0813 01:29:37.309004 2720 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-307916557-_eRC sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-3587324838: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:29:37.309389 kubelet[2720]: E0813 01:29:37.309048 2720 kuberuntime_image.go:42] "Failed to pull image" err="failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-307916557-_eRC sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-3587324838: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:29:37.309389 kubelet[2720]: E0813 01:29:37.309200 2720 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:coredns,Image:registry.k8s.io/coredns/coredns:v1.12.0,Command:[],Args:[-conf /etc/coredns/Corefile],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:dns,HostPort:0,ContainerPort:53,Protocol:UDP,HostIP:,},ContainerPort{Name:dns-tcp,HostPort:0,ContainerPort:53,Protocol:TCP,HostIP:,},ContainerPort{Name:metrics,HostPort:0,ContainerPort:9153,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{memory: {{178257920 0} {} 170Mi BinarySI},},Requests:ResourceList{cpu: {{100 -3} {} 100m DecimalSI},memory: {{73400320 0} {} 70Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:config-volume,ReadOnly:true,MountPath:/etc/coredns,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-lncr6,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{0 8080 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:60,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:5,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/ready,Port:{0 8181 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[NET_BIND_SERVICE],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b): ErrImagePull: failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-307916557-_eRC sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-3587324838: no space left on device" logger="UnhandledError" Aug 13 01:29:37.310443 kubelet[2720]: E0813 01:29:37.310393 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ErrImagePull: \"failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-307916557-_eRC sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-3587324838: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:29:37.689408 containerd[1567]: time="2025-08-13T01:29:37.689263577Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:37.792482 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:37.792706 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.724 [INFO][8634] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 14277 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.724 [INFO][8634] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.748 [INFO][8647] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" HandleID="k8s-pod-network.51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.749 [INFO][8647] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" HandleID="k8s-pod-network.51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f210), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:37.748155966 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.749 [INFO][8647] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.750 [INFO][8647] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.750 [INFO][8647] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.756 [INFO][8647] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" host="172-234-25-215" Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.762 [INFO][8647] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.767 [INFO][8647] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.768 [INFO][8647] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.771 [INFO][8647] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.771 [INFO][8647] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" host="172-234-25-215" Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.772 [INFO][8647] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.777 [INFO][8647] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" host="172-234-25-215" Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.786 [INFO][8647] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.45/26] block=192.168.62.0/26 handle="k8s-pod-network.51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" host="172-234-25-215" Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.815786 containerd[1567]: 2025-08-13 01:29:37.786 [INFO][8647] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.45/26] handle="k8s-pod-network.51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" host="172-234-25-215" Aug 13 01:29:37.815786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.817309 containerd[1567]: 2025-08-13 01:29:37.786 [INFO][8647] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:37.817309 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.817309 containerd[1567]: 2025-08-13 01:29:37.786 [INFO][8647] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.45/26] IPv6=[] ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" HandleID="k8s-pod-network.51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:37.817309 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.817309 containerd[1567]: 2025-08-13 01:29:37.788 [INFO][8634] cni-plugin/k8s.go 418: Populated endpoint ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14277", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c4fe7dbf1d1f61a2f35653562c071dafa2b53c7f67b207765b20f7ce7a2b51b7", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.45/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:37.817309 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.817309 containerd[1567]: 2025-08-13 01:29:37.789 [INFO][8634] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.45/32] ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:37.817309 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.817309 containerd[1567]: 2025-08-13 01:29:37.789 [INFO][8634] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:37.817309 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.817309 containerd[1567]: 2025-08-13 01:29:37.794 [INFO][8634] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:37.817309 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.817309 containerd[1567]: 2025-08-13 01:29:37.794 [INFO][8634] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14277", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.45/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"12:2b:2f:4e:9e:59", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:37.817309 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.817594 containerd[1567]: 2025-08-13 01:29:37.807 [INFO][8634] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:37.817594 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.882989 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:37.882998 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:37.968074 containerd[1567]: 2025-08-13 01:29:37.880 [INFO][8671] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Aug 13 01:29:37.968074 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.968074 containerd[1567]: 2025-08-13 01:29:37.881 [INFO][8671] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" iface="eth0" netns="/var/run/netns/cni-b953ca15-1516-d70e-507f-c2ce3cbda314" Aug 13 01:29:37.968074 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.968074 containerd[1567]: 2025-08-13 01:29:37.881 [INFO][8671] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" iface="eth0" netns="/var/run/netns/cni-b953ca15-1516-d70e-507f-c2ce3cbda314" Aug 13 01:29:37.968074 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.968074 containerd[1567]: 2025-08-13 01:29:37.891 [INFO][8671] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" after=10.37478ms iface="eth0" netns="/var/run/netns/cni-b953ca15-1516-d70e-507f-c2ce3cbda314" Aug 13 01:29:37.968074 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.968074 containerd[1567]: 2025-08-13 01:29:37.891 [INFO][8671] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Aug 13 01:29:37.968074 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.968074 containerd[1567]: 2025-08-13 01:29:37.891 [INFO][8671] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Aug 13 01:29:37.968074 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.968074 containerd[1567]: 2025-08-13 01:29:37.927 [INFO][8681] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" HandleID="k8s-pod-network.51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:37.968074 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.968074 containerd[1567]: 2025-08-13 01:29:37.927 [INFO][8681] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:37.968074 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.968074 containerd[1567]: 2025-08-13 01:29:37.927 [INFO][8681] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:37.968074 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.968074 containerd[1567]: 2025-08-13 01:29:37.960 [INFO][8681] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" HandleID="k8s-pod-network.51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:37.968074 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.968074 containerd[1567]: 2025-08-13 01:29:37.961 [INFO][8681] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" HandleID="k8s-pod-network.51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:37.968074 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.968074 containerd[1567]: 2025-08-13 01:29:37.963 [INFO][8681] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:37.968074 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.968074 containerd[1567]: 2025-08-13 01:29:37.965 [INFO][8671] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f" Aug 13 01:29:37.968074 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:37.973108 systemd[1]: run-netns-cni\x2db953ca15\x2d1516\x2dd70e\x2d507f\x2dc2ce3cbda314.mount: Deactivated successfully. Aug 13 01:29:37.974674 containerd[1567]: time="2025-08-13T01:29:37.973693610Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f-lo: no space left on device" Aug 13 01:29:37.975215 kubelet[2720]: E0813 01:29:37.974906 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f-lo: no space left on device" Aug 13 01:29:37.975215 kubelet[2720]: E0813 01:29:37.974968 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:37.975215 kubelet[2720]: E0813 01:29:37.974990 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:37.975215 kubelet[2720]: E0813 01:29:37.975045 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:38.177324 kubelet[2720]: I0813 01:29:38.177272 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:29:38.177324 kubelet[2720]: I0813 01:29:38.177315 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:29:38.179876 kubelet[2720]: I0813 01:29:38.179858 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:29:38.195732 kubelet[2720]: I0813 01:29:38.195540 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:29:38.195732 kubelet[2720]: I0813 01:29:38.195669 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:29:38.195732 kubelet[2720]: E0813 01:29:38.195699 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:38.195869 kubelet[2720]: E0813 01:29:38.195708 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:29:38.195969 kubelet[2720]: E0813 01:29:38.195958 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:29:38.196017 kubelet[2720]: E0813 01:29:38.196007 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:29:38.196152 kubelet[2720]: E0813 01:29:38.196075 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:29:38.196152 kubelet[2720]: E0813 01:29:38.196086 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:29:38.196152 kubelet[2720]: E0813 01:29:38.196093 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:29:38.196152 kubelet[2720]: E0813 01:29:38.196103 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:29:38.196152 kubelet[2720]: E0813 01:29:38.196110 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:29:38.196152 kubelet[2720]: E0813 01:29:38.196117 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:29:38.196339 kubelet[2720]: I0813 01:29:38.196232 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:29:38.280229 containerd[1567]: time="2025-08-13T01:29:38.280148612Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:29:38.690482 containerd[1567]: time="2025-08-13T01:29:38.690393038Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:38.781533 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:38.782833 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.720 [INFO][8689] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 14339 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.720 [INFO][8689] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.744 [INFO][8702] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" HandleID="k8s-pod-network.c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.744 [INFO][8702] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" HandleID="k8s-pod-network.c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5640), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:38.744050396 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.744 [INFO][8702] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.744 [INFO][8702] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.744 [INFO][8702] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.751 [INFO][8702] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" host="172-234-25-215" Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.755 [INFO][8702] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.760 [INFO][8702] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.762 [INFO][8702] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.764 [INFO][8702] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.764 [INFO][8702] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" host="172-234-25-215" Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.766 [INFO][8702] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398 Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.769 [INFO][8702] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" host="172-234-25-215" Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.774 [INFO][8702] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.46/26] block=192.168.62.0/26 handle="k8s-pod-network.c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" host="172-234-25-215" Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.804686 containerd[1567]: 2025-08-13 01:29:38.774 [INFO][8702] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.46/26] handle="k8s-pod-network.c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" host="172-234-25-215" Aug 13 01:29:38.804686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.805995 containerd[1567]: 2025-08-13 01:29:38.775 [INFO][8702] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:38.805995 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.805995 containerd[1567]: 2025-08-13 01:29:38.775 [INFO][8702] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.46/26] IPv6=[] ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" HandleID="k8s-pod-network.c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:38.805995 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.805995 containerd[1567]: 2025-08-13 01:29:38.778 [INFO][8689] cni-plugin/k8s.go 418: Populated endpoint ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14339", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"51b1d8a4f198135cd1a30e704487b1c4118f87f0b89ee0deb8ab654e330ca16f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.46/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:38.805995 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.805995 containerd[1567]: 2025-08-13 01:29:38.778 [INFO][8689] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.46/32] ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:38.805995 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.805995 containerd[1567]: 2025-08-13 01:29:38.778 [INFO][8689] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:38.805995 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.805995 containerd[1567]: 2025-08-13 01:29:38.781 [INFO][8689] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:38.805995 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.805995 containerd[1567]: 2025-08-13 01:29:38.783 [INFO][8689] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14339", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.46/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"62:72:ad:75:67:22", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:38.805995 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.806284 containerd[1567]: 2025-08-13 01:29:38.795 [INFO][8689] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:38.806284 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:38.921901 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:38.921909 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:39.046663 containerd[1567]: 2025-08-13 01:29:38.911 [INFO][8727] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Aug 13 01:29:39.046663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.046663 containerd[1567]: 2025-08-13 01:29:38.913 [INFO][8727] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" iface="eth0" netns="/var/run/netns/cni-0fa2b166-9778-fc13-b879-8b7b311cb1e7" Aug 13 01:29:39.046663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.046663 containerd[1567]: 2025-08-13 01:29:38.919 [INFO][8727] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" iface="eth0" netns="/var/run/netns/cni-0fa2b166-9778-fc13-b879-8b7b311cb1e7" Aug 13 01:29:39.046663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.046663 containerd[1567]: 2025-08-13 01:29:38.928 [INFO][8727] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" after=10.227471ms iface="eth0" netns="/var/run/netns/cni-0fa2b166-9778-fc13-b879-8b7b311cb1e7" Aug 13 01:29:39.046663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.046663 containerd[1567]: 2025-08-13 01:29:38.929 [INFO][8727] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Aug 13 01:29:39.046663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.046663 containerd[1567]: 2025-08-13 01:29:38.929 [INFO][8727] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Aug 13 01:29:39.046663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.046663 containerd[1567]: 2025-08-13 01:29:38.994 [INFO][8735] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" HandleID="k8s-pod-network.c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:39.046663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.046663 containerd[1567]: 2025-08-13 01:29:38.994 [INFO][8735] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:39.046663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.046663 containerd[1567]: 2025-08-13 01:29:38.995 [INFO][8735] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:39.046663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.046663 containerd[1567]: 2025-08-13 01:29:39.039 [INFO][8735] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" HandleID="k8s-pod-network.c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:39.046663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.046663 containerd[1567]: 2025-08-13 01:29:39.039 [INFO][8735] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" HandleID="k8s-pod-network.c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:39.046663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.046663 containerd[1567]: 2025-08-13 01:29:39.040 [INFO][8735] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:39.046663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.046663 containerd[1567]: 2025-08-13 01:29:39.043 [INFO][8727] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398" Aug 13 01:29:39.046663 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.051025 containerd[1567]: time="2025-08-13T01:29:39.050157409Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398-lo: no space left on device" Aug 13 01:29:39.051093 kubelet[2720]: E0813 01:29:39.050478 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398-lo: no space left on device" Aug 13 01:29:39.051093 kubelet[2720]: E0813 01:29:39.050550 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:39.051093 kubelet[2720]: E0813 01:29:39.050575 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:39.051093 kubelet[2720]: E0813 01:29:39.050645 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:39.049045 systemd[1]: run-netns-cni\x2d0fa2b166\x2d9778\x2dfc13\x2db879\x2d8b7b311cb1e7.mount: Deactivated successfully. Aug 13 01:29:39.075958 kubelet[2720]: E0813 01:29:39.075603 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:29:39.079198 containerd[1567]: time="2025-08-13T01:29:39.078825155Z" level=info msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\"" Aug 13 01:29:39.295670 containerd[1567]: time="2025-08-13T01:29:39.295609906Z" level=error msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\" failed" error="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-294491888-fYh9 sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4258209367: no space left on device" Aug 13 01:29:39.296048 containerd[1567]: time="2025-08-13T01:29:39.295690226Z" level=info msg="stop pulling image registry.k8s.io/coredns/coredns:v1.12.0: active requests=0, bytes read=0" Aug 13 01:29:39.296104 kubelet[2720]: E0813 01:29:39.295826 2720 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-294491888-fYh9 sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4258209367: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:29:39.296104 kubelet[2720]: E0813 01:29:39.295865 2720 kuberuntime_image.go:42] "Failed to pull image" err="failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-294491888-fYh9 sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4258209367: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:29:39.296104 kubelet[2720]: E0813 01:29:39.295998 2720 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:coredns,Image:registry.k8s.io/coredns/coredns:v1.12.0,Command:[],Args:[-conf /etc/coredns/Corefile],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:dns,HostPort:0,ContainerPort:53,Protocol:UDP,HostIP:,},ContainerPort{Name:dns-tcp,HostPort:0,ContainerPort:53,Protocol:TCP,HostIP:,},ContainerPort{Name:metrics,HostPort:0,ContainerPort:9153,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{memory: {{178257920 0} {} 170Mi BinarySI},},Requests:ResourceList{cpu: {{100 -3} {} 100m DecimalSI},memory: {{73400320 0} {} 70Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:config-volume,ReadOnly:true,MountPath:/etc/coredns,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-jdpj9,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{0 8080 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:60,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:5,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/ready,Port:{0 8181 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[NET_BIND_SERVICE],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8): ErrImagePull: failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-294491888-fYh9 sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4258209367: no space left on device" logger="UnhandledError" Aug 13 01:29:39.297289 kubelet[2720]: E0813 01:29:39.297231 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ErrImagePull: \"failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-294491888-fYh9 sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4258209367: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:29:39.355777 containerd[1567]: time="2025-08-13T01:29:39.355700343Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:39.692322 containerd[1567]: time="2025-08-13T01:29:39.692185888Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:39.824573 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:39.825840 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.738 [INFO][8746] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 14402 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.738 [INFO][8746] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.774 [INFO][8759] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" HandleID="k8s-pod-network.c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.774 [INFO][8759] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" HandleID="k8s-pod-network.c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cf610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:39.774360584 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.774 [INFO][8759] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.774 [INFO][8759] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.774 [INFO][8759] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.781 [INFO][8759] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" host="172-234-25-215" Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.785 [INFO][8759] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.795 [INFO][8759] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.798 [INFO][8759] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.800 [INFO][8759] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.800 [INFO][8759] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" host="172-234-25-215" Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.802 [INFO][8759] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133 Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.806 [INFO][8759] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" host="172-234-25-215" Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.817 [INFO][8759] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.47/26] block=192.168.62.0/26 handle="k8s-pod-network.c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" host="172-234-25-215" Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.847203 containerd[1567]: 2025-08-13 01:29:39.818 [INFO][8759] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.47/26] handle="k8s-pod-network.c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" host="172-234-25-215" Aug 13 01:29:39.847203 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.848816 containerd[1567]: 2025-08-13 01:29:39.818 [INFO][8759] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:39.848816 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.848816 containerd[1567]: 2025-08-13 01:29:39.818 [INFO][8759] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.47/26] IPv6=[] ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" HandleID="k8s-pod-network.c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:39.848816 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.848816 containerd[1567]: 2025-08-13 01:29:39.820 [INFO][8746] cni-plugin/k8s.go 418: Populated endpoint ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14402", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c1f125e481add99169a28f8f99b301f456ce1fd42eaefa651fecc3bf0fcfc398", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.47/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:39.848816 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.848816 containerd[1567]: 2025-08-13 01:29:39.820 [INFO][8746] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.47/32] ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:39.848816 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.848816 containerd[1567]: 2025-08-13 01:29:39.820 [INFO][8746] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:39.848816 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.848816 containerd[1567]: 2025-08-13 01:29:39.824 [INFO][8746] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:39.848816 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.848816 containerd[1567]: 2025-08-13 01:29:39.826 [INFO][8746] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14402", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.47/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"a2:38:00:88:4c:18", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:39.848816 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.849880 containerd[1567]: 2025-08-13 01:29:39.837 [INFO][8746] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:39.849880 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:39.922341 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:39.922527 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:40.013443 containerd[1567]: 2025-08-13 01:29:39.915 [INFO][8783] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Aug 13 01:29:40.013443 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.013443 containerd[1567]: 2025-08-13 01:29:39.916 [INFO][8783] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" iface="eth0" netns="/var/run/netns/cni-8bc8d575-b1ca-202a-8231-7b0b34d5a2d8" Aug 13 01:29:40.013443 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.013443 containerd[1567]: 2025-08-13 01:29:39.919 [INFO][8783] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" iface="eth0" netns="/var/run/netns/cni-8bc8d575-b1ca-202a-8231-7b0b34d5a2d8" Aug 13 01:29:40.013443 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.013443 containerd[1567]: 2025-08-13 01:29:39.930 [INFO][8783] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" after=14.634022ms iface="eth0" netns="/var/run/netns/cni-8bc8d575-b1ca-202a-8231-7b0b34d5a2d8" Aug 13 01:29:40.013443 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.013443 containerd[1567]: 2025-08-13 01:29:39.930 [INFO][8783] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Aug 13 01:29:40.013443 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.013443 containerd[1567]: 2025-08-13 01:29:39.930 [INFO][8783] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Aug 13 01:29:40.013443 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.013443 containerd[1567]: 2025-08-13 01:29:39.969 [INFO][8794] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" HandleID="k8s-pod-network.c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:40.013443 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.013443 containerd[1567]: 2025-08-13 01:29:39.969 [INFO][8794] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:40.013443 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.013443 containerd[1567]: 2025-08-13 01:29:39.969 [INFO][8794] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:40.013443 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.013443 containerd[1567]: 2025-08-13 01:29:40.003 [INFO][8794] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" HandleID="k8s-pod-network.c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:40.013443 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.013443 containerd[1567]: 2025-08-13 01:29:40.003 [INFO][8794] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" HandleID="k8s-pod-network.c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:40.013443 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.013443 containerd[1567]: 2025-08-13 01:29:40.005 [INFO][8794] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:40.013443 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.013443 containerd[1567]: 2025-08-13 01:29:40.010 [INFO][8783] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133" Aug 13 01:29:40.013443 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.015534 containerd[1567]: time="2025-08-13T01:29:40.015473020Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133-lo: no space left on device" Aug 13 01:29:40.016084 kubelet[2720]: E0813 01:29:40.015953 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133-lo: no space left on device" Aug 13 01:29:40.016169 kubelet[2720]: E0813 01:29:40.016047 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:40.016335 kubelet[2720]: E0813 01:29:40.016214 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:40.016493 kubelet[2720]: E0813 01:29:40.016298 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:40.018203 systemd[1]: run-netns-cni\x2d8bc8d575\x2db1ca\x2d202a\x2d8231\x2d7b0b34d5a2d8.mount: Deactivated successfully. Aug 13 01:29:40.694551 containerd[1567]: time="2025-08-13T01:29:40.694475251Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:40.804642 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:40.806352 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.732 [INFO][8801] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 14473 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.732 [INFO][8801] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.763 [INFO][8815] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" HandleID="k8s-pod-network.5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.764 [INFO][8815] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" HandleID="k8s-pod-network.5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:40.76397714 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.764 [INFO][8815] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.764 [INFO][8815] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.764 [INFO][8815] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.770 [INFO][8815] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" host="172-234-25-215" Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.774 [INFO][8815] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.778 [INFO][8815] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.780 [INFO][8815] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.783 [INFO][8815] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.783 [INFO][8815] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" host="172-234-25-215" Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.785 [INFO][8815] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.790 [INFO][8815] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" host="172-234-25-215" Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.795 [INFO][8815] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.48/26] block=192.168.62.0/26 handle="k8s-pod-network.5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" host="172-234-25-215" Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.821402 containerd[1567]: 2025-08-13 01:29:40.795 [INFO][8815] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.48/26] handle="k8s-pod-network.5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" host="172-234-25-215" Aug 13 01:29:40.821402 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.822327 containerd[1567]: 2025-08-13 01:29:40.795 [INFO][8815] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:40.822327 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.822327 containerd[1567]: 2025-08-13 01:29:40.795 [INFO][8815] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.48/26] IPv6=[] ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" HandleID="k8s-pod-network.5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:40.822327 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.822327 containerd[1567]: 2025-08-13 01:29:40.799 [INFO][8801] cni-plugin/k8s.go 418: Populated endpoint ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14473", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c68426466051af8852178eb88b948b419dcae4132ddf4b531e6018eaf04d0133", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.48/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:40.822327 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.822327 containerd[1567]: 2025-08-13 01:29:40.799 [INFO][8801] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.48/32] ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:40.822327 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.822327 containerd[1567]: 2025-08-13 01:29:40.799 [INFO][8801] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:40.822327 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.822327 containerd[1567]: 2025-08-13 01:29:40.807 [INFO][8801] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:40.822327 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.822327 containerd[1567]: 2025-08-13 01:29:40.808 [INFO][8801] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14473", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.48/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"a6:87:3f:b2:d3:50", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:40.822327 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.822620 containerd[1567]: 2025-08-13 01:29:40.815 [INFO][8801] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:40.822620 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.888624 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:40.888635 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:40.971851 containerd[1567]: 2025-08-13 01:29:40.886 [INFO][8839] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Aug 13 01:29:40.971851 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.971851 containerd[1567]: 2025-08-13 01:29:40.887 [INFO][8839] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" iface="eth0" netns="/var/run/netns/cni-a092bd7a-38cc-068b-66d6-c97a11524f3a" Aug 13 01:29:40.971851 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.971851 containerd[1567]: 2025-08-13 01:29:40.887 [INFO][8839] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" iface="eth0" netns="/var/run/netns/cni-a092bd7a-38cc-068b-66d6-c97a11524f3a" Aug 13 01:29:40.971851 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.971851 containerd[1567]: 2025-08-13 01:29:40.895 [INFO][8839] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" after=8.694894ms iface="eth0" netns="/var/run/netns/cni-a092bd7a-38cc-068b-66d6-c97a11524f3a" Aug 13 01:29:40.971851 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.971851 containerd[1567]: 2025-08-13 01:29:40.895 [INFO][8839] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Aug 13 01:29:40.971851 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.971851 containerd[1567]: 2025-08-13 01:29:40.896 [INFO][8839] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Aug 13 01:29:40.971851 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.971851 containerd[1567]: 2025-08-13 01:29:40.932 [INFO][8846] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" HandleID="k8s-pod-network.5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:40.971851 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.971851 containerd[1567]: 2025-08-13 01:29:40.933 [INFO][8846] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:40.971851 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.971851 containerd[1567]: 2025-08-13 01:29:40.933 [INFO][8846] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:40.971851 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.971851 containerd[1567]: 2025-08-13 01:29:40.965 [INFO][8846] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" HandleID="k8s-pod-network.5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:40.971851 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.971851 containerd[1567]: 2025-08-13 01:29:40.965 [INFO][8846] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" HandleID="k8s-pod-network.5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:40.971851 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.971851 containerd[1567]: 2025-08-13 01:29:40.967 [INFO][8846] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:40.971851 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.971851 containerd[1567]: 2025-08-13 01:29:40.969 [INFO][8839] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b" Aug 13 01:29:40.971851 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:40.975616 containerd[1567]: time="2025-08-13T01:29:40.973925074Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b-lo: no space left on device" Aug 13 01:29:40.974364 systemd[1]: run-netns-cni\x2da092bd7a\x2d38cc\x2d068b\x2d66d6\x2dc97a11524f3a.mount: Deactivated successfully. Aug 13 01:29:40.976416 kubelet[2720]: E0813 01:29:40.976105 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b-lo: no space left on device" Aug 13 01:29:40.976416 kubelet[2720]: E0813 01:29:40.976160 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:40.976416 kubelet[2720]: E0813 01:29:40.976182 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:40.976416 kubelet[2720]: E0813 01:29:40.976232 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:41.256655 systemd[1]: Started sshd@33-172.234.25.215:22-147.75.109.163:53080.service - OpenSSH per-connection server daemon (147.75.109.163:53080). Aug 13 01:29:41.612752 sshd[8856]: Accepted publickey for core from 147.75.109.163 port 53080 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:29:41.615303 sshd-session[8856]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:29:41.621799 systemd-logind[1543]: New session 34 of user core. Aug 13 01:29:41.629915 systemd[1]: Started session-34.scope - Session 34 of User core. Aug 13 01:29:41.696934 containerd[1567]: time="2025-08-13T01:29:41.696853633Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:41.843344 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:41.847080 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.736 [INFO][8859] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 14534 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.736 [INFO][8859] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.772 [INFO][8874] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" HandleID="k8s-pod-network.f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.772 [INFO][8874] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" HandleID="k8s-pod-network.f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f040), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:41.772297431 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.772 [INFO][8874] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.773 [INFO][8874] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.773 [INFO][8874] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.779 [INFO][8874] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" host="172-234-25-215" Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.784 [INFO][8874] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.792 [INFO][8874] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.794 [INFO][8874] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.797 [INFO][8874] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.797 [INFO][8874] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" host="172-234-25-215" Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.800 [INFO][8874] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.804 [INFO][8874] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" host="172-234-25-215" Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.823 [INFO][8874] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.49/26] block=192.168.62.0/26 handle="k8s-pod-network.f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" host="172-234-25-215" Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.878390 containerd[1567]: 2025-08-13 01:29:41.823 [INFO][8874] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.49/26] handle="k8s-pod-network.f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" host="172-234-25-215" Aug 13 01:29:41.878390 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.884657 containerd[1567]: 2025-08-13 01:29:41.823 [INFO][8874] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:41.884657 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.884657 containerd[1567]: 2025-08-13 01:29:41.823 [INFO][8874] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.49/26] IPv6=[] ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" HandleID="k8s-pod-network.f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:41.884657 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.884657 containerd[1567]: 2025-08-13 01:29:41.827 [INFO][8859] cni-plugin/k8s.go 418: Populated endpoint ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14534", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"5201ee4666b5f1749f3eee0af3dad5980211b203c298d98257840389e7aaf95b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.49/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:41.884657 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.884657 containerd[1567]: 2025-08-13 01:29:41.828 [INFO][8859] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.49/32] ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:41.884657 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.884657 containerd[1567]: 2025-08-13 01:29:41.828 [INFO][8859] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:41.884657 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.884657 containerd[1567]: 2025-08-13 01:29:41.847 [INFO][8859] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:41.884657 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.884657 containerd[1567]: 2025-08-13 01:29:41.849 [INFO][8859] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14534", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.49/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"4e:3e:93:55:48:86", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:41.884657 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.885998 containerd[1567]: 2025-08-13 01:29:41.869 [INFO][8859] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:41.885998 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:41.944433 sshd[8858]: Connection closed by 147.75.109.163 port 53080 Aug 13 01:29:41.947811 sshd-session[8856]: pam_unix(sshd:session): session closed for user core Aug 13 01:29:41.952890 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:41.952900 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:41.958027 systemd[1]: sshd@33-172.234.25.215:22-147.75.109.163:53080.service: Deactivated successfully. Aug 13 01:29:41.963654 systemd[1]: session-34.scope: Deactivated successfully. Aug 13 01:29:41.972928 systemd-logind[1543]: Session 34 logged out. Waiting for processes to exit. Aug 13 01:29:41.977250 systemd-logind[1543]: Removed session 34. Aug 13 01:29:42.089426 containerd[1567]: 2025-08-13 01:29:41.950 [INFO][8903] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Aug 13 01:29:42.089426 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.089426 containerd[1567]: 2025-08-13 01:29:41.951 [INFO][8903] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" iface="eth0" netns="/var/run/netns/cni-f5b5d0ab-7af5-3b0c-b537-89c7a567095f" Aug 13 01:29:42.089426 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.089426 containerd[1567]: 2025-08-13 01:29:41.951 [INFO][8903] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" iface="eth0" netns="/var/run/netns/cni-f5b5d0ab-7af5-3b0c-b537-89c7a567095f" Aug 13 01:29:42.089426 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.089426 containerd[1567]: 2025-08-13 01:29:41.961 [INFO][8903] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" after=9.737232ms iface="eth0" netns="/var/run/netns/cni-f5b5d0ab-7af5-3b0c-b537-89c7a567095f" Aug 13 01:29:42.089426 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.089426 containerd[1567]: 2025-08-13 01:29:41.962 [INFO][8903] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Aug 13 01:29:42.089426 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.089426 containerd[1567]: 2025-08-13 01:29:41.962 [INFO][8903] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Aug 13 01:29:42.089426 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.089426 containerd[1567]: 2025-08-13 01:29:42.029 [INFO][8914] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" HandleID="k8s-pod-network.f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:42.089426 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.089426 containerd[1567]: 2025-08-13 01:29:42.030 [INFO][8914] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:42.089426 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.089426 containerd[1567]: 2025-08-13 01:29:42.030 [INFO][8914] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:42.089426 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.089426 containerd[1567]: 2025-08-13 01:29:42.079 [INFO][8914] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" HandleID="k8s-pod-network.f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:42.089426 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.089426 containerd[1567]: 2025-08-13 01:29:42.080 [INFO][8914] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" HandleID="k8s-pod-network.f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:42.089426 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.089426 containerd[1567]: 2025-08-13 01:29:42.082 [INFO][8914] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:42.089426 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.089426 containerd[1567]: 2025-08-13 01:29:42.087 [INFO][8903] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c" Aug 13 01:29:42.089426 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.092023 containerd[1567]: time="2025-08-13T01:29:42.091937149Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c-lo: no space left on device" Aug 13 01:29:42.093749 kubelet[2720]: E0813 01:29:42.092406 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c-lo: no space left on device" Aug 13 01:29:42.093749 kubelet[2720]: E0813 01:29:42.092459 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:42.093749 kubelet[2720]: E0813 01:29:42.092478 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:42.093749 kubelet[2720]: E0813 01:29:42.092520 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:42.095698 systemd[1]: run-netns-cni\x2df5b5d0ab\x2d7af5\x2d3b0c\x2db537\x2d89c7a567095f.mount: Deactivated successfully. Aug 13 01:29:42.355944 containerd[1567]: time="2025-08-13T01:29:42.355879423Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:42.700221 containerd[1567]: time="2025-08-13T01:29:42.700145066Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:42.801917 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:42.803920 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.733 [INFO][8925] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 14616 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.733 [INFO][8925] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.761 [INFO][8939] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" HandleID="k8s-pod-network.1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.762 [INFO][8939] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" HandleID="k8s-pod-network.1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f240), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:42.76176083 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.762 [INFO][8939] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.762 [INFO][8939] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.762 [INFO][8939] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.769 [INFO][8939] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" host="172-234-25-215" Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.774 [INFO][8939] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.778 [INFO][8939] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.779 [INFO][8939] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.782 [INFO][8939] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.782 [INFO][8939] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" host="172-234-25-215" Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.783 [INFO][8939] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626 Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.787 [INFO][8939] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" host="172-234-25-215" Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.792 [INFO][8939] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.50/26] block=192.168.62.0/26 handle="k8s-pod-network.1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" host="172-234-25-215" Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.829857 containerd[1567]: 2025-08-13 01:29:42.793 [INFO][8939] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.50/26] handle="k8s-pod-network.1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" host="172-234-25-215" Aug 13 01:29:42.829857 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.830618 containerd[1567]: 2025-08-13 01:29:42.793 [INFO][8939] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:42.830618 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.830618 containerd[1567]: 2025-08-13 01:29:42.793 [INFO][8939] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.50/26] IPv6=[] ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" HandleID="k8s-pod-network.1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:42.830618 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.830618 containerd[1567]: 2025-08-13 01:29:42.795 [INFO][8925] cni-plugin/k8s.go 418: Populated endpoint ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14616", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"f160a16b09e892b623da2ea3ddafa2e37d6ea53a51f97033d1c2ecc8d6671f7c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.50/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:42.830618 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.830618 containerd[1567]: 2025-08-13 01:29:42.795 [INFO][8925] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.50/32] ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:42.830618 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.830618 containerd[1567]: 2025-08-13 01:29:42.795 [INFO][8925] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:42.830618 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.830618 containerd[1567]: 2025-08-13 01:29:42.804 [INFO][8925] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:42.830618 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.830618 containerd[1567]: 2025-08-13 01:29:42.805 [INFO][8925] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14616", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.50/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"0a:6d:da:48:3a:9e", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:42.830618 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.832077 containerd[1567]: 2025-08-13 01:29:42.821 [INFO][8925] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:42.832077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.889609 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:42.891006 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:42.987585 containerd[1567]: 2025-08-13 01:29:42.885 [INFO][8963] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Aug 13 01:29:42.987585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.987585 containerd[1567]: 2025-08-13 01:29:42.886 [INFO][8963] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" iface="eth0" netns="/var/run/netns/cni-73540d3f-dd08-39cc-27a4-f2344d8f9d9f" Aug 13 01:29:42.987585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.987585 containerd[1567]: 2025-08-13 01:29:42.888 [INFO][8963] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" iface="eth0" netns="/var/run/netns/cni-73540d3f-dd08-39cc-27a4-f2344d8f9d9f" Aug 13 01:29:42.987585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.987585 containerd[1567]: 2025-08-13 01:29:42.896 [INFO][8963] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" after=7.884455ms iface="eth0" netns="/var/run/netns/cni-73540d3f-dd08-39cc-27a4-f2344d8f9d9f" Aug 13 01:29:42.987585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.987585 containerd[1567]: 2025-08-13 01:29:42.896 [INFO][8963] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Aug 13 01:29:42.987585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.987585 containerd[1567]: 2025-08-13 01:29:42.896 [INFO][8963] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Aug 13 01:29:42.987585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.987585 containerd[1567]: 2025-08-13 01:29:42.940 [INFO][8972] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" HandleID="k8s-pod-network.1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:42.987585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.987585 containerd[1567]: 2025-08-13 01:29:42.941 [INFO][8972] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:42.987585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.987585 containerd[1567]: 2025-08-13 01:29:42.941 [INFO][8972] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:42.987585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.987585 containerd[1567]: 2025-08-13 01:29:42.980 [INFO][8972] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" HandleID="k8s-pod-network.1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:42.987585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.987585 containerd[1567]: 2025-08-13 01:29:42.980 [INFO][8972] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" HandleID="k8s-pod-network.1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:42.987585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.987585 containerd[1567]: 2025-08-13 01:29:42.982 [INFO][8972] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:42.987585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.987585 containerd[1567]: 2025-08-13 01:29:42.984 [INFO][8963] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626" Aug 13 01:29:42.987585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:42.990610 containerd[1567]: time="2025-08-13T01:29:42.990559669Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626-lo: no space left on device" Aug 13 01:29:42.991034 kubelet[2720]: E0813 01:29:42.990962 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626-lo: no space left on device" Aug 13 01:29:42.991146 kubelet[2720]: E0813 01:29:42.991116 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:42.991199 kubelet[2720]: E0813 01:29:42.991154 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:42.991292 kubelet[2720]: E0813 01:29:42.991260 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:42.991967 systemd[1]: run-netns-cni\x2d73540d3f\x2ddd08\x2d39cc\x2d27a4\x2df2344d8f9d9f.mount: Deactivated successfully. Aug 13 01:29:43.701582 containerd[1567]: time="2025-08-13T01:29:43.701493483Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:43.807853 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:43.812109 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.740 [INFO][8983] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 14694 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.740 [INFO][8983] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.766 [INFO][8995] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" HandleID="k8s-pod-network.0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.766 [INFO][8995] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" HandleID="k8s-pod-network.0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024eff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:43.766249882 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.766 [INFO][8995] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.766 [INFO][8995] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.766 [INFO][8995] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.773 [INFO][8995] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" host="172-234-25-215" Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.777 [INFO][8995] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.781 [INFO][8995] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.783 [INFO][8995] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.785 [INFO][8995] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.785 [INFO][8995] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" host="172-234-25-215" Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.786 [INFO][8995] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188 Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.790 [INFO][8995] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" host="172-234-25-215" Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.796 [INFO][8995] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.51/26] block=192.168.62.0/26 handle="k8s-pod-network.0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" host="172-234-25-215" Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.832419 containerd[1567]: 2025-08-13 01:29:43.797 [INFO][8995] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.51/26] handle="k8s-pod-network.0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" host="172-234-25-215" Aug 13 01:29:43.832419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.835260 containerd[1567]: 2025-08-13 01:29:43.797 [INFO][8995] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:43.835260 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.835260 containerd[1567]: 2025-08-13 01:29:43.797 [INFO][8995] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.51/26] IPv6=[] ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" HandleID="k8s-pod-network.0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:43.835260 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.835260 containerd[1567]: 2025-08-13 01:29:43.801 [INFO][8983] cni-plugin/k8s.go 418: Populated endpoint ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14694", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"1840d5db368ae59df44222f3ab7c494bff04eb0dfb247a34356e747d52632626", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.51/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:43.835260 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.835260 containerd[1567]: 2025-08-13 01:29:43.801 [INFO][8983] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.51/32] ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:43.835260 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.835260 containerd[1567]: 2025-08-13 01:29:43.801 [INFO][8983] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:43.835260 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.835260 containerd[1567]: 2025-08-13 01:29:43.813 [INFO][8983] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:43.835260 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.835260 containerd[1567]: 2025-08-13 01:29:43.814 [INFO][8983] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14694", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.51/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"4e:fe:9e:c4:92:c4", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:43.835260 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.835828 containerd[1567]: 2025-08-13 01:29:43.823 [INFO][8983] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:43.835828 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:43.909299 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:43.909308 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:44.022087 containerd[1567]: 2025-08-13 01:29:43.905 [INFO][9018] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Aug 13 01:29:44.022087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.022087 containerd[1567]: 2025-08-13 01:29:43.908 [INFO][9018] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" iface="eth0" netns="/var/run/netns/cni-a12f3b7b-8b6f-9ab7-7095-e8c17fb14ee2" Aug 13 01:29:44.022087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.022087 containerd[1567]: 2025-08-13 01:29:43.908 [INFO][9018] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" iface="eth0" netns="/var/run/netns/cni-a12f3b7b-8b6f-9ab7-7095-e8c17fb14ee2" Aug 13 01:29:44.022087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.022087 containerd[1567]: 2025-08-13 01:29:43.917 [INFO][9018] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" after=8.634184ms iface="eth0" netns="/var/run/netns/cni-a12f3b7b-8b6f-9ab7-7095-e8c17fb14ee2" Aug 13 01:29:44.022087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.022087 containerd[1567]: 2025-08-13 01:29:43.917 [INFO][9018] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Aug 13 01:29:44.022087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.022087 containerd[1567]: 2025-08-13 01:29:43.917 [INFO][9018] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Aug 13 01:29:44.022087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.022087 containerd[1567]: 2025-08-13 01:29:43.951 [INFO][9027] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" HandleID="k8s-pod-network.0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:44.022087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.022087 containerd[1567]: 2025-08-13 01:29:43.952 [INFO][9027] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:44.022087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.022087 containerd[1567]: 2025-08-13 01:29:43.952 [INFO][9027] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:44.022087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.022087 containerd[1567]: 2025-08-13 01:29:44.014 [INFO][9027] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" HandleID="k8s-pod-network.0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:44.022087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.022087 containerd[1567]: 2025-08-13 01:29:44.014 [INFO][9027] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" HandleID="k8s-pod-network.0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:44.022087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.022087 containerd[1567]: 2025-08-13 01:29:44.017 [INFO][9027] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:44.022087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.022087 containerd[1567]: 2025-08-13 01:29:44.019 [INFO][9018] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188" Aug 13 01:29:44.022087 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.025398 containerd[1567]: time="2025-08-13T01:29:44.025329294Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188-lo: no space left on device" Aug 13 01:29:44.026495 systemd[1]: run-netns-cni\x2da12f3b7b\x2d8b6f\x2d9ab7\x2d7095\x2de8c17fb14ee2.mount: Deactivated successfully. Aug 13 01:29:44.027425 kubelet[2720]: E0813 01:29:44.026636 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188-lo: no space left on device" Aug 13 01:29:44.027425 kubelet[2720]: E0813 01:29:44.026697 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:44.027425 kubelet[2720]: E0813 01:29:44.026759 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:44.027425 kubelet[2720]: E0813 01:29:44.026822 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:44.704507 containerd[1567]: time="2025-08-13T01:29:44.704212089Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:44.820865 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:44.822869 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.742 [INFO][9037] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 14758 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.742 [INFO][9037] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.773 [INFO][9050] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" HandleID="k8s-pod-network.cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.773 [INFO][9050] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" HandleID="k8s-pod-network.cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cf100), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:44.77310812 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.773 [INFO][9050] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.773 [INFO][9050] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.773 [INFO][9050] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.780 [INFO][9050] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" host="172-234-25-215" Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.786 [INFO][9050] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.789 [INFO][9050] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.792 [INFO][9050] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.794 [INFO][9050] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.795 [INFO][9050] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" host="172-234-25-215" Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.796 [INFO][9050] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.802 [INFO][9050] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" host="172-234-25-215" Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.810 [INFO][9050] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.52/26] block=192.168.62.0/26 handle="k8s-pod-network.cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" host="172-234-25-215" Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.839036 containerd[1567]: 2025-08-13 01:29:44.811 [INFO][9050] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.52/26] handle="k8s-pod-network.cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" host="172-234-25-215" Aug 13 01:29:44.839036 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.841413 containerd[1567]: 2025-08-13 01:29:44.811 [INFO][9050] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:44.841413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.841413 containerd[1567]: 2025-08-13 01:29:44.812 [INFO][9050] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.52/26] IPv6=[] ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" HandleID="k8s-pod-network.cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:44.841413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.841413 containerd[1567]: 2025-08-13 01:29:44.815 [INFO][9037] cni-plugin/k8s.go 418: Populated endpoint ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14758", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0f5d32acfdcc2f020d44e6c70c12d25a03d36cbb3a6eb3391b785b70cbfc6188", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.52/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:44.841413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.841413 containerd[1567]: 2025-08-13 01:29:44.815 [INFO][9037] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.52/32] ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:44.841413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.841413 containerd[1567]: 2025-08-13 01:29:44.815 [INFO][9037] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:44.841413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.841413 containerd[1567]: 2025-08-13 01:29:44.823 [INFO][9037] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:44.841413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.841413 containerd[1567]: 2025-08-13 01:29:44.824 [INFO][9037] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14758", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.52/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"3e:8e:5b:36:f5:7e", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:44.841413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.842706 containerd[1567]: 2025-08-13 01:29:44.832 [INFO][9037] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:44.842706 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:44.905798 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:44.906699 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:45.004702 containerd[1567]: 2025-08-13 01:29:44.903 [INFO][9074] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Aug 13 01:29:45.004702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.004702 containerd[1567]: 2025-08-13 01:29:44.903 [INFO][9074] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" iface="eth0" netns="/var/run/netns/cni-9a7c697a-df10-5237-1164-26d0075f6c9f" Aug 13 01:29:45.004702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.004702 containerd[1567]: 2025-08-13 01:29:44.904 [INFO][9074] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" iface="eth0" netns="/var/run/netns/cni-9a7c697a-df10-5237-1164-26d0075f6c9f" Aug 13 01:29:45.004702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.004702 containerd[1567]: 2025-08-13 01:29:44.914 [INFO][9074] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" after=10.50604ms iface="eth0" netns="/var/run/netns/cni-9a7c697a-df10-5237-1164-26d0075f6c9f" Aug 13 01:29:45.004702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.004702 containerd[1567]: 2025-08-13 01:29:44.914 [INFO][9074] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Aug 13 01:29:45.004702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.004702 containerd[1567]: 2025-08-13 01:29:44.914 [INFO][9074] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Aug 13 01:29:45.004702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.004702 containerd[1567]: 2025-08-13 01:29:44.959 [INFO][9082] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" HandleID="k8s-pod-network.cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:45.004702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.004702 containerd[1567]: 2025-08-13 01:29:44.959 [INFO][9082] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:45.004702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.004702 containerd[1567]: 2025-08-13 01:29:44.959 [INFO][9082] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:45.004702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.004702 containerd[1567]: 2025-08-13 01:29:44.995 [INFO][9082] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" HandleID="k8s-pod-network.cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:45.004702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.004702 containerd[1567]: 2025-08-13 01:29:44.995 [INFO][9082] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" HandleID="k8s-pod-network.cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:45.004702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.004702 containerd[1567]: 2025-08-13 01:29:44.997 [INFO][9082] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:45.004702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.004702 containerd[1567]: 2025-08-13 01:29:45.000 [INFO][9074] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f" Aug 13 01:29:45.004702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.012117 containerd[1567]: time="2025-08-13T01:29:45.012028361Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f-lo: no space left on device" Aug 13 01:29:45.012665 systemd[1]: run-netns-cni\x2d9a7c697a\x2ddf10\x2d5237\x2d1164\x2d26d0075f6c9f.mount: Deactivated successfully. Aug 13 01:29:45.013680 kubelet[2720]: E0813 01:29:45.013569 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f-lo: no space left on device" Aug 13 01:29:45.013680 kubelet[2720]: E0813 01:29:45.013623 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:45.013680 kubelet[2720]: E0813 01:29:45.013643 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:45.013866 kubelet[2720]: E0813 01:29:45.013831 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:45.356058 containerd[1567]: time="2025-08-13T01:29:45.355956676Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:45.706219 containerd[1567]: time="2025-08-13T01:29:45.705913439Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:45.826998 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:45.829865 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.745 [INFO][9096] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 14831 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.746 [INFO][9096] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.770 [INFO][9107] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" HandleID="k8s-pod-network.189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.771 [INFO][9107] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" HandleID="k8s-pod-network.189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002c56c0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:45.770927776 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.771 [INFO][9107] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.771 [INFO][9107] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.771 [INFO][9107] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.776 [INFO][9107] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" host="172-234-25-215" Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.780 [INFO][9107] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.784 [INFO][9107] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.786 [INFO][9107] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.787 [INFO][9107] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.788 [INFO][9107] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" host="172-234-25-215" Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.789 [INFO][9107] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.795 [INFO][9107] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" host="172-234-25-215" Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.816 [INFO][9107] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.53/26] block=192.168.62.0/26 handle="k8s-pod-network.189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" host="172-234-25-215" Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.854362 containerd[1567]: 2025-08-13 01:29:45.816 [INFO][9107] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.53/26] handle="k8s-pod-network.189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" host="172-234-25-215" Aug 13 01:29:45.854362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.856159 containerd[1567]: 2025-08-13 01:29:45.816 [INFO][9107] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:45.856159 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.856159 containerd[1567]: 2025-08-13 01:29:45.816 [INFO][9107] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.53/26] IPv6=[] ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" HandleID="k8s-pod-network.189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:45.856159 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.856159 containerd[1567]: 2025-08-13 01:29:45.820 [INFO][9096] cni-plugin/k8s.go 418: Populated endpoint ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14831", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"cdb3d3963c0d936e889ccac5d6796c76c8b0a2dd5fc9f3387d6436fa70f9a63f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.53/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:45.856159 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.856159 containerd[1567]: 2025-08-13 01:29:45.820 [INFO][9096] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.53/32] ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:45.856159 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.856159 containerd[1567]: 2025-08-13 01:29:45.820 [INFO][9096] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:45.856159 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.856159 containerd[1567]: 2025-08-13 01:29:45.829 [INFO][9096] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:45.856159 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.856159 containerd[1567]: 2025-08-13 01:29:45.830 [INFO][9096] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14831", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.53/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"1e:3a:20:e3:09:71", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:45.856159 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.856457 containerd[1567]: 2025-08-13 01:29:45.843 [INFO][9096] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:45.856457 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:45.923876 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:45.923885 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:46.038362 containerd[1567]: 2025-08-13 01:29:45.922 [INFO][9129] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Aug 13 01:29:46.038362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.038362 containerd[1567]: 2025-08-13 01:29:45.922 [INFO][9129] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" iface="eth0" netns="/var/run/netns/cni-16c4c24c-fb01-602b-1dd9-b9cb35b82d50" Aug 13 01:29:46.038362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.038362 containerd[1567]: 2025-08-13 01:29:45.922 [INFO][9129] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" iface="eth0" netns="/var/run/netns/cni-16c4c24c-fb01-602b-1dd9-b9cb35b82d50" Aug 13 01:29:46.038362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.038362 containerd[1567]: 2025-08-13 01:29:45.931 [INFO][9129] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" after=9.194563ms iface="eth0" netns="/var/run/netns/cni-16c4c24c-fb01-602b-1dd9-b9cb35b82d50" Aug 13 01:29:46.038362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.038362 containerd[1567]: 2025-08-13 01:29:45.931 [INFO][9129] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Aug 13 01:29:46.038362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.038362 containerd[1567]: 2025-08-13 01:29:45.932 [INFO][9129] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Aug 13 01:29:46.038362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.038362 containerd[1567]: 2025-08-13 01:29:45.972 [INFO][9136] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" HandleID="k8s-pod-network.189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:46.038362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.038362 containerd[1567]: 2025-08-13 01:29:45.973 [INFO][9136] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:46.038362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.038362 containerd[1567]: 2025-08-13 01:29:45.973 [INFO][9136] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:46.038362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.038362 containerd[1567]: 2025-08-13 01:29:46.026 [INFO][9136] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" HandleID="k8s-pod-network.189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:46.038362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.038362 containerd[1567]: 2025-08-13 01:29:46.026 [INFO][9136] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" HandleID="k8s-pod-network.189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:46.038362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.038362 containerd[1567]: 2025-08-13 01:29:46.030 [INFO][9136] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:46.038362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.038362 containerd[1567]: 2025-08-13 01:29:46.034 [INFO][9129] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb" Aug 13 01:29:46.038362 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.043481 systemd[1]: run-netns-cni\x2d16c4c24c\x2dfb01\x2d602b\x2d1dd9\x2db9cb35b82d50.mount: Deactivated successfully. Aug 13 01:29:46.044055 containerd[1567]: time="2025-08-13T01:29:46.043701635Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb-lo: no space left on device" Aug 13 01:29:46.047705 kubelet[2720]: E0813 01:29:46.047666 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb-lo: no space left on device" Aug 13 01:29:46.048731 kubelet[2720]: E0813 01:29:46.048108 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:46.048731 kubelet[2720]: E0813 01:29:46.048136 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:46.048731 kubelet[2720]: E0813 01:29:46.048188 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:46.707830 containerd[1567]: time="2025-08-13T01:29:46.707776428Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:46.808644 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:46.809069 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.743 [INFO][9149] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 14910 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.743 [INFO][9149] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.769 [INFO][9161] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" HandleID="k8s-pod-network.7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.769 [INFO][9161] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" HandleID="k8s-pod-network.7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:46.769212233 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.769 [INFO][9161] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.769 [INFO][9161] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.769 [INFO][9161] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.775 [INFO][9161] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" host="172-234-25-215" Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.779 [INFO][9161] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.783 [INFO][9161] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.784 [INFO][9161] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.788 [INFO][9161] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.788 [INFO][9161] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" host="172-234-25-215" Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.789 [INFO][9161] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171 Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.793 [INFO][9161] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" host="172-234-25-215" Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.800 [INFO][9161] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.54/26] block=192.168.62.0/26 handle="k8s-pod-network.7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" host="172-234-25-215" Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827048 containerd[1567]: 2025-08-13 01:29:46.800 [INFO][9161] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.54/26] handle="k8s-pod-network.7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" host="172-234-25-215" Aug 13 01:29:46.827048 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827735 containerd[1567]: 2025-08-13 01:29:46.800 [INFO][9161] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:46.827735 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827735 containerd[1567]: 2025-08-13 01:29:46.800 [INFO][9161] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.54/26] IPv6=[] ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" HandleID="k8s-pod-network.7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:46.827735 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827735 containerd[1567]: 2025-08-13 01:29:46.804 [INFO][9149] cni-plugin/k8s.go 418: Populated endpoint ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14910", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"189bede06c0d6a34f7cf9c2796e46ab760d1e37bffc24e87cc941c9dadc453bb", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.54/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:46.827735 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827735 containerd[1567]: 2025-08-13 01:29:46.804 [INFO][9149] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.54/32] ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:46.827735 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827735 containerd[1567]: 2025-08-13 01:29:46.804 [INFO][9149] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:46.827735 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827735 containerd[1567]: 2025-08-13 01:29:46.809 [INFO][9149] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:46.827735 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.827735 containerd[1567]: 2025-08-13 01:29:46.810 [INFO][9149] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14910", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.54/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ca:94:c5:11:da:f3", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:46.827735 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.828023 containerd[1567]: 2025-08-13 01:29:46.818 [INFO][9149] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:46.828023 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.897155 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:46.897190 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:46.979325 containerd[1567]: 2025-08-13 01:29:46.893 [INFO][9185] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Aug 13 01:29:46.979325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.979325 containerd[1567]: 2025-08-13 01:29:46.894 [INFO][9185] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" iface="eth0" netns="/var/run/netns/cni-c9b21137-4207-6608-e643-53b7bfded004" Aug 13 01:29:46.979325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.979325 containerd[1567]: 2025-08-13 01:29:46.894 [INFO][9185] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" iface="eth0" netns="/var/run/netns/cni-c9b21137-4207-6608-e643-53b7bfded004" Aug 13 01:29:46.979325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.979325 containerd[1567]: 2025-08-13 01:29:46.904 [INFO][9185] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" after=9.290543ms iface="eth0" netns="/var/run/netns/cni-c9b21137-4207-6608-e643-53b7bfded004" Aug 13 01:29:46.979325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.979325 containerd[1567]: 2025-08-13 01:29:46.904 [INFO][9185] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Aug 13 01:29:46.979325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.979325 containerd[1567]: 2025-08-13 01:29:46.904 [INFO][9185] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Aug 13 01:29:46.979325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.979325 containerd[1567]: 2025-08-13 01:29:46.936 [INFO][9193] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" HandleID="k8s-pod-network.7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:46.979325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.979325 containerd[1567]: 2025-08-13 01:29:46.937 [INFO][9193] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:46.979325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.979325 containerd[1567]: 2025-08-13 01:29:46.937 [INFO][9193] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:46.979325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.979325 containerd[1567]: 2025-08-13 01:29:46.973 [INFO][9193] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" HandleID="k8s-pod-network.7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:46.979325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.979325 containerd[1567]: 2025-08-13 01:29:46.973 [INFO][9193] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" HandleID="k8s-pod-network.7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:46.979325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.979325 containerd[1567]: 2025-08-13 01:29:46.975 [INFO][9193] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:46.979325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.979325 containerd[1567]: 2025-08-13 01:29:46.977 [INFO][9185] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171" Aug 13 01:29:46.979325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:46.983399 containerd[1567]: time="2025-08-13T01:29:46.980319378Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171-lo: no space left on device" Aug 13 01:29:46.983458 kubelet[2720]: E0813 01:29:46.980595 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171-lo: no space left on device" Aug 13 01:29:46.983458 kubelet[2720]: E0813 01:29:46.980669 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:46.983458 kubelet[2720]: E0813 01:29:46.980690 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:46.983458 kubelet[2720]: E0813 01:29:46.980798 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:46.984506 systemd[1]: run-netns-cni\x2dc9b21137\x2d4207\x2d6608\x2de643\x2d53b7bfded004.mount: Deactivated successfully. Aug 13 01:29:47.010126 systemd[1]: Started sshd@34-172.234.25.215:22-147.75.109.163:53088.service - OpenSSH per-connection server daemon (147.75.109.163:53088). Aug 13 01:29:47.348883 sshd[9204]: Accepted publickey for core from 147.75.109.163 port 53088 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:29:47.352416 sshd-session[9204]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:29:47.358896 systemd-logind[1543]: New session 35 of user core. Aug 13 01:29:47.366098 systemd[1]: Started session-35.scope - Session 35 of User core. Aug 13 01:29:47.668182 sshd[9206]: Connection closed by 147.75.109.163 port 53088 Aug 13 01:29:47.670183 sshd-session[9204]: pam_unix(sshd:session): session closed for user core Aug 13 01:29:47.673858 systemd-logind[1543]: Session 35 logged out. Waiting for processes to exit. Aug 13 01:29:47.676237 systemd[1]: sshd@34-172.234.25.215:22-147.75.109.163:53088.service: Deactivated successfully. Aug 13 01:29:47.680543 systemd[1]: session-35.scope: Deactivated successfully. Aug 13 01:29:47.683541 containerd[1567]: time="2025-08-13T01:29:47.683482959Z" level=info msg="TaskExit event in podsandbox handler container_id:\"05a7e6943f56eff710d015cc4d373aa0238d7506c1cf61f23b36bc5ebcb7ce84\" id:\"925a042ef4710b90f66918f846dece236be6715bfe2a75e0f4ea1ae28f05052c\" pid:9229 exited_at:{seconds:1755048587 nanos:682232211}" Aug 13 01:29:47.683813 systemd-logind[1543]: Removed session 35. Aug 13 01:29:47.709406 containerd[1567]: time="2025-08-13T01:29:47.709318190Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:47.824382 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:47.825153 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.750 [INFO][9246] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 14965 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.751 [INFO][9246] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.783 [INFO][9259] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" HandleID="k8s-pod-network.11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.783 [INFO][9259] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" HandleID="k8s-pod-network.11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f230), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:47.783483841 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.783 [INFO][9259] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.783 [INFO][9259] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.783 [INFO][9259] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.789 [INFO][9259] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" host="172-234-25-215" Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.793 [INFO][9259] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.797 [INFO][9259] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.799 [INFO][9259] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.804 [INFO][9259] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.804 [INFO][9259] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" host="172-234-25-215" Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.806 [INFO][9259] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.809 [INFO][9259] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" host="172-234-25-215" Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.815 [INFO][9259] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.55/26] block=192.168.62.0/26 handle="k8s-pod-network.11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" host="172-234-25-215" Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.845077 containerd[1567]: 2025-08-13 01:29:47.815 [INFO][9259] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.55/26] handle="k8s-pod-network.11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" host="172-234-25-215" Aug 13 01:29:47.845077 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.846922 containerd[1567]: 2025-08-13 01:29:47.815 [INFO][9259] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:47.846922 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.846922 containerd[1567]: 2025-08-13 01:29:47.815 [INFO][9259] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.55/26] IPv6=[] ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" HandleID="k8s-pod-network.11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:47.846922 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.846922 containerd[1567]: 2025-08-13 01:29:47.820 [INFO][9246] cni-plugin/k8s.go 418: Populated endpoint ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14965", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"7f0aac6f30657fcb68387d1f6a9bea439e0ae5415e5a09ddfcbad9b257fcb171", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.55/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:47.846922 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.846922 containerd[1567]: 2025-08-13 01:29:47.820 [INFO][9246] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.55/32] ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:47.846922 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.846922 containerd[1567]: 2025-08-13 01:29:47.820 [INFO][9246] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:47.846922 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.846922 containerd[1567]: 2025-08-13 01:29:47.825 [INFO][9246] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:47.846922 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.846922 containerd[1567]: 2025-08-13 01:29:47.826 [INFO][9246] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"14965", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.55/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"be:ee:51:77:74:fa", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:47.846922 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.848799 containerd[1567]: 2025-08-13 01:29:47.837 [INFO][9246] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:47.848799 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:47.914803 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:47.914960 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:48.020709 containerd[1567]: 2025-08-13 01:29:47.912 [INFO][9282] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Aug 13 01:29:48.020709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.020709 containerd[1567]: 2025-08-13 01:29:47.912 [INFO][9282] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" iface="eth0" netns="/var/run/netns/cni-e699c4a8-1f6b-3fb4-ff27-6da2a9af08b6" Aug 13 01:29:48.020709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.020709 containerd[1567]: 2025-08-13 01:29:47.913 [INFO][9282] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" iface="eth0" netns="/var/run/netns/cni-e699c4a8-1f6b-3fb4-ff27-6da2a9af08b6" Aug 13 01:29:48.020709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.020709 containerd[1567]: 2025-08-13 01:29:47.920 [INFO][9282] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" after=7.965955ms iface="eth0" netns="/var/run/netns/cni-e699c4a8-1f6b-3fb4-ff27-6da2a9af08b6" Aug 13 01:29:48.020709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.020709 containerd[1567]: 2025-08-13 01:29:47.920 [INFO][9282] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Aug 13 01:29:48.020709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.020709 containerd[1567]: 2025-08-13 01:29:47.920 [INFO][9282] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Aug 13 01:29:48.020709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.020709 containerd[1567]: 2025-08-13 01:29:47.953 [INFO][9291] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" HandleID="k8s-pod-network.11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:48.020709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.020709 containerd[1567]: 2025-08-13 01:29:47.953 [INFO][9291] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:48.020709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.020709 containerd[1567]: 2025-08-13 01:29:47.954 [INFO][9291] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:48.020709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.020709 containerd[1567]: 2025-08-13 01:29:48.011 [INFO][9291] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" HandleID="k8s-pod-network.11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:48.020709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.020709 containerd[1567]: 2025-08-13 01:29:48.011 [INFO][9291] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" HandleID="k8s-pod-network.11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:48.020709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.020709 containerd[1567]: 2025-08-13 01:29:48.013 [INFO][9291] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:48.020709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.020709 containerd[1567]: 2025-08-13 01:29:48.017 [INFO][9282] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b" Aug 13 01:29:48.020709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.024038 systemd[1]: run-netns-cni\x2de699c4a8\x2d1f6b\x2d3fb4\x2dff27\x2d6da2a9af08b6.mount: Deactivated successfully. Aug 13 01:29:48.024571 containerd[1567]: time="2025-08-13T01:29:48.024507199Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b-lo: no space left on device" Aug 13 01:29:48.024915 kubelet[2720]: E0813 01:29:48.024871 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b-lo: no space left on device" Aug 13 01:29:48.025176 kubelet[2720]: E0813 01:29:48.024950 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:48.025176 kubelet[2720]: E0813 01:29:48.024971 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:48.025176 kubelet[2720]: E0813 01:29:48.025041 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:48.216558 kubelet[2720]: I0813 01:29:48.216527 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:29:48.216660 kubelet[2720]: I0813 01:29:48.216571 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:29:48.219615 kubelet[2720]: I0813 01:29:48.219593 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:29:48.233695 kubelet[2720]: I0813 01:29:48.233659 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:29:48.233827 kubelet[2720]: I0813 01:29:48.233794 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-b2ds8","kube-system/coredns-674b8bbfcf-jlpx4","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:29:48.233827 kubelet[2720]: E0813 01:29:48.233825 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:48.233906 kubelet[2720]: E0813 01:29:48.233835 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:29:48.233906 kubelet[2720]: E0813 01:29:48.233842 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:29:48.233906 kubelet[2720]: E0813 01:29:48.233851 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:29:48.233906 kubelet[2720]: E0813 01:29:48.233860 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:29:48.233906 kubelet[2720]: E0813 01:29:48.233868 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:29:48.233906 kubelet[2720]: E0813 01:29:48.233875 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:29:48.233906 kubelet[2720]: E0813 01:29:48.233885 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:29:48.233906 kubelet[2720]: E0813 01:29:48.233893 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:29:48.233906 kubelet[2720]: E0813 01:29:48.233900 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:29:48.233906 kubelet[2720]: I0813 01:29:48.233908 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:29:48.247146 containerd[1567]: time="2025-08-13T01:29:48.247075363Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:29:48.356230 containerd[1567]: time="2025-08-13T01:29:48.356123478Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:48.711933 containerd[1567]: time="2025-08-13T01:29:48.711702903Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:48.811945 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:48.812145 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.746 [INFO][9299] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15047 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.746 [INFO][9299] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.773 [INFO][9311] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" HandleID="k8s-pod-network.c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.773 [INFO][9311] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" HandleID="k8s-pod-network.c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5640), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:48.773030187 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.773 [INFO][9311] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.773 [INFO][9311] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.773 [INFO][9311] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.780 [INFO][9311] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" host="172-234-25-215" Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.783 [INFO][9311] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.787 [INFO][9311] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.789 [INFO][9311] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.791 [INFO][9311] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.791 [INFO][9311] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" host="172-234-25-215" Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.792 [INFO][9311] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.797 [INFO][9311] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" host="172-234-25-215" Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.805 [INFO][9311] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.56/26] block=192.168.62.0/26 handle="k8s-pod-network.c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" host="172-234-25-215" Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.832830 containerd[1567]: 2025-08-13 01:29:48.805 [INFO][9311] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.56/26] handle="k8s-pod-network.c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" host="172-234-25-215" Aug 13 01:29:48.832830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.833520 containerd[1567]: 2025-08-13 01:29:48.805 [INFO][9311] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:48.833520 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.833520 containerd[1567]: 2025-08-13 01:29:48.805 [INFO][9311] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.56/26] IPv6=[] ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" HandleID="k8s-pod-network.c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:48.833520 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.833520 containerd[1567]: 2025-08-13 01:29:48.808 [INFO][9299] cni-plugin/k8s.go 418: Populated endpoint ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15047", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"11511e6daf290667bd43db7c175215ac42661ae9bca7ebb712273c3ba8d07e8b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.56/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:48.833520 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.833520 containerd[1567]: 2025-08-13 01:29:48.808 [INFO][9299] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.56/32] ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:48.833520 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.833520 containerd[1567]: 2025-08-13 01:29:48.809 [INFO][9299] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:48.833520 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.833520 containerd[1567]: 2025-08-13 01:29:48.811 [INFO][9299] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:48.833520 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.833520 containerd[1567]: 2025-08-13 01:29:48.813 [INFO][9299] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15047", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.56/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"82:d7:52:5b:4e:49", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:48.833520 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.837793 containerd[1567]: 2025-08-13 01:29:48.824 [INFO][9299] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:48.837793 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.910365 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:48.910997 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:48.992936 containerd[1567]: 2025-08-13 01:29:48.907 [INFO][9335] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Aug 13 01:29:48.992936 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.992936 containerd[1567]: 2025-08-13 01:29:48.907 [INFO][9335] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" iface="eth0" netns="/var/run/netns/cni-9a89d97f-478c-d946-69bc-fa7489860727" Aug 13 01:29:48.992936 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.992936 containerd[1567]: 2025-08-13 01:29:48.908 [INFO][9335] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" iface="eth0" netns="/var/run/netns/cni-9a89d97f-478c-d946-69bc-fa7489860727" Aug 13 01:29:48.992936 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.992936 containerd[1567]: 2025-08-13 01:29:48.917 [INFO][9335] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" after=10.066891ms iface="eth0" netns="/var/run/netns/cni-9a89d97f-478c-d946-69bc-fa7489860727" Aug 13 01:29:48.992936 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.992936 containerd[1567]: 2025-08-13 01:29:48.917 [INFO][9335] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Aug 13 01:29:48.992936 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.992936 containerd[1567]: 2025-08-13 01:29:48.917 [INFO][9335] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Aug 13 01:29:48.992936 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.992936 containerd[1567]: 2025-08-13 01:29:48.955 [INFO][9344] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" HandleID="k8s-pod-network.c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:48.992936 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.992936 containerd[1567]: 2025-08-13 01:29:48.955 [INFO][9344] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:48.992936 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.992936 containerd[1567]: 2025-08-13 01:29:48.955 [INFO][9344] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:48.992936 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.992936 containerd[1567]: 2025-08-13 01:29:48.987 [INFO][9344] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" HandleID="k8s-pod-network.c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:48.992936 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.992936 containerd[1567]: 2025-08-13 01:29:48.987 [INFO][9344] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" HandleID="k8s-pod-network.c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:48.992936 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.992936 containerd[1567]: 2025-08-13 01:29:48.988 [INFO][9344] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:48.992936 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.992936 containerd[1567]: 2025-08-13 01:29:48.990 [INFO][9335] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d" Aug 13 01:29:48.992936 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:48.997744 containerd[1567]: time="2025-08-13T01:29:48.997647356Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d-lo: no space left on device" Aug 13 01:29:48.997837 systemd[1]: run-netns-cni\x2d9a89d97f\x2d478c\x2dd946\x2d69bc\x2dfa7489860727.mount: Deactivated successfully. Aug 13 01:29:48.998380 kubelet[2720]: E0813 01:29:48.998273 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d-lo: no space left on device" Aug 13 01:29:48.998380 kubelet[2720]: E0813 01:29:48.998326 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:48.998380 kubelet[2720]: E0813 01:29:48.998348 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:48.998644 kubelet[2720]: E0813 01:29:48.998493 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:49.713657 containerd[1567]: time="2025-08-13T01:29:49.713603375Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:49.827016 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:49.827849 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.749 [INFO][9354] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15109 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.749 [INFO][9354] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.780 [INFO][9367] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" HandleID="k8s-pod-network.25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.780 [INFO][9367] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" HandleID="k8s-pod-network.25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d51c0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:49.78052959 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.780 [INFO][9367] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.780 [INFO][9367] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.780 [INFO][9367] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.788 [INFO][9367] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" host="172-234-25-215" Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.793 [INFO][9367] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.799 [INFO][9367] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.803 [INFO][9367] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.807 [INFO][9367] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.807 [INFO][9367] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" host="172-234-25-215" Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.809 [INFO][9367] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5 Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.814 [INFO][9367] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" host="172-234-25-215" Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.820 [INFO][9367] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.57/26] block=192.168.62.0/26 handle="k8s-pod-network.25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" host="172-234-25-215" Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.858281 containerd[1567]: 2025-08-13 01:29:49.820 [INFO][9367] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.57/26] handle="k8s-pod-network.25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" host="172-234-25-215" Aug 13 01:29:49.858281 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.860248 containerd[1567]: 2025-08-13 01:29:49.820 [INFO][9367] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:49.860248 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.860248 containerd[1567]: 2025-08-13 01:29:49.821 [INFO][9367] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.57/26] IPv6=[] ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" HandleID="k8s-pod-network.25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:49.860248 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.860248 containerd[1567]: 2025-08-13 01:29:49.823 [INFO][9354] cni-plugin/k8s.go 418: Populated endpoint ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15109", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c14b2de75e17375b362e365494c106d8c45820a8c4428c9f598f39bc40a8649d", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.57/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:49.860248 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.860248 containerd[1567]: 2025-08-13 01:29:49.823 [INFO][9354] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.57/32] ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:49.860248 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.860248 containerd[1567]: 2025-08-13 01:29:49.823 [INFO][9354] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:49.860248 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.860248 containerd[1567]: 2025-08-13 01:29:49.828 [INFO][9354] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:49.860248 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.860248 containerd[1567]: 2025-08-13 01:29:49.829 [INFO][9354] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15109", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.57/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"36:16:09:6f:f5:f9", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:49.860248 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.860548 containerd[1567]: 2025-08-13 01:29:49.845 [INFO][9354] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:49.860548 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:49.921523 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:49.921535 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:50.012205 containerd[1567]: 2025-08-13 01:29:49.919 [INFO][9390] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Aug 13 01:29:50.012205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.012205 containerd[1567]: 2025-08-13 01:29:49.919 [INFO][9390] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" iface="eth0" netns="/var/run/netns/cni-5cd6757d-3273-6a12-097c-d169ee9f14a6" Aug 13 01:29:50.012205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.012205 containerd[1567]: 2025-08-13 01:29:49.919 [INFO][9390] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" iface="eth0" netns="/var/run/netns/cni-5cd6757d-3273-6a12-097c-d169ee9f14a6" Aug 13 01:29:50.012205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.012205 containerd[1567]: 2025-08-13 01:29:49.929 [INFO][9390] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" after=9.350313ms iface="eth0" netns="/var/run/netns/cni-5cd6757d-3273-6a12-097c-d169ee9f14a6" Aug 13 01:29:50.012205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.012205 containerd[1567]: 2025-08-13 01:29:49.929 [INFO][9390] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Aug 13 01:29:50.012205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.012205 containerd[1567]: 2025-08-13 01:29:49.929 [INFO][9390] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Aug 13 01:29:50.012205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.012205 containerd[1567]: 2025-08-13 01:29:49.953 [INFO][9400] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" HandleID="k8s-pod-network.25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:50.012205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.012205 containerd[1567]: 2025-08-13 01:29:49.953 [INFO][9400] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:50.012205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.012205 containerd[1567]: 2025-08-13 01:29:49.954 [INFO][9400] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:50.012205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.012205 containerd[1567]: 2025-08-13 01:29:50.001 [INFO][9400] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" HandleID="k8s-pod-network.25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:50.012205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.012205 containerd[1567]: 2025-08-13 01:29:50.001 [INFO][9400] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" HandleID="k8s-pod-network.25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:50.012205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.012205 containerd[1567]: 2025-08-13 01:29:50.004 [INFO][9400] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:50.012205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.012205 containerd[1567]: 2025-08-13 01:29:50.007 [INFO][9390] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5" Aug 13 01:29:50.012205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.014166 systemd[1]: run-netns-cni\x2d5cd6757d\x2d3273\x2d6a12\x2d097c\x2dd169ee9f14a6.mount: Deactivated successfully. Aug 13 01:29:50.014888 containerd[1567]: time="2025-08-13T01:29:50.014473112Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5-lo: no space left on device" Aug 13 01:29:50.015417 kubelet[2720]: E0813 01:29:50.015207 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5-lo: no space left on device" Aug 13 01:29:50.015800 kubelet[2720]: E0813 01:29:50.015439 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:50.015800 kubelet[2720]: E0813 01:29:50.015464 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:50.015800 kubelet[2720]: E0813 01:29:50.015784 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:50.074785 kubelet[2720]: E0813 01:29:50.073628 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:29:50.075485 kubelet[2720]: E0813 01:29:50.075464 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:29:50.076395 kubelet[2720]: E0813 01:29:50.076356 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-294491888-fYh9 sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4258209367: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:29:50.077495 kubelet[2720]: E0813 01:29:50.077439 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-307916557-_eRC sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-3587324838: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:29:50.715505 containerd[1567]: time="2025-08-13T01:29:50.715468430Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:50.813526 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:50.814354 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.753 [INFO][9410] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15180 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.753 [INFO][9410] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.777 [INFO][9421] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" HandleID="k8s-pod-network.4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.778 [INFO][9421] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" HandleID="k8s-pod-network.4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5970), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:50.777871613 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.778 [INFO][9421] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.778 [INFO][9421] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.778 [INFO][9421] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.783 [INFO][9421] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" host="172-234-25-215" Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.787 [INFO][9421] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.790 [INFO][9421] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.793 [INFO][9421] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.795 [INFO][9421] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.795 [INFO][9421] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" host="172-234-25-215" Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.796 [INFO][9421] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.799 [INFO][9421] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" host="172-234-25-215" Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.804 [INFO][9421] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.58/26] block=192.168.62.0/26 handle="k8s-pod-network.4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" host="172-234-25-215" Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.830140 containerd[1567]: 2025-08-13 01:29:50.804 [INFO][9421] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.58/26] handle="k8s-pod-network.4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" host="172-234-25-215" Aug 13 01:29:50.830140 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.832548 containerd[1567]: 2025-08-13 01:29:50.804 [INFO][9421] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:50.832548 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.832548 containerd[1567]: 2025-08-13 01:29:50.804 [INFO][9421] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.58/26] IPv6=[] ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" HandleID="k8s-pod-network.4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:50.832548 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.832548 containerd[1567]: 2025-08-13 01:29:50.807 [INFO][9410] cni-plugin/k8s.go 418: Populated endpoint ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15180", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"25477355c8df4681029d61af377980d18fd36d0d31a6d50df29388de34c2bde5", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.58/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:50.832548 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.832548 containerd[1567]: 2025-08-13 01:29:50.807 [INFO][9410] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.58/32] ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:50.832548 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.832548 containerd[1567]: 2025-08-13 01:29:50.807 [INFO][9410] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:50.832548 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.832548 containerd[1567]: 2025-08-13 01:29:50.813 [INFO][9410] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:50.832548 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.832548 containerd[1567]: 2025-08-13 01:29:50.816 [INFO][9410] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15180", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.58/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"7e:6d:c7:d4:e5:a6", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:50.832548 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.833842 containerd[1567]: 2025-08-13 01:29:50.826 [INFO][9410] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:50.833842 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.886104 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:50.886112 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:50.991287 containerd[1567]: 2025-08-13 01:29:50.884 [INFO][9448] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Aug 13 01:29:50.991287 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.991287 containerd[1567]: 2025-08-13 01:29:50.884 [INFO][9448] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" iface="eth0" netns="/var/run/netns/cni-e0c2fa2a-d5a7-520e-35a3-2ff81834cb60" Aug 13 01:29:50.991287 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.991287 containerd[1567]: 2025-08-13 01:29:50.885 [INFO][9448] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" iface="eth0" netns="/var/run/netns/cni-e0c2fa2a-d5a7-520e-35a3-2ff81834cb60" Aug 13 01:29:50.991287 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.991287 containerd[1567]: 2025-08-13 01:29:50.893 [INFO][9448] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" after=8.342585ms iface="eth0" netns="/var/run/netns/cni-e0c2fa2a-d5a7-520e-35a3-2ff81834cb60" Aug 13 01:29:50.991287 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.991287 containerd[1567]: 2025-08-13 01:29:50.894 [INFO][9448] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Aug 13 01:29:50.991287 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.991287 containerd[1567]: 2025-08-13 01:29:50.894 [INFO][9448] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Aug 13 01:29:50.991287 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.991287 containerd[1567]: 2025-08-13 01:29:50.931 [INFO][9456] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" HandleID="k8s-pod-network.4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:50.991287 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.991287 containerd[1567]: 2025-08-13 01:29:50.932 [INFO][9456] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:50.991287 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.991287 containerd[1567]: 2025-08-13 01:29:50.932 [INFO][9456] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:50.991287 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.991287 containerd[1567]: 2025-08-13 01:29:50.985 [INFO][9456] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" HandleID="k8s-pod-network.4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:50.991287 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.991287 containerd[1567]: 2025-08-13 01:29:50.985 [INFO][9456] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" HandleID="k8s-pod-network.4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:50.991287 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.991287 containerd[1567]: 2025-08-13 01:29:50.986 [INFO][9456] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:50.991287 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.991287 containerd[1567]: 2025-08-13 01:29:50.989 [INFO][9448] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af" Aug 13 01:29:50.991287 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:50.993496 containerd[1567]: time="2025-08-13T01:29:50.992292062Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af-lo: no space left on device" Aug 13 01:29:50.993770 kubelet[2720]: E0813 01:29:50.992979 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af-lo: no space left on device" Aug 13 01:29:50.993770 kubelet[2720]: E0813 01:29:50.993242 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:50.993770 kubelet[2720]: E0813 01:29:50.993558 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:50.993770 kubelet[2720]: E0813 01:29:50.993607 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:50.995132 systemd[1]: run-netns-cni\x2de0c2fa2a\x2dd5a7\x2d520e\x2d35a3\x2d2ff81834cb60.mount: Deactivated successfully. Aug 13 01:29:51.356528 containerd[1567]: time="2025-08-13T01:29:51.356326861Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:51.716596 containerd[1567]: time="2025-08-13T01:29:51.716551877Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:51.820633 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:51.821498 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.755 [INFO][9466] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15233 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.755 [INFO][9466] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.783 [INFO][9479] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" HandleID="k8s-pod-network.e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.784 [INFO][9479] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" HandleID="k8s-pod-network.e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002ad490), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:51.783970001 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.784 [INFO][9479] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.784 [INFO][9479] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.784 [INFO][9479] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.790 [INFO][9479] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" host="172-234-25-215" Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.795 [INFO][9479] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.799 [INFO][9479] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.801 [INFO][9479] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.804 [INFO][9479] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.804 [INFO][9479] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" host="172-234-25-215" Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.806 [INFO][9479] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.808 [INFO][9479] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" host="172-234-25-215" Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.815 [INFO][9479] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.59/26] block=192.168.62.0/26 handle="k8s-pod-network.e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" host="172-234-25-215" Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.839630 containerd[1567]: 2025-08-13 01:29:51.815 [INFO][9479] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.59/26] handle="k8s-pod-network.e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" host="172-234-25-215" Aug 13 01:29:51.839630 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.842131 containerd[1567]: 2025-08-13 01:29:51.815 [INFO][9479] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:51.842131 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.842131 containerd[1567]: 2025-08-13 01:29:51.815 [INFO][9479] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.59/26] IPv6=[] ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" HandleID="k8s-pod-network.e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:51.842131 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.842131 containerd[1567]: 2025-08-13 01:29:51.818 [INFO][9466] cni-plugin/k8s.go 418: Populated endpoint ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15233", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"4b1e03f56ed0c1634f31271aa3503313ea476af5210e4e4c48b07463bf2330af", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.59/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:51.842131 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.842131 containerd[1567]: 2025-08-13 01:29:51.818 [INFO][9466] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.59/32] ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:51.842131 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.842131 containerd[1567]: 2025-08-13 01:29:51.818 [INFO][9466] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:51.842131 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.842131 containerd[1567]: 2025-08-13 01:29:51.821 [INFO][9466] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:51.842131 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.842131 containerd[1567]: 2025-08-13 01:29:51.822 [INFO][9466] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15233", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.59/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"5e:28:24:07:80:49", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:51.842131 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.842631 containerd[1567]: 2025-08-13 01:29:51.830 [INFO][9466] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:51.842631 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.911154 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:51.911170 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:51.991528 containerd[1567]: 2025-08-13 01:29:51.909 [INFO][9502] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Aug 13 01:29:51.991528 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.991528 containerd[1567]: 2025-08-13 01:29:51.909 [INFO][9502] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" iface="eth0" netns="/var/run/netns/cni-05c3c3bc-a7d8-59e9-2497-4e18b95983a1" Aug 13 01:29:51.991528 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.991528 containerd[1567]: 2025-08-13 01:29:51.910 [INFO][9502] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" iface="eth0" netns="/var/run/netns/cni-05c3c3bc-a7d8-59e9-2497-4e18b95983a1" Aug 13 01:29:51.991528 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.991528 containerd[1567]: 2025-08-13 01:29:51.915 [INFO][9502] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" after=5.839499ms iface="eth0" netns="/var/run/netns/cni-05c3c3bc-a7d8-59e9-2497-4e18b95983a1" Aug 13 01:29:51.991528 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.991528 containerd[1567]: 2025-08-13 01:29:51.915 [INFO][9502] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Aug 13 01:29:51.991528 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.991528 containerd[1567]: 2025-08-13 01:29:51.915 [INFO][9502] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Aug 13 01:29:51.991528 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.991528 containerd[1567]: 2025-08-13 01:29:51.953 [INFO][9511] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" HandleID="k8s-pod-network.e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:51.991528 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.991528 containerd[1567]: 2025-08-13 01:29:51.954 [INFO][9511] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:51.991528 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.991528 containerd[1567]: 2025-08-13 01:29:51.954 [INFO][9511] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:51.991528 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.991528 containerd[1567]: 2025-08-13 01:29:51.984 [INFO][9511] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" HandleID="k8s-pod-network.e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:51.991528 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.991528 containerd[1567]: 2025-08-13 01:29:51.984 [INFO][9511] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" HandleID="k8s-pod-network.e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:51.991528 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.991528 containerd[1567]: 2025-08-13 01:29:51.986 [INFO][9511] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:51.991528 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.991528 containerd[1567]: 2025-08-13 01:29:51.988 [INFO][9502] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c" Aug 13 01:29:51.991528 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:51.993968 containerd[1567]: time="2025-08-13T01:29:51.993703109Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c-lo: no space left on device" Aug 13 01:29:51.995770 kubelet[2720]: E0813 01:29:51.994875 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c-lo: no space left on device" Aug 13 01:29:51.995770 kubelet[2720]: E0813 01:29:51.994929 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:51.995770 kubelet[2720]: E0813 01:29:51.994948 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:51.995770 kubelet[2720]: E0813 01:29:51.994993 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:51.997906 systemd[1]: run-netns-cni\x2d05c3c3bc\x2da7d8\x2d59e9\x2d2497\x2d4e18b95983a1.mount: Deactivated successfully. Aug 13 01:29:52.718769 containerd[1567]: time="2025-08-13T01:29:52.718706524Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:52.743057 systemd[1]: Started sshd@35-172.234.25.215:22-147.75.109.163:53756.service - OpenSSH per-connection server daemon (147.75.109.163:53756). Aug 13 01:29:52.837750 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:52.838299 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.772 [INFO][9520] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15296 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.772 [INFO][9520] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.798 [INFO][9537] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" HandleID="k8s-pod-network.90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.799 [INFO][9537] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" HandleID="k8s-pod-network.90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f1a0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:52.798944273 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.799 [INFO][9537] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.799 [INFO][9537] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.799 [INFO][9537] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.807 [INFO][9537] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" host="172-234-25-215" Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.811 [INFO][9537] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.815 [INFO][9537] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.816 [INFO][9537] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.818 [INFO][9537] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.818 [INFO][9537] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" host="172-234-25-215" Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.820 [INFO][9537] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62 Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.823 [INFO][9537] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" host="172-234-25-215" Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.828 [INFO][9537] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.60/26] block=192.168.62.0/26 handle="k8s-pod-network.90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" host="172-234-25-215" Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.856764 containerd[1567]: 2025-08-13 01:29:52.829 [INFO][9537] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.60/26] handle="k8s-pod-network.90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" host="172-234-25-215" Aug 13 01:29:52.856764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.857644 containerd[1567]: 2025-08-13 01:29:52.829 [INFO][9537] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:52.857644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.857644 containerd[1567]: 2025-08-13 01:29:52.830 [INFO][9537] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.60/26] IPv6=[] ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" HandleID="k8s-pod-network.90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:52.857644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.857644 containerd[1567]: 2025-08-13 01:29:52.833 [INFO][9520] cni-plugin/k8s.go 418: Populated endpoint ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15296", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e2e894283e234fb1db490d3d14c5ddc8038c3cdd75457179902ef9ec3a3ccd5c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.60/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:52.857644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.857644 containerd[1567]: 2025-08-13 01:29:52.833 [INFO][9520] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.60/32] ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:52.857644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.857644 containerd[1567]: 2025-08-13 01:29:52.833 [INFO][9520] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:52.857644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.857644 containerd[1567]: 2025-08-13 01:29:52.836 [INFO][9520] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:52.857644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.857644 containerd[1567]: 2025-08-13 01:29:52.837 [INFO][9520] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15296", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.60/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"12:1c:17:74:1b:b2", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:52.857644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.857978 containerd[1567]: 2025-08-13 01:29:52.845 [INFO][9520] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:52.857978 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:52.922526 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:52.923384 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:53.035754 containerd[1567]: 2025-08-13 01:29:52.919 [INFO][9560] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Aug 13 01:29:53.035754 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.035754 containerd[1567]: 2025-08-13 01:29:52.920 [INFO][9560] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" iface="eth0" netns="/var/run/netns/cni-2016de64-48f1-d065-e0a6-df3393296c88" Aug 13 01:29:53.035754 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.035754 containerd[1567]: 2025-08-13 01:29:52.920 [INFO][9560] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" iface="eth0" netns="/var/run/netns/cni-2016de64-48f1-d065-e0a6-df3393296c88" Aug 13 01:29:53.035754 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.035754 containerd[1567]: 2025-08-13 01:29:52.929 [INFO][9560] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" after=9.436343ms iface="eth0" netns="/var/run/netns/cni-2016de64-48f1-d065-e0a6-df3393296c88" Aug 13 01:29:53.035754 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.035754 containerd[1567]: 2025-08-13 01:29:52.929 [INFO][9560] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Aug 13 01:29:53.035754 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.035754 containerd[1567]: 2025-08-13 01:29:52.929 [INFO][9560] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Aug 13 01:29:53.035754 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.035754 containerd[1567]: 2025-08-13 01:29:52.968 [INFO][9570] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" HandleID="k8s-pod-network.90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:53.035754 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.035754 containerd[1567]: 2025-08-13 01:29:52.969 [INFO][9570] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:53.035754 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.035754 containerd[1567]: 2025-08-13 01:29:52.969 [INFO][9570] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:53.035754 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.035754 containerd[1567]: 2025-08-13 01:29:53.025 [INFO][9570] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" HandleID="k8s-pod-network.90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:53.035754 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.035754 containerd[1567]: 2025-08-13 01:29:53.025 [INFO][9570] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" HandleID="k8s-pod-network.90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:53.035754 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.035754 containerd[1567]: 2025-08-13 01:29:53.030 [INFO][9570] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:53.035754 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.035754 containerd[1567]: 2025-08-13 01:29:53.033 [INFO][9560] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62" Aug 13 01:29:53.035754 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.043819 containerd[1567]: time="2025-08-13T01:29:53.043645766Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62-lo: no space left on device" Aug 13 01:29:53.044873 systemd[1]: run-netns-cni\x2d2016de64\x2d48f1\x2dd065\x2de0a6\x2ddf3393296c88.mount: Deactivated successfully. Aug 13 01:29:53.048115 kubelet[2720]: E0813 01:29:53.047288 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62-lo: no space left on device" Aug 13 01:29:53.048115 kubelet[2720]: E0813 01:29:53.047340 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:53.048115 kubelet[2720]: E0813 01:29:53.047359 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:53.048115 kubelet[2720]: E0813 01:29:53.047401 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:53.108792 sshd[9531]: Accepted publickey for core from 147.75.109.163 port 53756 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:29:53.112853 sshd-session[9531]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:29:53.121062 systemd-logind[1543]: New session 36 of user core. Aug 13 01:29:53.125964 systemd[1]: Started session-36.scope - Session 36 of User core. Aug 13 01:29:53.428456 sshd[9579]: Connection closed by 147.75.109.163 port 53756 Aug 13 01:29:53.429030 sshd-session[9531]: pam_unix(sshd:session): session closed for user core Aug 13 01:29:53.433584 systemd[1]: sshd@35-172.234.25.215:22-147.75.109.163:53756.service: Deactivated successfully. Aug 13 01:29:53.436212 systemd[1]: session-36.scope: Deactivated successfully. Aug 13 01:29:53.437185 systemd-logind[1543]: Session 36 logged out. Waiting for processes to exit. Aug 13 01:29:53.439273 systemd-logind[1543]: Removed session 36. Aug 13 01:29:53.721148 containerd[1567]: time="2025-08-13T01:29:53.721108751Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:53.833548 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:53.834740 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.767 [INFO][9590] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15357 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.767 [INFO][9590] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.793 [INFO][9605] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" HandleID="k8s-pod-network.30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.793 [INFO][9605] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" HandleID="k8s-pod-network.30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5600), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:53.793744484 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.793 [INFO][9605] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.794 [INFO][9605] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.794 [INFO][9605] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.800 [INFO][9605] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" host="172-234-25-215" Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.806 [INFO][9605] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.810 [INFO][9605] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.811 [INFO][9605] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.814 [INFO][9605] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.814 [INFO][9605] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" host="172-234-25-215" Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.815 [INFO][9605] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57 Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.820 [INFO][9605] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" host="172-234-25-215" Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.827 [INFO][9605] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.61/26] block=192.168.62.0/26 handle="k8s-pod-network.30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" host="172-234-25-215" Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.852017 containerd[1567]: 2025-08-13 01:29:53.827 [INFO][9605] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.61/26] handle="k8s-pod-network.30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" host="172-234-25-215" Aug 13 01:29:53.852017 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.853589 containerd[1567]: 2025-08-13 01:29:53.827 [INFO][9605] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:53.853589 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.853589 containerd[1567]: 2025-08-13 01:29:53.827 [INFO][9605] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.61/26] IPv6=[] ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" HandleID="k8s-pod-network.30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:53.853589 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.853589 containerd[1567]: 2025-08-13 01:29:53.830 [INFO][9590] cni-plugin/k8s.go 418: Populated endpoint ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15357", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"90b7607aef1949d11bb35ae738c5149c5110d10a531db5f35f32e6e7c2a29f62", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.61/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:53.853589 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.853589 containerd[1567]: 2025-08-13 01:29:53.830 [INFO][9590] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.61/32] ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:53.853589 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.853589 containerd[1567]: 2025-08-13 01:29:53.830 [INFO][9590] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:53.853589 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.853589 containerd[1567]: 2025-08-13 01:29:53.836 [INFO][9590] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:53.853589 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.853589 containerd[1567]: 2025-08-13 01:29:53.837 [INFO][9590] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15357", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.61/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"16:ed:eb:46:0b:51", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:53.853589 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.853910 containerd[1567]: 2025-08-13 01:29:53.849 [INFO][9590] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:53.853910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.913586 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:53.913596 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:53.996774 containerd[1567]: 2025-08-13 01:29:53.911 [INFO][9628] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Aug 13 01:29:53.996774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.996774 containerd[1567]: 2025-08-13 01:29:53.912 [INFO][9628] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" iface="eth0" netns="/var/run/netns/cni-caa96571-426e-0e19-adb8-4c41e7018c00" Aug 13 01:29:53.996774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.996774 containerd[1567]: 2025-08-13 01:29:53.912 [INFO][9628] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" iface="eth0" netns="/var/run/netns/cni-caa96571-426e-0e19-adb8-4c41e7018c00" Aug 13 01:29:53.996774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.996774 containerd[1567]: 2025-08-13 01:29:53.920 [INFO][9628] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" after=8.160814ms iface="eth0" netns="/var/run/netns/cni-caa96571-426e-0e19-adb8-4c41e7018c00" Aug 13 01:29:53.996774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.996774 containerd[1567]: 2025-08-13 01:29:53.920 [INFO][9628] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Aug 13 01:29:53.996774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.996774 containerd[1567]: 2025-08-13 01:29:53.920 [INFO][9628] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Aug 13 01:29:53.996774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.996774 containerd[1567]: 2025-08-13 01:29:53.952 [INFO][9637] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" HandleID="k8s-pod-network.30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:53.996774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.996774 containerd[1567]: 2025-08-13 01:29:53.953 [INFO][9637] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:53.996774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.996774 containerd[1567]: 2025-08-13 01:29:53.953 [INFO][9637] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:53.996774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.996774 containerd[1567]: 2025-08-13 01:29:53.988 [INFO][9637] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" HandleID="k8s-pod-network.30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:53.996774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.996774 containerd[1567]: 2025-08-13 01:29:53.988 [INFO][9637] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" HandleID="k8s-pod-network.30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:53.996774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.996774 containerd[1567]: 2025-08-13 01:29:53.989 [INFO][9637] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:53.996774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.996774 containerd[1567]: 2025-08-13 01:29:53.992 [INFO][9628] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57" Aug 13 01:29:53.996774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:53.999420 containerd[1567]: time="2025-08-13T01:29:53.999092981Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57-lo: no space left on device" Aug 13 01:29:54.000046 kubelet[2720]: E0813 01:29:53.999915 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57-lo: no space left on device" Aug 13 01:29:54.000046 kubelet[2720]: E0813 01:29:53.999984 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:54.000046 kubelet[2720]: E0813 01:29:54.000004 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:54.000412 kubelet[2720]: E0813 01:29:54.000049 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:54.001185 systemd[1]: run-netns-cni\x2dcaa96571\x2d426e\x2d0e19\x2dadb8\x2d4c41e7018c00.mount: Deactivated successfully. Aug 13 01:29:54.357129 containerd[1567]: time="2025-08-13T01:29:54.357012112Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:54.722936 containerd[1567]: time="2025-08-13T01:29:54.722880779Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:54.824628 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:54.825877 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.757 [INFO][9647] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15420 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.757 [INFO][9647] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.781 [INFO][9658] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" HandleID="k8s-pod-network.7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.781 [INFO][9658] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" HandleID="k8s-pod-network.7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:54.78110392 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.781 [INFO][9658] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.781 [INFO][9658] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.781 [INFO][9658] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.787 [INFO][9658] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" host="172-234-25-215" Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.792 [INFO][9658] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.796 [INFO][9658] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.798 [INFO][9658] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.802 [INFO][9658] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.803 [INFO][9658] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" host="172-234-25-215" Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.807 [INFO][9658] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6 Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.812 [INFO][9658] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" host="172-234-25-215" Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.818 [INFO][9658] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.62/26] block=192.168.62.0/26 handle="k8s-pod-network.7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" host="172-234-25-215" Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.848141 containerd[1567]: 2025-08-13 01:29:54.818 [INFO][9658] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.62/26] handle="k8s-pod-network.7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" host="172-234-25-215" Aug 13 01:29:54.848141 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.849965 containerd[1567]: 2025-08-13 01:29:54.818 [INFO][9658] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:54.849965 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.849965 containerd[1567]: 2025-08-13 01:29:54.818 [INFO][9658] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.62/26] IPv6=[] ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" HandleID="k8s-pod-network.7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:54.849965 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.849965 containerd[1567]: 2025-08-13 01:29:54.821 [INFO][9647] cni-plugin/k8s.go 418: Populated endpoint ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15420", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"30f8b0612d0ed2f84097a3577ab63b1a88a95b3fbfe544aadfdf99b0cb316b57", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.62/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:54.849965 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.849965 containerd[1567]: 2025-08-13 01:29:54.821 [INFO][9647] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.62/32] ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:54.849965 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.849965 containerd[1567]: 2025-08-13 01:29:54.821 [INFO][9647] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:54.849965 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.849965 containerd[1567]: 2025-08-13 01:29:54.825 [INFO][9647] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:54.849965 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.849965 containerd[1567]: 2025-08-13 01:29:54.826 [INFO][9647] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15420", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.62/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"96:54:f2:6c:db:b5", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:54.849965 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.851026 containerd[1567]: 2025-08-13 01:29:54.838 [INFO][9647] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:54.851026 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:54.916306 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:54.916322 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:55.027126 containerd[1567]: 2025-08-13 01:29:54.914 [INFO][9682] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Aug 13 01:29:55.027126 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.027126 containerd[1567]: 2025-08-13 01:29:54.915 [INFO][9682] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" iface="eth0" netns="/var/run/netns/cni-fd22e9ba-a87a-4fc4-5126-9d3c23113ba1" Aug 13 01:29:55.027126 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.027126 containerd[1567]: 2025-08-13 01:29:54.915 [INFO][9682] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" iface="eth0" netns="/var/run/netns/cni-fd22e9ba-a87a-4fc4-5126-9d3c23113ba1" Aug 13 01:29:55.027126 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.027126 containerd[1567]: 2025-08-13 01:29:54.921 [INFO][9682] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" after=5.880049ms iface="eth0" netns="/var/run/netns/cni-fd22e9ba-a87a-4fc4-5126-9d3c23113ba1" Aug 13 01:29:55.027126 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.027126 containerd[1567]: 2025-08-13 01:29:54.921 [INFO][9682] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Aug 13 01:29:55.027126 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.027126 containerd[1567]: 2025-08-13 01:29:54.921 [INFO][9682] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Aug 13 01:29:55.027126 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.027126 containerd[1567]: 2025-08-13 01:29:54.964 [INFO][9691] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" HandleID="k8s-pod-network.7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:55.027126 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.027126 containerd[1567]: 2025-08-13 01:29:54.964 [INFO][9691] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:55.027126 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.027126 containerd[1567]: 2025-08-13 01:29:54.965 [INFO][9691] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:55.027126 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.027126 containerd[1567]: 2025-08-13 01:29:55.020 [INFO][9691] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" HandleID="k8s-pod-network.7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:55.027126 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.027126 containerd[1567]: 2025-08-13 01:29:55.021 [INFO][9691] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" HandleID="k8s-pod-network.7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:55.027126 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.027126 containerd[1567]: 2025-08-13 01:29:55.022 [INFO][9691] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:55.027126 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.027126 containerd[1567]: 2025-08-13 01:29:55.024 [INFO][9682] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6" Aug 13 01:29:55.027126 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.030937 containerd[1567]: time="2025-08-13T01:29:55.030499124Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6-lo: no space left on device" Aug 13 01:29:55.031218 kubelet[2720]: E0813 01:29:55.031161 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6-lo: no space left on device" Aug 13 01:29:55.031484 kubelet[2720]: E0813 01:29:55.031219 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:55.031484 kubelet[2720]: E0813 01:29:55.031239 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:55.031484 kubelet[2720]: E0813 01:29:55.031292 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:55.032480 systemd[1]: run-netns-cni\x2dfd22e9ba\x2da87a\x2d4fc4\x2d5126\x2d9d3c23113ba1.mount: Deactivated successfully. Aug 13 01:29:55.725197 containerd[1567]: time="2025-08-13T01:29:55.725135108Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:55.833034 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:55.837832 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.764 [INFO][9699] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15490 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.764 [INFO][9699] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.791 [INFO][9712] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" HandleID="k8s-pod-network.b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.791 [INFO][9712] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" HandleID="k8s-pod-network.b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5010), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:55.791240664 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.791 [INFO][9712] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.791 [INFO][9712] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.791 [INFO][9712] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.798 [INFO][9712] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" host="172-234-25-215" Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.804 [INFO][9712] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.811 [INFO][9712] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.813 [INFO][9712] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.815 [INFO][9712] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.815 [INFO][9712] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" host="172-234-25-215" Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.816 [INFO][9712] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894 Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.819 [INFO][9712] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" host="172-234-25-215" Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.826 [INFO][9712] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.63/26] block=192.168.62.0/26 handle="k8s-pod-network.b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" host="172-234-25-215" Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.860614 containerd[1567]: 2025-08-13 01:29:55.826 [INFO][9712] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.63/26] handle="k8s-pod-network.b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" host="172-234-25-215" Aug 13 01:29:55.860614 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.862986 containerd[1567]: 2025-08-13 01:29:55.826 [INFO][9712] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:55.862986 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.862986 containerd[1567]: 2025-08-13 01:29:55.826 [INFO][9712] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.63/26] IPv6=[] ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" HandleID="k8s-pod-network.b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:55.862986 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.862986 containerd[1567]: 2025-08-13 01:29:55.829 [INFO][9699] cni-plugin/k8s.go 418: Populated endpoint ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15490", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"7fdb0abc466d7378fc3dd01ca30239f0db4c11fe306f6c977c19f0bec6f079c6", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.63/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:55.862986 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.862986 containerd[1567]: 2025-08-13 01:29:55.829 [INFO][9699] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.63/32] ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:55.862986 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.862986 containerd[1567]: 2025-08-13 01:29:55.829 [INFO][9699] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:55.862986 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.862986 containerd[1567]: 2025-08-13 01:29:55.836 [INFO][9699] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:55.862986 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.862986 containerd[1567]: 2025-08-13 01:29:55.840 [INFO][9699] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15490", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.63/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"0a:fb:8b:d7:1e:5c", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:55.862986 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.863631 containerd[1567]: 2025-08-13 01:29:55.855 [INFO][9699] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:55.863631 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:55.933137 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:55.933147 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:56.026773 containerd[1567]: 2025-08-13 01:29:55.924 [INFO][9735] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Aug 13 01:29:56.026773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.026773 containerd[1567]: 2025-08-13 01:29:55.925 [INFO][9735] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" iface="eth0" netns="/var/run/netns/cni-0ef97b1f-bac4-cb74-3779-dc7c6cf50f23" Aug 13 01:29:56.026773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.026773 containerd[1567]: 2025-08-13 01:29:55.926 [INFO][9735] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" iface="eth0" netns="/var/run/netns/cni-0ef97b1f-bac4-cb74-3779-dc7c6cf50f23" Aug 13 01:29:56.026773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.026773 containerd[1567]: 2025-08-13 01:29:55.940 [INFO][9735] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" after=14.146364ms iface="eth0" netns="/var/run/netns/cni-0ef97b1f-bac4-cb74-3779-dc7c6cf50f23" Aug 13 01:29:56.026773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.026773 containerd[1567]: 2025-08-13 01:29:55.940 [INFO][9735] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Aug 13 01:29:56.026773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.026773 containerd[1567]: 2025-08-13 01:29:55.940 [INFO][9735] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Aug 13 01:29:56.026773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.026773 containerd[1567]: 2025-08-13 01:29:55.975 [INFO][9744] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" HandleID="k8s-pod-network.b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:56.026773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.026773 containerd[1567]: 2025-08-13 01:29:55.976 [INFO][9744] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:56.026773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.026773 containerd[1567]: 2025-08-13 01:29:55.976 [INFO][9744] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:56.026773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.026773 containerd[1567]: 2025-08-13 01:29:56.018 [INFO][9744] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" HandleID="k8s-pod-network.b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:56.026773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.026773 containerd[1567]: 2025-08-13 01:29:56.018 [INFO][9744] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" HandleID="k8s-pod-network.b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:56.026773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.026773 containerd[1567]: 2025-08-13 01:29:56.019 [INFO][9744] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:56.026773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.026773 containerd[1567]: 2025-08-13 01:29:56.021 [INFO][9735] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894" Aug 13 01:29:56.026773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.026615 systemd[1]: run-netns-cni\x2d0ef97b1f\x2dbac4\x2dcb74\x2d3779\x2ddc7c6cf50f23.mount: Deactivated successfully. Aug 13 01:29:56.028647 containerd[1567]: time="2025-08-13T01:29:56.028413652Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894-lo: no space left on device" Aug 13 01:29:56.029833 kubelet[2720]: E0813 01:29:56.029787 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894-lo: no space left on device" Aug 13 01:29:56.029880 kubelet[2720]: E0813 01:29:56.029852 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:56.029880 kubelet[2720]: E0813 01:29:56.029875 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:56.029971 kubelet[2720]: E0813 01:29:56.029943 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:56.727006 containerd[1567]: time="2025-08-13T01:29:56.726966249Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:56.829783 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:56.830802 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.762 [INFO][9754] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15551 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.762 [INFO][9754] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.788 [INFO][9767] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" HandleID="k8s-pod-network.c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.788 [INFO][9767] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" HandleID="k8s-pod-network.c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002ac5b0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:56.788124704 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.788 [INFO][9767] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.788 [INFO][9767] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.788 [INFO][9767] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.794 [INFO][9767] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" host="172-234-25-215" Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.801 [INFO][9767] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.806 [INFO][9767] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.808 [INFO][9767] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.811 [INFO][9767] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.811 [INFO][9767] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" host="172-234-25-215" Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.813 [INFO][9767] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2 Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.817 [INFO][9767] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" host="172-234-25-215" Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.823 [INFO][9767] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.4/26] block=192.168.62.0/26 handle="k8s-pod-network.c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" host="172-234-25-215" Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.846893 containerd[1567]: 2025-08-13 01:29:56.823 [INFO][9767] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.4/26] handle="k8s-pod-network.c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" host="172-234-25-215" Aug 13 01:29:56.846893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.849148 containerd[1567]: 2025-08-13 01:29:56.823 [INFO][9767] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:56.849148 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.849148 containerd[1567]: 2025-08-13 01:29:56.823 [INFO][9767] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.4/26] IPv6=[] ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" HandleID="k8s-pod-network.c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:56.849148 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.849148 containerd[1567]: 2025-08-13 01:29:56.826 [INFO][9754] cni-plugin/k8s.go 418: Populated endpoint ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15551", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"b7a5233cfd3b06636b2692aed41225f921921784fc11df04e495eda2b3b63894", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.4/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:56.849148 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.849148 containerd[1567]: 2025-08-13 01:29:56.826 [INFO][9754] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.4/32] ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:56.849148 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.849148 containerd[1567]: 2025-08-13 01:29:56.826 [INFO][9754] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:56.849148 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.849148 containerd[1567]: 2025-08-13 01:29:56.830 [INFO][9754] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:56.849148 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.849148 containerd[1567]: 2025-08-13 01:29:56.831 [INFO][9754] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15551", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.4/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"fe:50:f8:ad:94:5f", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:56.849148 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.850241 containerd[1567]: 2025-08-13 01:29:56.840 [INFO][9754] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:56.850241 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:56.920283 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:56.920295 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:57.021774 containerd[1567]: 2025-08-13 01:29:56.915 [INFO][9791] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Aug 13 01:29:57.021774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.021774 containerd[1567]: 2025-08-13 01:29:56.916 [INFO][9791] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" iface="eth0" netns="/var/run/netns/cni-6b61b2fe-f384-121a-04cd-b37013258b34" Aug 13 01:29:57.021774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.021774 containerd[1567]: 2025-08-13 01:29:56.917 [INFO][9791] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" iface="eth0" netns="/var/run/netns/cni-6b61b2fe-f384-121a-04cd-b37013258b34" Aug 13 01:29:57.021774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.021774 containerd[1567]: 2025-08-13 01:29:56.924 [INFO][9791] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" after=8.369035ms iface="eth0" netns="/var/run/netns/cni-6b61b2fe-f384-121a-04cd-b37013258b34" Aug 13 01:29:57.021774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.021774 containerd[1567]: 2025-08-13 01:29:56.924 [INFO][9791] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Aug 13 01:29:57.021774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.021774 containerd[1567]: 2025-08-13 01:29:56.925 [INFO][9791] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Aug 13 01:29:57.021774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.021774 containerd[1567]: 2025-08-13 01:29:56.953 [INFO][9801] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" HandleID="k8s-pod-network.c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:57.021774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.021774 containerd[1567]: 2025-08-13 01:29:56.954 [INFO][9801] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:57.021774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.021774 containerd[1567]: 2025-08-13 01:29:56.954 [INFO][9801] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:57.021774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.021774 containerd[1567]: 2025-08-13 01:29:57.015 [INFO][9801] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" HandleID="k8s-pod-network.c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:57.021774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.021774 containerd[1567]: 2025-08-13 01:29:57.015 [INFO][9801] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" HandleID="k8s-pod-network.c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:57.021774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.021774 containerd[1567]: 2025-08-13 01:29:57.016 [INFO][9801] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:57.021774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.021774 containerd[1567]: 2025-08-13 01:29:57.019 [INFO][9791] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2" Aug 13 01:29:57.021774 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.026121 containerd[1567]: time="2025-08-13T01:29:57.026059710Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2-lo: no space left on device" Aug 13 01:29:57.026682 kubelet[2720]: E0813 01:29:57.026383 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2-lo: no space left on device" Aug 13 01:29:57.026682 kubelet[2720]: E0813 01:29:57.026437 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:57.026682 kubelet[2720]: E0813 01:29:57.026456 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:57.026682 kubelet[2720]: E0813 01:29:57.026504 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:57.027175 systemd[1]: run-netns-cni\x2d6b61b2fe\x2df384\x2d121a\x2d04cd\x2db37013258b34.mount: Deactivated successfully. Aug 13 01:29:57.356818 containerd[1567]: time="2025-08-13T01:29:57.356707903Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:29:57.728876 containerd[1567]: time="2025-08-13T01:29:57.728798039Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:57.844851 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:57.845526 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.773 [INFO][9811] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15622 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.773 [INFO][9811] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.799 [INFO][9821] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" HandleID="k8s-pod-network.ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.800 [INFO][9821] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" HandleID="k8s-pod-network.ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f210), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:57.799829307 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.800 [INFO][9821] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.801 [INFO][9821] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.801 [INFO][9821] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.809 [INFO][9821] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" host="172-234-25-215" Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.813 [INFO][9821] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.817 [INFO][9821] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.819 [INFO][9821] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.821 [INFO][9821] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.821 [INFO][9821] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" host="172-234-25-215" Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.823 [INFO][9821] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3 Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.827 [INFO][9821] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" host="172-234-25-215" Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.835 [INFO][9821] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.5/26] block=192.168.62.0/26 handle="k8s-pod-network.ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" host="172-234-25-215" Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.862702 containerd[1567]: 2025-08-13 01:29:57.836 [INFO][9821] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.5/26] handle="k8s-pod-network.ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" host="172-234-25-215" Aug 13 01:29:57.862702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.863592 containerd[1567]: 2025-08-13 01:29:57.836 [INFO][9821] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:57.863592 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.863592 containerd[1567]: 2025-08-13 01:29:57.836 [INFO][9821] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.5/26] IPv6=[] ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" HandleID="k8s-pod-network.ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:57.863592 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.863592 containerd[1567]: 2025-08-13 01:29:57.840 [INFO][9811] cni-plugin/k8s.go 418: Populated endpoint ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15622", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c6c8f87d51c5d412c439baf51d93476a7f612ece39d9ee293cae83cc779ff8d2", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.5/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:57.863592 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.863592 containerd[1567]: 2025-08-13 01:29:57.840 [INFO][9811] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.5/32] ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:57.863592 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.863592 containerd[1567]: 2025-08-13 01:29:57.840 [INFO][9811] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:57.863592 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.863592 containerd[1567]: 2025-08-13 01:29:57.845 [INFO][9811] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:57.863592 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.863592 containerd[1567]: 2025-08-13 01:29:57.846 [INFO][9811] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15622", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.5/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"06:3f:96:b0:07:df", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:57.863592 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.867531 containerd[1567]: 2025-08-13 01:29:57.857 [INFO][9811] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:57.867531 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:57.926546 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:57.926559 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:58.037249 containerd[1567]: 2025-08-13 01:29:57.923 [INFO][9846] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Aug 13 01:29:58.037249 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.037249 containerd[1567]: 2025-08-13 01:29:57.924 [INFO][9846] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" iface="eth0" netns="/var/run/netns/cni-2a3e8e10-80ef-3c7a-19ae-30760f505d41" Aug 13 01:29:58.037249 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.037249 containerd[1567]: 2025-08-13 01:29:57.924 [INFO][9846] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" iface="eth0" netns="/var/run/netns/cni-2a3e8e10-80ef-3c7a-19ae-30760f505d41" Aug 13 01:29:58.037249 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.037249 containerd[1567]: 2025-08-13 01:29:57.931 [INFO][9846] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" after=6.669117ms iface="eth0" netns="/var/run/netns/cni-2a3e8e10-80ef-3c7a-19ae-30760f505d41" Aug 13 01:29:58.037249 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.037249 containerd[1567]: 2025-08-13 01:29:57.931 [INFO][9846] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Aug 13 01:29:58.037249 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.037249 containerd[1567]: 2025-08-13 01:29:57.931 [INFO][9846] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Aug 13 01:29:58.037249 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.037249 containerd[1567]: 2025-08-13 01:29:57.993 [INFO][9856] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" HandleID="k8s-pod-network.ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:58.037249 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.037249 containerd[1567]: 2025-08-13 01:29:57.993 [INFO][9856] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:58.037249 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.037249 containerd[1567]: 2025-08-13 01:29:57.993 [INFO][9856] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:58.037249 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.037249 containerd[1567]: 2025-08-13 01:29:58.029 [INFO][9856] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" HandleID="k8s-pod-network.ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:58.037249 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.037249 containerd[1567]: 2025-08-13 01:29:58.029 [INFO][9856] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" HandleID="k8s-pod-network.ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:58.037249 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.037249 containerd[1567]: 2025-08-13 01:29:58.030 [INFO][9856] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:58.037249 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.037249 containerd[1567]: 2025-08-13 01:29:58.033 [INFO][9846] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3" Aug 13 01:29:58.037249 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.040628 systemd[1]: run-netns-cni\x2d2a3e8e10\x2d80ef\x2d3c7a\x2d19ae\x2d30760f505d41.mount: Deactivated successfully. Aug 13 01:29:58.042933 containerd[1567]: time="2025-08-13T01:29:58.042812893Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3-lo: no space left on device" Aug 13 01:29:58.044440 kubelet[2720]: E0813 01:29:58.043244 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3-lo: no space left on device" Aug 13 01:29:58.044440 kubelet[2720]: E0813 01:29:58.043368 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:58.044440 kubelet[2720]: E0813 01:29:58.043390 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:58.044440 kubelet[2720]: E0813 01:29:58.043493 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:58.256605 kubelet[2720]: I0813 01:29:58.256566 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:29:58.256605 kubelet[2720]: I0813 01:29:58.256612 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:29:58.259294 kubelet[2720]: I0813 01:29:58.259276 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:29:58.275010 kubelet[2720]: I0813 01:29:58.274989 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:29:58.275123 kubelet[2720]: I0813 01:29:58.275102 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-kube-controllers-b57d47f84-lznkx","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:29:58.275247 kubelet[2720]: E0813 01:29:58.275147 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:29:58.275247 kubelet[2720]: E0813 01:29:58.275157 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:29:58.275247 kubelet[2720]: E0813 01:29:58.275164 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:58.275247 kubelet[2720]: E0813 01:29:58.275207 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:29:58.275247 kubelet[2720]: E0813 01:29:58.275217 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:29:58.275247 kubelet[2720]: E0813 01:29:58.275226 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:29:58.275247 kubelet[2720]: E0813 01:29:58.275233 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:29:58.275247 kubelet[2720]: E0813 01:29:58.275243 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:29:58.275247 kubelet[2720]: E0813 01:29:58.275251 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:29:58.275426 kubelet[2720]: E0813 01:29:58.275259 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:29:58.275426 kubelet[2720]: I0813 01:29:58.275268 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:29:58.492500 systemd[1]: Started sshd@36-172.234.25.215:22-147.75.109.163:35056.service - OpenSSH per-connection server daemon (147.75.109.163:35056). Aug 13 01:29:58.730931 containerd[1567]: time="2025-08-13T01:29:58.730891221Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:58.837371 sshd[9866]: Accepted publickey for core from 147.75.109.163 port 35056 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:29:58.842250 sshd-session[9866]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:29:58.843332 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:58.845234 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:58.858130 systemd-logind[1543]: New session 37 of user core. Aug 13 01:29:58.864025 systemd[1]: Started session-37.scope - Session 37 of User core. Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.769 [INFO][9870] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15694 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.770 [INFO][9870] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.797 [INFO][9883] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" HandleID="k8s-pod-network.ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.797 [INFO][9883] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" HandleID="k8s-pod-network.ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d51e0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:58.797325747 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.797 [INFO][9883] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.797 [INFO][9883] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.797 [INFO][9883] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.808 [INFO][9883] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" host="172-234-25-215" Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.812 [INFO][9883] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.816 [INFO][9883] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.818 [INFO][9883] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.820 [INFO][9883] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.820 [INFO][9883] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" host="172-234-25-215" Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.821 [INFO][9883] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0 Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.825 [INFO][9883] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" host="172-234-25-215" Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.831 [INFO][9883] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.6/26] block=192.168.62.0/26 handle="k8s-pod-network.ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" host="172-234-25-215" Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.877907 containerd[1567]: 2025-08-13 01:29:58.831 [INFO][9883] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.6/26] handle="k8s-pod-network.ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" host="172-234-25-215" Aug 13 01:29:58.877907 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.879568 containerd[1567]: 2025-08-13 01:29:58.831 [INFO][9883] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:58.879568 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.879568 containerd[1567]: 2025-08-13 01:29:58.832 [INFO][9883] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.6/26] IPv6=[] ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" HandleID="k8s-pod-network.ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:58.879568 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.879568 containerd[1567]: 2025-08-13 01:29:58.836 [INFO][9870] cni-plugin/k8s.go 418: Populated endpoint ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15694", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ef73f311923f2cdb28132548b3c9219262c801f99216b591a6b1009cc50ebce3", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.6/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:58.879568 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.879568 containerd[1567]: 2025-08-13 01:29:58.837 [INFO][9870] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.6/32] ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:58.879568 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.879568 containerd[1567]: 2025-08-13 01:29:58.837 [INFO][9870] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:58.879568 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.879568 containerd[1567]: 2025-08-13 01:29:58.844 [INFO][9870] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:58.879568 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.879568 containerd[1567]: 2025-08-13 01:29:58.845 [INFO][9870] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15694", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.6/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"0a:0d:2e:f4:df:f2", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:58.879568 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.880253 containerd[1567]: 2025-08-13 01:29:58.871 [INFO][9870] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:58.880253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:58.945755 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:58.945768 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:29:59.057576 containerd[1567]: 2025-08-13 01:29:58.942 [INFO][9908] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Aug 13 01:29:59.057576 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.057576 containerd[1567]: 2025-08-13 01:29:58.943 [INFO][9908] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" iface="eth0" netns="/var/run/netns/cni-787d1606-c832-807b-72b5-c2091f483226" Aug 13 01:29:59.057576 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.057576 containerd[1567]: 2025-08-13 01:29:58.944 [INFO][9908] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" iface="eth0" netns="/var/run/netns/cni-787d1606-c832-807b-72b5-c2091f483226" Aug 13 01:29:59.057576 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.057576 containerd[1567]: 2025-08-13 01:29:58.952 [INFO][9908] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" after=8.901303ms iface="eth0" netns="/var/run/netns/cni-787d1606-c832-807b-72b5-c2091f483226" Aug 13 01:29:59.057576 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.057576 containerd[1567]: 2025-08-13 01:29:58.952 [INFO][9908] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Aug 13 01:29:59.057576 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.057576 containerd[1567]: 2025-08-13 01:29:58.952 [INFO][9908] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Aug 13 01:29:59.057576 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.057576 containerd[1567]: 2025-08-13 01:29:58.985 [INFO][9917] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" HandleID="k8s-pod-network.ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:59.057576 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.057576 containerd[1567]: 2025-08-13 01:29:58.987 [INFO][9917] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:59.057576 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.057576 containerd[1567]: 2025-08-13 01:29:58.987 [INFO][9917] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:59.057576 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.057576 containerd[1567]: 2025-08-13 01:29:59.041 [INFO][9917] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" HandleID="k8s-pod-network.ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:59.057576 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.057576 containerd[1567]: 2025-08-13 01:29:59.041 [INFO][9917] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" HandleID="k8s-pod-network.ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:59.057576 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.057576 containerd[1567]: 2025-08-13 01:29:59.042 [INFO][9917] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:59.057576 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.057576 containerd[1567]: 2025-08-13 01:29:59.050 [INFO][9908] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0" Aug 13 01:29:59.057576 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.062182 systemd[1]: run-netns-cni\x2d787d1606\x2dc832\x2d807b\x2d72b5\x2dc2091f483226.mount: Deactivated successfully. Aug 13 01:29:59.064149 containerd[1567]: time="2025-08-13T01:29:59.064025310Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0-lo: no space left on device" Aug 13 01:29:59.065254 kubelet[2720]: E0813 01:29:59.064811 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0-lo: no space left on device" Aug 13 01:29:59.065254 kubelet[2720]: E0813 01:29:59.064903 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:59.065254 kubelet[2720]: E0813 01:29:59.064943 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:29:59.065254 kubelet[2720]: E0813 01:29:59.065020 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:29:59.166032 sshd[9891]: Connection closed by 147.75.109.163 port 35056 Aug 13 01:29:59.167106 sshd-session[9866]: pam_unix(sshd:session): session closed for user core Aug 13 01:29:59.172651 systemd[1]: sshd@36-172.234.25.215:22-147.75.109.163:35056.service: Deactivated successfully. Aug 13 01:29:59.175079 systemd[1]: session-37.scope: Deactivated successfully. Aug 13 01:29:59.175990 systemd-logind[1543]: Session 37 logged out. Waiting for processes to exit. Aug 13 01:29:59.178481 systemd-logind[1543]: Removed session 37. Aug 13 01:29:59.214186 containerd[1567]: time="2025-08-13T01:29:59.214133830Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:29:59.733143 containerd[1567]: time="2025-08-13T01:29:59.733100874Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:29:59.843063 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:29:59.843824 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.770 [INFO][9940] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15769 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.773 [INFO][9940] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.797 [INFO][9950] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" HandleID="k8s-pod-network.d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.798 [INFO][9950] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" HandleID="k8s-pod-network.d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5640), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:29:59.797910913 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.798 [INFO][9950] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.798 [INFO][9950] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.798 [INFO][9950] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.807 [INFO][9950] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" host="172-234-25-215" Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.813 [INFO][9950] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.818 [INFO][9950] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.819 [INFO][9950] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.822 [INFO][9950] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.822 [INFO][9950] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" host="172-234-25-215" Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.824 [INFO][9950] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029 Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.828 [INFO][9950] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" host="172-234-25-215" Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.833 [INFO][9950] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.7/26] block=192.168.62.0/26 handle="k8s-pod-network.d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" host="172-234-25-215" Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.872263 containerd[1567]: 2025-08-13 01:29:59.833 [INFO][9950] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.7/26] handle="k8s-pod-network.d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" host="172-234-25-215" Aug 13 01:29:59.872263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.876095 containerd[1567]: 2025-08-13 01:29:59.834 [INFO][9950] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:29:59.876095 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.876095 containerd[1567]: 2025-08-13 01:29:59.834 [INFO][9950] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.7/26] IPv6=[] ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" HandleID="k8s-pod-network.d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:59.876095 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.876095 containerd[1567]: 2025-08-13 01:29:59.837 [INFO][9940] cni-plugin/k8s.go 418: Populated endpoint ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15769", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ea28239a131f2da7d53e529d088610faabf3ac309cc18689e6124ffcd5224cd0", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.7/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:59.876095 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.876095 containerd[1567]: 2025-08-13 01:29:59.837 [INFO][9940] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.7/32] ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:59.876095 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.876095 containerd[1567]: 2025-08-13 01:29:59.837 [INFO][9940] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:59.876095 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.876095 containerd[1567]: 2025-08-13 01:29:59.844 [INFO][9940] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:59.876095 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.876095 containerd[1567]: 2025-08-13 01:29:59.845 [INFO][9940] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15769", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.7/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"22:ce:27:85:37:cd", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:29:59.876095 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.876407 containerd[1567]: 2025-08-13 01:29:59.862 [INFO][9940] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:29:59.876407 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:29:59.970345 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:29:59.970354 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:00.097783 containerd[1567]: 2025-08-13 01:29:59.966 [INFO][9973] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Aug 13 01:30:00.097783 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.097783 containerd[1567]: 2025-08-13 01:29:59.968 [INFO][9973] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" iface="eth0" netns="/var/run/netns/cni-c79f3fc6-cb09-56a6-a2a1-9ec46bea2356" Aug 13 01:30:00.097783 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.097783 containerd[1567]: 2025-08-13 01:29:59.968 [INFO][9973] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" iface="eth0" netns="/var/run/netns/cni-c79f3fc6-cb09-56a6-a2a1-9ec46bea2356" Aug 13 01:30:00.097783 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.097783 containerd[1567]: 2025-08-13 01:29:59.980 [INFO][9973] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" after=12.189167ms iface="eth0" netns="/var/run/netns/cni-c79f3fc6-cb09-56a6-a2a1-9ec46bea2356" Aug 13 01:30:00.097783 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.097783 containerd[1567]: 2025-08-13 01:29:59.980 [INFO][9973] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Aug 13 01:30:00.097783 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.097783 containerd[1567]: 2025-08-13 01:29:59.981 [INFO][9973] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Aug 13 01:30:00.097783 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.097783 containerd[1567]: 2025-08-13 01:30:00.016 [INFO][9982] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" HandleID="k8s-pod-network.d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:00.097783 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.097783 containerd[1567]: 2025-08-13 01:30:00.017 [INFO][9982] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:00.097783 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.097783 containerd[1567]: 2025-08-13 01:30:00.017 [INFO][9982] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:00.097783 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.097783 containerd[1567]: 2025-08-13 01:30:00.088 [INFO][9982] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" HandleID="k8s-pod-network.d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:00.097783 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.097783 containerd[1567]: 2025-08-13 01:30:00.088 [INFO][9982] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" HandleID="k8s-pod-network.d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:00.097783 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.097783 containerd[1567]: 2025-08-13 01:30:00.090 [INFO][9982] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:00.097783 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.097783 containerd[1567]: 2025-08-13 01:30:00.094 [INFO][9973] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029" Aug 13 01:30:00.097783 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.104204 containerd[1567]: time="2025-08-13T01:30:00.104156842Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029-lo: no space left on device" Aug 13 01:30:00.104597 kubelet[2720]: E0813 01:30:00.104529 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029-lo: no space left on device" Aug 13 01:30:00.104597 kubelet[2720]: E0813 01:30:00.104581 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:00.104914 kubelet[2720]: E0813 01:30:00.104601 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:00.104914 kubelet[2720]: E0813 01:30:00.104644 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:00.106389 systemd[1]: run-netns-cni\x2dc79f3fc6\x2dcb09\x2d56a6\x2da2a1\x2d9ec46bea2356.mount: Deactivated successfully. Aug 13 01:30:00.357019 containerd[1567]: time="2025-08-13T01:30:00.356894741Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:00.734249 containerd[1567]: time="2025-08-13T01:30:00.733932090Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:00.834307 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:00.835156 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.769 [INFO][9993] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15830 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.769 [INFO][9993] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.794 [INFO][10005] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" HandleID="k8s-pod-network.7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.794 [INFO][10005] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" HandleID="k8s-pod-network.7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f100), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:00.794473127 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.794 [INFO][10005] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.794 [INFO][10005] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.794 [INFO][10005] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.803 [INFO][10005] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" host="172-234-25-215" Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.807 [INFO][10005] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.811 [INFO][10005] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.813 [INFO][10005] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.816 [INFO][10005] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.816 [INFO][10005] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" host="172-234-25-215" Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.817 [INFO][10005] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689 Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.821 [INFO][10005] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" host="172-234-25-215" Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.827 [INFO][10005] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.8/26] block=192.168.62.0/26 handle="k8s-pod-network.7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" host="172-234-25-215" Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.850096 containerd[1567]: 2025-08-13 01:30:00.827 [INFO][10005] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.8/26] handle="k8s-pod-network.7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" host="172-234-25-215" Aug 13 01:30:00.850096 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.852492 containerd[1567]: 2025-08-13 01:30:00.827 [INFO][10005] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:00.852492 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.852492 containerd[1567]: 2025-08-13 01:30:00.828 [INFO][10005] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.8/26] IPv6=[] ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" HandleID="k8s-pod-network.7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:00.852492 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.852492 containerd[1567]: 2025-08-13 01:30:00.830 [INFO][9993] cni-plugin/k8s.go 418: Populated endpoint ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15830", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"d0282a058716529029f21559f3635445d5f390c8a7ee82fd4de4a968af8ea029", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.8/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:00.852492 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.852492 containerd[1567]: 2025-08-13 01:30:00.830 [INFO][9993] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.8/32] ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:00.852492 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.852492 containerd[1567]: 2025-08-13 01:30:00.830 [INFO][9993] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:00.852492 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.852492 containerd[1567]: 2025-08-13 01:30:00.834 [INFO][9993] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:00.852492 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.852492 containerd[1567]: 2025-08-13 01:30:00.834 [INFO][9993] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15830", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.8/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"32:01:8a:48:aa:6b", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:00.852492 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.853099 containerd[1567]: 2025-08-13 01:30:00.845 [INFO][9993] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:00.853099 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:00.914874 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:00.914881 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:01.015091 containerd[1567]: 2025-08-13 01:30:00.912 [INFO][10028] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Aug 13 01:30:01.015091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.015091 containerd[1567]: 2025-08-13 01:30:00.912 [INFO][10028] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" iface="eth0" netns="/var/run/netns/cni-78c46f2f-c854-1946-bccc-42834f35f2a4" Aug 13 01:30:01.015091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.015091 containerd[1567]: 2025-08-13 01:30:00.912 [INFO][10028] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" iface="eth0" netns="/var/run/netns/cni-78c46f2f-c854-1946-bccc-42834f35f2a4" Aug 13 01:30:01.015091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.015091 containerd[1567]: 2025-08-13 01:30:00.920 [INFO][10028] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" after=7.374206ms iface="eth0" netns="/var/run/netns/cni-78c46f2f-c854-1946-bccc-42834f35f2a4" Aug 13 01:30:01.015091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.015091 containerd[1567]: 2025-08-13 01:30:00.920 [INFO][10028] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Aug 13 01:30:01.015091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.015091 containerd[1567]: 2025-08-13 01:30:00.920 [INFO][10028] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Aug 13 01:30:01.015091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.015091 containerd[1567]: 2025-08-13 01:30:00.951 [INFO][10038] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" HandleID="k8s-pod-network.7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:01.015091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.015091 containerd[1567]: 2025-08-13 01:30:00.951 [INFO][10038] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:01.015091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.015091 containerd[1567]: 2025-08-13 01:30:00.952 [INFO][10038] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:01.015091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.015091 containerd[1567]: 2025-08-13 01:30:01.007 [INFO][10038] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" HandleID="k8s-pod-network.7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:01.015091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.015091 containerd[1567]: 2025-08-13 01:30:01.008 [INFO][10038] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" HandleID="k8s-pod-network.7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:01.015091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.015091 containerd[1567]: 2025-08-13 01:30:01.009 [INFO][10038] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:01.015091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.015091 containerd[1567]: 2025-08-13 01:30:01.012 [INFO][10028] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689" Aug 13 01:30:01.015091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.018908 containerd[1567]: time="2025-08-13T01:30:01.018832669Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689-lo: no space left on device" Aug 13 01:30:01.019251 kubelet[2720]: E0813 01:30:01.019210 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689-lo: no space left on device" Aug 13 01:30:01.019308 kubelet[2720]: E0813 01:30:01.019270 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:01.019308 kubelet[2720]: E0813 01:30:01.019289 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:01.019357 kubelet[2720]: E0813 01:30:01.019332 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:01.019859 systemd[1]: run-netns-cni\x2d78c46f2f\x2dc854\x2d1946\x2dbccc\x2d42834f35f2a4.mount: Deactivated successfully. Aug 13 01:30:01.736250 containerd[1567]: time="2025-08-13T01:30:01.736183523Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:01.863918 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:01.866303 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.777 [INFO][10052] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15897 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.777 [INFO][10052] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.810 [INFO][10061] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" HandleID="k8s-pod-network.e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.811 [INFO][10061] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" HandleID="k8s-pod-network.e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cd8e0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:01.810549575 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.811 [INFO][10061] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.811 [INFO][10061] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.811 [INFO][10061] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.819 [INFO][10061] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" host="172-234-25-215" Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.824 [INFO][10061] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.829 [INFO][10061] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.831 [INFO][10061] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.834 [INFO][10061] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.835 [INFO][10061] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" host="172-234-25-215" Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.837 [INFO][10061] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45 Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.841 [INFO][10061] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" host="172-234-25-215" Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.852 [INFO][10061] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.9/26] block=192.168.62.0/26 handle="k8s-pod-network.e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" host="172-234-25-215" Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.885544 containerd[1567]: 2025-08-13 01:30:01.852 [INFO][10061] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.9/26] handle="k8s-pod-network.e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" host="172-234-25-215" Aug 13 01:30:01.885544 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.887484 containerd[1567]: 2025-08-13 01:30:01.852 [INFO][10061] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:01.887484 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.887484 containerd[1567]: 2025-08-13 01:30:01.853 [INFO][10061] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.9/26] IPv6=[] ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" HandleID="k8s-pod-network.e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:01.887484 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.887484 containerd[1567]: 2025-08-13 01:30:01.857 [INFO][10052] cni-plugin/k8s.go 418: Populated endpoint ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15897", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"7eba694e94aa4076c15bb87b8694ebf04d2d9420e0da40d03802a6b923c2a689", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.9/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:01.887484 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.887484 containerd[1567]: 2025-08-13 01:30:01.857 [INFO][10052] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.9/32] ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:01.887484 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.887484 containerd[1567]: 2025-08-13 01:30:01.857 [INFO][10052] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:01.887484 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.887484 containerd[1567]: 2025-08-13 01:30:01.866 [INFO][10052] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:01.887484 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.887484 containerd[1567]: 2025-08-13 01:30:01.868 [INFO][10052] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15897", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.9/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"0a:32:23:03:ad:0d", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:01.887484 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.887812 containerd[1567]: 2025-08-13 01:30:01.879 [INFO][10052] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:01.887812 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:01.965983 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:01.965996 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:02.075796 containerd[1567]: 2025-08-13 01:30:01.962 [INFO][10085] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Aug 13 01:30:02.075796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.075796 containerd[1567]: 2025-08-13 01:30:01.963 [INFO][10085] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" iface="eth0" netns="/var/run/netns/cni-51796ef5-72c7-fcd6-258a-580de3f25352" Aug 13 01:30:02.075796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.075796 containerd[1567]: 2025-08-13 01:30:01.963 [INFO][10085] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" iface="eth0" netns="/var/run/netns/cni-51796ef5-72c7-fcd6-258a-580de3f25352" Aug 13 01:30:02.075796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.075796 containerd[1567]: 2025-08-13 01:30:01.977 [INFO][10085] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" after=13.972494ms iface="eth0" netns="/var/run/netns/cni-51796ef5-72c7-fcd6-258a-580de3f25352" Aug 13 01:30:02.075796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.075796 containerd[1567]: 2025-08-13 01:30:01.978 [INFO][10085] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Aug 13 01:30:02.075796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.075796 containerd[1567]: 2025-08-13 01:30:01.978 [INFO][10085] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Aug 13 01:30:02.075796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.075796 containerd[1567]: 2025-08-13 01:30:02.028 [INFO][10095] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" HandleID="k8s-pod-network.e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:02.075796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.075796 containerd[1567]: 2025-08-13 01:30:02.028 [INFO][10095] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:02.075796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.075796 containerd[1567]: 2025-08-13 01:30:02.028 [INFO][10095] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:02.075796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.075796 containerd[1567]: 2025-08-13 01:30:02.066 [INFO][10095] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" HandleID="k8s-pod-network.e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:02.075796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.075796 containerd[1567]: 2025-08-13 01:30:02.066 [INFO][10095] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" HandleID="k8s-pod-network.e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:02.075796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.075796 containerd[1567]: 2025-08-13 01:30:02.068 [INFO][10095] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:02.075796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.075796 containerd[1567]: 2025-08-13 01:30:02.071 [INFO][10085] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45" Aug 13 01:30:02.075796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.082766 containerd[1567]: time="2025-08-13T01:30:02.081704840Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45-lo: no space left on device" Aug 13 01:30:02.082848 kubelet[2720]: E0813 01:30:02.082814 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45-lo: no space left on device" Aug 13 01:30:02.083117 kubelet[2720]: E0813 01:30:02.082855 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:02.083117 kubelet[2720]: E0813 01:30:02.082873 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:02.083117 kubelet[2720]: E0813 01:30:02.082973 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:02.083922 systemd[1]: run-netns-cni\x2d51796ef5\x2d72c7\x2dfcd6\x2d258a\x2d580de3f25352.mount: Deactivated successfully. Aug 13 01:30:02.737959 containerd[1567]: time="2025-08-13T01:30:02.737898659Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:02.863118 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:02.864031 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.779 [INFO][10103] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 15961 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.779 [INFO][10103] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.805 [INFO][10117] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" HandleID="k8s-pod-network.eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.805 [INFO][10117] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" HandleID="k8s-pod-network.eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f100), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:02.805177254 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.805 [INFO][10117] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.805 [INFO][10117] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.805 [INFO][10117] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.813 [INFO][10117] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" host="172-234-25-215" Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.819 [INFO][10117] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.824 [INFO][10117] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.826 [INFO][10117] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.829 [INFO][10117] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.829 [INFO][10117] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" host="172-234-25-215" Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.831 [INFO][10117] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6 Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.836 [INFO][10117] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" host="172-234-25-215" Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.846 [INFO][10117] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.10/26] block=192.168.62.0/26 handle="k8s-pod-network.eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" host="172-234-25-215" Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.882508 containerd[1567]: 2025-08-13 01:30:02.846 [INFO][10117] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.10/26] handle="k8s-pod-network.eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" host="172-234-25-215" Aug 13 01:30:02.882508 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.885955 containerd[1567]: 2025-08-13 01:30:02.846 [INFO][10117] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:02.885955 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.885955 containerd[1567]: 2025-08-13 01:30:02.846 [INFO][10117] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.10/26] IPv6=[] ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" HandleID="k8s-pod-network.eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:02.885955 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.885955 containerd[1567]: 2025-08-13 01:30:02.850 [INFO][10103] cni-plugin/k8s.go 418: Populated endpoint ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15961", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e259d5a587659e718073912f479a27a48ae7aa688f7b313614f1272a1ce2ed45", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.10/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:02.885955 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.885955 containerd[1567]: 2025-08-13 01:30:02.850 [INFO][10103] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.10/32] ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:02.885955 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.885955 containerd[1567]: 2025-08-13 01:30:02.850 [INFO][10103] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:02.885955 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.885955 containerd[1567]: 2025-08-13 01:30:02.863 [INFO][10103] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:02.885955 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.885955 containerd[1567]: 2025-08-13 01:30:02.863 [INFO][10103] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"15961", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.10/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"0e:12:2a:26:d2:8f", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:02.885955 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:02.886288 containerd[1567]: 2025-08-13 01:30:02.878 [INFO][10103] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:02.886288 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.009346 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:03.010459 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:03.073557 kubelet[2720]: E0813 01:30:03.073527 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:30:03.075197 kubelet[2720]: E0813 01:30:03.073996 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:30:03.077734 kubelet[2720]: E0813 01:30:03.077636 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-294491888-fYh9 sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4258209367: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:30:03.077891 kubelet[2720]: E0813 01:30:03.077870 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-307916557-_eRC sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-3587324838: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:30:03.118793 containerd[1567]: 2025-08-13 01:30:03.003 [INFO][10140] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Aug 13 01:30:03.118793 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.118793 containerd[1567]: 2025-08-13 01:30:03.005 [INFO][10140] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" iface="eth0" netns="/var/run/netns/cni-0f5efa31-c017-91ed-29e1-11f0052340c8" Aug 13 01:30:03.118793 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.118793 containerd[1567]: 2025-08-13 01:30:03.006 [INFO][10140] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" iface="eth0" netns="/var/run/netns/cni-0f5efa31-c017-91ed-29e1-11f0052340c8" Aug 13 01:30:03.118793 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.118793 containerd[1567]: 2025-08-13 01:30:03.020 [INFO][10140] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" after=15.080961ms iface="eth0" netns="/var/run/netns/cni-0f5efa31-c017-91ed-29e1-11f0052340c8" Aug 13 01:30:03.118793 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.118793 containerd[1567]: 2025-08-13 01:30:03.020 [INFO][10140] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Aug 13 01:30:03.118793 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.118793 containerd[1567]: 2025-08-13 01:30:03.021 [INFO][10140] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Aug 13 01:30:03.118793 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.118793 containerd[1567]: 2025-08-13 01:30:03.058 [INFO][10149] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" HandleID="k8s-pod-network.eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:03.118793 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.118793 containerd[1567]: 2025-08-13 01:30:03.058 [INFO][10149] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:03.118793 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.118793 containerd[1567]: 2025-08-13 01:30:03.058 [INFO][10149] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:03.118793 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.118793 containerd[1567]: 2025-08-13 01:30:03.102 [INFO][10149] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" HandleID="k8s-pod-network.eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:03.118793 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.118793 containerd[1567]: 2025-08-13 01:30:03.102 [INFO][10149] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" HandleID="k8s-pod-network.eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:03.118793 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.118793 containerd[1567]: 2025-08-13 01:30:03.108 [INFO][10149] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:03.118793 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.118793 containerd[1567]: 2025-08-13 01:30:03.113 [INFO][10140] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6" Aug 13 01:30:03.118793 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.123001 containerd[1567]: time="2025-08-13T01:30:03.122925302Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6-lo: no space left on device" Aug 13 01:30:03.123399 systemd[1]: run-netns-cni\x2d0f5efa31\x2dc017\x2d91ed\x2d29e1\x2d11f0052340c8.mount: Deactivated successfully. Aug 13 01:30:03.123982 kubelet[2720]: E0813 01:30:03.123953 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6-lo: no space left on device" Aug 13 01:30:03.124248 kubelet[2720]: E0813 01:30:03.124231 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:03.124307 kubelet[2720]: E0813 01:30:03.124294 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:03.124566 kubelet[2720]: E0813 01:30:03.124540 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:03.357786 containerd[1567]: time="2025-08-13T01:30:03.357153227Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:03.740025 containerd[1567]: time="2025-08-13T01:30:03.739967495Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:03.870754 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:03.870979 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.787 [INFO][10160] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16033 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.787 [INFO][10160] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.820 [INFO][10172] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" HandleID="k8s-pod-network.9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.820 [INFO][10172] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" HandleID="k8s-pod-network.9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002ad660), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:03.820057526 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.820 [INFO][10172] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.820 [INFO][10172] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.820 [INFO][10172] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.827 [INFO][10172] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" host="172-234-25-215" Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.831 [INFO][10172] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.836 [INFO][10172] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.837 [INFO][10172] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.842 [INFO][10172] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.842 [INFO][10172] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" host="172-234-25-215" Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.845 [INFO][10172] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.852 [INFO][10172] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" host="172-234-25-215" Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.860 [INFO][10172] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.11/26] block=192.168.62.0/26 handle="k8s-pod-network.9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" host="172-234-25-215" Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.892385 containerd[1567]: 2025-08-13 01:30:03.860 [INFO][10172] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.11/26] handle="k8s-pod-network.9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" host="172-234-25-215" Aug 13 01:30:03.892385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.893176 containerd[1567]: 2025-08-13 01:30:03.861 [INFO][10172] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:03.893176 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.893176 containerd[1567]: 2025-08-13 01:30:03.861 [INFO][10172] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.11/26] IPv6=[] ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" HandleID="k8s-pod-network.9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:03.893176 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.893176 containerd[1567]: 2025-08-13 01:30:03.864 [INFO][10160] cni-plugin/k8s.go 418: Populated endpoint ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16033", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"eda3a44cadcf7714597b675158e3f3c19ce62c9b52b051437cef1a249af455c6", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.11/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:03.893176 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.893176 containerd[1567]: 2025-08-13 01:30:03.864 [INFO][10160] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.11/32] ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:03.893176 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.893176 containerd[1567]: 2025-08-13 01:30:03.864 [INFO][10160] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:03.893176 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.893176 containerd[1567]: 2025-08-13 01:30:03.870 [INFO][10160] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:03.893176 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.893176 containerd[1567]: 2025-08-13 01:30:03.871 [INFO][10160] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16033", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.11/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"52:27:66:19:1d:8c", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:03.893176 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.894324 containerd[1567]: 2025-08-13 01:30:03.885 [INFO][10160] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:03.894324 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:03.997061 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:03.997070 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:04.092164 containerd[1567]: 2025-08-13 01:30:03.992 [INFO][10195] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Aug 13 01:30:04.092164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.092164 containerd[1567]: 2025-08-13 01:30:03.993 [INFO][10195] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" iface="eth0" netns="/var/run/netns/cni-f1e1f1a0-702d-0171-480d-b30b1858b6f0" Aug 13 01:30:04.092164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.092164 containerd[1567]: 2025-08-13 01:30:03.993 [INFO][10195] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" iface="eth0" netns="/var/run/netns/cni-f1e1f1a0-702d-0171-480d-b30b1858b6f0" Aug 13 01:30:04.092164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.092164 containerd[1567]: 2025-08-13 01:30:04.005 [INFO][10195] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" after=12.693367ms iface="eth0" netns="/var/run/netns/cni-f1e1f1a0-702d-0171-480d-b30b1858b6f0" Aug 13 01:30:04.092164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.092164 containerd[1567]: 2025-08-13 01:30:04.005 [INFO][10195] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Aug 13 01:30:04.092164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.092164 containerd[1567]: 2025-08-13 01:30:04.006 [INFO][10195] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Aug 13 01:30:04.092164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.092164 containerd[1567]: 2025-08-13 01:30:04.035 [INFO][10205] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" HandleID="k8s-pod-network.9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:04.092164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.092164 containerd[1567]: 2025-08-13 01:30:04.036 [INFO][10205] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:04.092164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.092164 containerd[1567]: 2025-08-13 01:30:04.036 [INFO][10205] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:04.092164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.092164 containerd[1567]: 2025-08-13 01:30:04.085 [INFO][10205] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" HandleID="k8s-pod-network.9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:04.092164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.092164 containerd[1567]: 2025-08-13 01:30:04.085 [INFO][10205] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" HandleID="k8s-pod-network.9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:04.092164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.092164 containerd[1567]: 2025-08-13 01:30:04.086 [INFO][10205] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:04.092164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.092164 containerd[1567]: 2025-08-13 01:30:04.089 [INFO][10195] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b" Aug 13 01:30:04.092164 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.096362 containerd[1567]: time="2025-08-13T01:30:04.096173083Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b-lo: no space left on device" Aug 13 01:30:04.096740 kubelet[2720]: E0813 01:30:04.096680 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b-lo: no space left on device" Aug 13 01:30:04.096821 kubelet[2720]: E0813 01:30:04.096793 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:04.096857 kubelet[2720]: E0813 01:30:04.096847 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:04.096969 kubelet[2720]: E0813 01:30:04.096899 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:04.098836 systemd[1]: run-netns-cni\x2df1e1f1a0\x2d702d\x2d0171\x2d480d\x2db30b1858b6f0.mount: Deactivated successfully. Aug 13 01:30:04.234414 systemd[1]: Started sshd@37-172.234.25.215:22-147.75.109.163:35058.service - OpenSSH per-connection server daemon (147.75.109.163:35058). Aug 13 01:30:04.576615 sshd[10215]: Accepted publickey for core from 147.75.109.163 port 35058 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:30:04.578191 sshd-session[10215]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:30:04.584000 systemd-logind[1543]: New session 38 of user core. Aug 13 01:30:04.589872 systemd[1]: Started session-38.scope - Session 38 of User core. Aug 13 01:30:04.741882 containerd[1567]: time="2025-08-13T01:30:04.741824411Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:04.896618 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:04.898946 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:04.920636 sshd[10217]: Connection closed by 147.75.109.163 port 35058 Aug 13 01:30:04.921889 sshd-session[10215]: pam_unix(sshd:session): session closed for user core Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.807 [INFO][10221] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16104 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.807 [INFO][10221] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.842 [INFO][10238] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" HandleID="k8s-pod-network.8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.843 [INFO][10238] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" HandleID="k8s-pod-network.8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002ad670), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:04.842806084 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.843 [INFO][10238] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.843 [INFO][10238] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.843 [INFO][10238] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.858 [INFO][10238] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" host="172-234-25-215" Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.864 [INFO][10238] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.869 [INFO][10238] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.871 [INFO][10238] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.873 [INFO][10238] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.873 [INFO][10238] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" host="172-234-25-215" Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.875 [INFO][10238] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166 Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.878 [INFO][10238] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" host="172-234-25-215" Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.887 [INFO][10238] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.12/26] block=192.168.62.0/26 handle="k8s-pod-network.8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" host="172-234-25-215" Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.926895 containerd[1567]: 2025-08-13 01:30:04.887 [INFO][10238] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.12/26] handle="k8s-pod-network.8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" host="172-234-25-215" Aug 13 01:30:04.926895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.927551 containerd[1567]: 2025-08-13 01:30:04.887 [INFO][10238] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:04.927551 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.927551 containerd[1567]: 2025-08-13 01:30:04.887 [INFO][10238] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.12/26] IPv6=[] ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" HandleID="k8s-pod-network.8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:04.927551 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.927551 containerd[1567]: 2025-08-13 01:30:04.890 [INFO][10221] cni-plugin/k8s.go 418: Populated endpoint ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16104", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"9a5ae1da27dd9f94f0f15fa0a5a1440cad17fc4661e7a16aa4df4191bb48265b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.12/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:04.927551 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.927551 containerd[1567]: 2025-08-13 01:30:04.891 [INFO][10221] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.12/32] ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:04.927551 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.927551 containerd[1567]: 2025-08-13 01:30:04.891 [INFO][10221] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:04.927551 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.927551 containerd[1567]: 2025-08-13 01:30:04.898 [INFO][10221] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:04.927551 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.927551 containerd[1567]: 2025-08-13 01:30:04.901 [INFO][10221] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16104", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.12/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"86:c2:de:40:c6:08", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:04.927551 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.930653 containerd[1567]: 2025-08-13 01:30:04.918 [INFO][10221] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:04.930653 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:04.932124 systemd[1]: sshd@37-172.234.25.215:22-147.75.109.163:35058.service: Deactivated successfully. Aug 13 01:30:04.937274 systemd[1]: session-38.scope: Deactivated successfully. Aug 13 01:30:04.939798 systemd-logind[1543]: Session 38 logged out. Waiting for processes to exit. Aug 13 01:30:04.944011 systemd-logind[1543]: Removed session 38. Aug 13 01:30:05.008186 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:05.008214 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:05.119758 containerd[1567]: 2025-08-13 01:30:05.004 [INFO][10266] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Aug 13 01:30:05.119758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.119758 containerd[1567]: 2025-08-13 01:30:05.005 [INFO][10266] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" iface="eth0" netns="/var/run/netns/cni-5551f21c-b0fb-7c8f-5616-1661eb9270ae" Aug 13 01:30:05.119758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.119758 containerd[1567]: 2025-08-13 01:30:05.006 [INFO][10266] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" iface="eth0" netns="/var/run/netns/cni-5551f21c-b0fb-7c8f-5616-1661eb9270ae" Aug 13 01:30:05.119758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.119758 containerd[1567]: 2025-08-13 01:30:05.015 [INFO][10266] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" after=10.45142ms iface="eth0" netns="/var/run/netns/cni-5551f21c-b0fb-7c8f-5616-1661eb9270ae" Aug 13 01:30:05.119758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.119758 containerd[1567]: 2025-08-13 01:30:05.015 [INFO][10266] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Aug 13 01:30:05.119758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.119758 containerd[1567]: 2025-08-13 01:30:05.015 [INFO][10266] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Aug 13 01:30:05.119758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.119758 containerd[1567]: 2025-08-13 01:30:05.073 [INFO][10274] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" HandleID="k8s-pod-network.8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:05.119758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.119758 containerd[1567]: 2025-08-13 01:30:05.073 [INFO][10274] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:05.119758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.119758 containerd[1567]: 2025-08-13 01:30:05.073 [INFO][10274] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:05.119758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.119758 containerd[1567]: 2025-08-13 01:30:05.110 [INFO][10274] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" HandleID="k8s-pod-network.8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:05.119758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.119758 containerd[1567]: 2025-08-13 01:30:05.110 [INFO][10274] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" HandleID="k8s-pod-network.8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:05.119758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.119758 containerd[1567]: 2025-08-13 01:30:05.111 [INFO][10274] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:05.119758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.119758 containerd[1567]: 2025-08-13 01:30:05.114 [INFO][10266] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166" Aug 13 01:30:05.119758 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.122253 containerd[1567]: time="2025-08-13T01:30:05.121216166Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166-lo: no space left on device" Aug 13 01:30:05.121266 systemd[1]: run-netns-cni\x2d5551f21c\x2db0fb\x2d7c8f\x2d5616\x2d1661eb9270ae.mount: Deactivated successfully. Aug 13 01:30:05.123428 kubelet[2720]: E0813 01:30:05.123386 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166-lo: no space left on device" Aug 13 01:30:05.123821 kubelet[2720]: E0813 01:30:05.123460 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:05.123821 kubelet[2720]: E0813 01:30:05.123480 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:05.123821 kubelet[2720]: E0813 01:30:05.123552 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:05.743363 containerd[1567]: time="2025-08-13T01:30:05.743313789Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:05.874883 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:05.875349 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.791 [INFO][10285] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16176 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.791 [INFO][10285] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.826 [INFO][10298] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" HandleID="k8s-pod-network.f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.826 [INFO][10298] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" HandleID="k8s-pod-network.f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f100), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:05.826000805 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.826 [INFO][10298] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.826 [INFO][10298] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.826 [INFO][10298] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.832 [INFO][10298] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" host="172-234-25-215" Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.836 [INFO][10298] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.841 [INFO][10298] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.844 [INFO][10298] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.848 [INFO][10298] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.848 [INFO][10298] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" host="172-234-25-215" Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.852 [INFO][10298] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.856 [INFO][10298] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" host="172-234-25-215" Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.863 [INFO][10298] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.13/26] block=192.168.62.0/26 handle="k8s-pod-network.f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" host="172-234-25-215" Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.894372 containerd[1567]: 2025-08-13 01:30:05.864 [INFO][10298] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.13/26] handle="k8s-pod-network.f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" host="172-234-25-215" Aug 13 01:30:05.894372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.896091 containerd[1567]: 2025-08-13 01:30:05.864 [INFO][10298] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:05.896091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.896091 containerd[1567]: 2025-08-13 01:30:05.864 [INFO][10298] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.13/26] IPv6=[] ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" HandleID="k8s-pod-network.f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:05.896091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.896091 containerd[1567]: 2025-08-13 01:30:05.866 [INFO][10285] cni-plugin/k8s.go 418: Populated endpoint ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16176", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"8134235fe8de22b266d308f878d9129b33c2c2c4ea0b63ad83226a489fdee166", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.13/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:05.896091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.896091 containerd[1567]: 2025-08-13 01:30:05.866 [INFO][10285] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.13/32] ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:05.896091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.896091 containerd[1567]: 2025-08-13 01:30:05.866 [INFO][10285] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:05.896091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.896091 containerd[1567]: 2025-08-13 01:30:05.875 [INFO][10285] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:05.896091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.896091 containerd[1567]: 2025-08-13 01:30:05.876 [INFO][10285] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16176", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.13/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ee:07:79:21:51:a9", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:05.896091 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.897251 containerd[1567]: 2025-08-13 01:30:05.889 [INFO][10285] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:05.897251 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:05.988915 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:05.989124 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:06.072831 containerd[1567]: 2025-08-13 01:30:05.983 [INFO][10323] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Aug 13 01:30:06.072831 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.072831 containerd[1567]: 2025-08-13 01:30:05.984 [INFO][10323] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" iface="eth0" netns="/var/run/netns/cni-13305ba6-122a-1d92-bb2a-0f433c205dca" Aug 13 01:30:06.072831 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.072831 containerd[1567]: 2025-08-13 01:30:05.985 [INFO][10323] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" iface="eth0" netns="/var/run/netns/cni-13305ba6-122a-1d92-bb2a-0f433c205dca" Aug 13 01:30:06.072831 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.072831 containerd[1567]: 2025-08-13 01:30:05.993 [INFO][10323] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" after=8.858914ms iface="eth0" netns="/var/run/netns/cni-13305ba6-122a-1d92-bb2a-0f433c205dca" Aug 13 01:30:06.072831 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.072831 containerd[1567]: 2025-08-13 01:30:05.993 [INFO][10323] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Aug 13 01:30:06.072831 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.072831 containerd[1567]: 2025-08-13 01:30:05.993 [INFO][10323] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Aug 13 01:30:06.072831 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.072831 containerd[1567]: 2025-08-13 01:30:06.025 [INFO][10335] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" HandleID="k8s-pod-network.f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:06.072831 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.072831 containerd[1567]: 2025-08-13 01:30:06.026 [INFO][10335] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:06.072831 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.072831 containerd[1567]: 2025-08-13 01:30:06.026 [INFO][10335] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:06.072831 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.072831 containerd[1567]: 2025-08-13 01:30:06.064 [INFO][10335] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" HandleID="k8s-pod-network.f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:06.072831 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.072831 containerd[1567]: 2025-08-13 01:30:06.065 [INFO][10335] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" HandleID="k8s-pod-network.f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:06.072831 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.072831 containerd[1567]: 2025-08-13 01:30:06.067 [INFO][10335] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:06.072831 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.072831 containerd[1567]: 2025-08-13 01:30:06.069 [INFO][10323] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f" Aug 13 01:30:06.072831 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.076622 systemd[1]: run-netns-cni\x2d13305ba6\x2d122a\x2d1d92\x2dbb2a\x2d0f433c205dca.mount: Deactivated successfully. Aug 13 01:30:06.079057 containerd[1567]: time="2025-08-13T01:30:06.077980677Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f-lo: no space left on device" Aug 13 01:30:06.079143 kubelet[2720]: E0813 01:30:06.078911 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f-lo: no space left on device" Aug 13 01:30:06.079143 kubelet[2720]: E0813 01:30:06.078970 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:06.079143 kubelet[2720]: E0813 01:30:06.078987 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:06.079143 kubelet[2720]: E0813 01:30:06.079047 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:06.360182 containerd[1567]: time="2025-08-13T01:30:06.360069733Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:06.745703 containerd[1567]: time="2025-08-13T01:30:06.745656956Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:06.865461 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:06.865689 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.788 [INFO][10345] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16244 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.788 [INFO][10345] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.818 [INFO][10359] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" HandleID="k8s-pod-network.4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.819 [INFO][10359] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" HandleID="k8s-pod-network.4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f210), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:06.81888659 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.819 [INFO][10359] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.819 [INFO][10359] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.819 [INFO][10359] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.825 [INFO][10359] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" host="172-234-25-215" Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.829 [INFO][10359] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.834 [INFO][10359] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.836 [INFO][10359] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.839 [INFO][10359] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.839 [INFO][10359] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" host="172-234-25-215" Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.841 [INFO][10359] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.847 [INFO][10359] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" host="172-234-25-215" Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.853 [INFO][10359] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.14/26] block=192.168.62.0/26 handle="k8s-pod-network.4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" host="172-234-25-215" Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.881859 containerd[1567]: 2025-08-13 01:30:06.853 [INFO][10359] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.14/26] handle="k8s-pod-network.4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" host="172-234-25-215" Aug 13 01:30:06.881859 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.884325 containerd[1567]: 2025-08-13 01:30:06.853 [INFO][10359] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:06.884325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.884325 containerd[1567]: 2025-08-13 01:30:06.853 [INFO][10359] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.14/26] IPv6=[] ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" HandleID="k8s-pod-network.4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:06.884325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.884325 containerd[1567]: 2025-08-13 01:30:06.857 [INFO][10345] cni-plugin/k8s.go 418: Populated endpoint ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16244", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"f6d82c4b78c8c830af345fe4b1b7c4c38f656bc00c75be5770b80fd3708b867f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.14/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:06.884325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.884325 containerd[1567]: 2025-08-13 01:30:06.857 [INFO][10345] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.14/32] ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:06.884325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.884325 containerd[1567]: 2025-08-13 01:30:06.857 [INFO][10345] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:06.884325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.884325 containerd[1567]: 2025-08-13 01:30:06.865 [INFO][10345] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:06.884325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.884325 containerd[1567]: 2025-08-13 01:30:06.866 [INFO][10345] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16244", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.14/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"8a:cf:9a:bd:7e:26", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:06.884325 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.884884 containerd[1567]: 2025-08-13 01:30:06.875 [INFO][10345] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:06.884884 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:06.953780 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:06.953789 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:07.082943 containerd[1567]: 2025-08-13 01:30:06.949 [INFO][10386] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Aug 13 01:30:07.082943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.082943 containerd[1567]: 2025-08-13 01:30:06.950 [INFO][10386] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" iface="eth0" netns="/var/run/netns/cni-fc0191bb-7b49-79b7-b21f-6a13a42c3b8f" Aug 13 01:30:07.082943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.082943 containerd[1567]: 2025-08-13 01:30:06.951 [INFO][10386] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" iface="eth0" netns="/var/run/netns/cni-fc0191bb-7b49-79b7-b21f-6a13a42c3b8f" Aug 13 01:30:07.082943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.082943 containerd[1567]: 2025-08-13 01:30:06.959 [INFO][10386] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" after=8.983144ms iface="eth0" netns="/var/run/netns/cni-fc0191bb-7b49-79b7-b21f-6a13a42c3b8f" Aug 13 01:30:07.082943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.082943 containerd[1567]: 2025-08-13 01:30:06.959 [INFO][10386] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Aug 13 01:30:07.082943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.082943 containerd[1567]: 2025-08-13 01:30:06.959 [INFO][10386] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Aug 13 01:30:07.082943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.082943 containerd[1567]: 2025-08-13 01:30:07.002 [INFO][10394] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" HandleID="k8s-pod-network.4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:07.082943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.082943 containerd[1567]: 2025-08-13 01:30:07.003 [INFO][10394] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:07.082943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.082943 containerd[1567]: 2025-08-13 01:30:07.003 [INFO][10394] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:07.082943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.082943 containerd[1567]: 2025-08-13 01:30:07.068 [INFO][10394] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" HandleID="k8s-pod-network.4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:07.082943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.082943 containerd[1567]: 2025-08-13 01:30:07.068 [INFO][10394] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" HandleID="k8s-pod-network.4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:07.082943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.082943 containerd[1567]: 2025-08-13 01:30:07.075 [INFO][10394] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:07.082943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.082943 containerd[1567]: 2025-08-13 01:30:07.079 [INFO][10386] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb" Aug 13 01:30:07.082943 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.088118 containerd[1567]: time="2025-08-13T01:30:07.088003651Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb-lo: no space left on device" Aug 13 01:30:07.090352 kubelet[2720]: E0813 01:30:07.088350 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb-lo: no space left on device" Aug 13 01:30:07.090352 kubelet[2720]: E0813 01:30:07.088413 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:07.090352 kubelet[2720]: E0813 01:30:07.088472 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:07.090352 kubelet[2720]: E0813 01:30:07.088525 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:07.091648 systemd[1]: run-netns-cni\x2dfc0191bb\x2d7b49\x2d79b7\x2db21f\x2d6a13a42c3b8f.mount: Deactivated successfully. Aug 13 01:30:07.748181 containerd[1567]: time="2025-08-13T01:30:07.748128173Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:07.872671 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:07.874894 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.794 [INFO][10404] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16304 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.794 [INFO][10404] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.832 [INFO][10416] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" HandleID="k8s-pod-network.62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.832 [INFO][10416] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" HandleID="k8s-pod-network.62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f020), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:07.832387477 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.832 [INFO][10416] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.832 [INFO][10416] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.832 [INFO][10416] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.838 [INFO][10416] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" host="172-234-25-215" Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.842 [INFO][10416] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.846 [INFO][10416] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.848 [INFO][10416] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.851 [INFO][10416] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.851 [INFO][10416] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" host="172-234-25-215" Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.854 [INFO][10416] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.859 [INFO][10416] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" host="172-234-25-215" Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.866 [INFO][10416] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.15/26] block=192.168.62.0/26 handle="k8s-pod-network.62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" host="172-234-25-215" Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.890328 containerd[1567]: 2025-08-13 01:30:07.866 [INFO][10416] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.15/26] handle="k8s-pod-network.62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" host="172-234-25-215" Aug 13 01:30:07.890328 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.891103 containerd[1567]: 2025-08-13 01:30:07.866 [INFO][10416] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:07.891103 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.891103 containerd[1567]: 2025-08-13 01:30:07.866 [INFO][10416] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.15/26] IPv6=[] ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" HandleID="k8s-pod-network.62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:07.891103 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.891103 containerd[1567]: 2025-08-13 01:30:07.869 [INFO][10404] cni-plugin/k8s.go 418: Populated endpoint ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16304", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"4c76dfbeddb13929e7eb7603f8354fbae27a0544ccba876b83e9218784083ebb", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.15/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:07.891103 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.891103 containerd[1567]: 2025-08-13 01:30:07.869 [INFO][10404] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.15/32] ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:07.891103 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.891103 containerd[1567]: 2025-08-13 01:30:07.869 [INFO][10404] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:07.891103 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.891103 containerd[1567]: 2025-08-13 01:30:07.873 [INFO][10404] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:07.891103 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.891103 containerd[1567]: 2025-08-13 01:30:07.875 [INFO][10404] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16304", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.15/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"e6:f8:d3:07:b4:65", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:07.891103 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.891418 containerd[1567]: 2025-08-13 01:30:07.887 [INFO][10404] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:07.891418 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:07.961169 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:07.961178 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:08.064877 containerd[1567]: 2025-08-13 01:30:07.956 [INFO][10439] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Aug 13 01:30:08.064877 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.064877 containerd[1567]: 2025-08-13 01:30:07.957 [INFO][10439] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" iface="eth0" netns="/var/run/netns/cni-ed600688-fe17-e532-1577-2de2057fc9f5" Aug 13 01:30:08.064877 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.064877 containerd[1567]: 2025-08-13 01:30:07.957 [INFO][10439] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" iface="eth0" netns="/var/run/netns/cni-ed600688-fe17-e532-1577-2de2057fc9f5" Aug 13 01:30:08.064877 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.064877 containerd[1567]: 2025-08-13 01:30:07.965 [INFO][10439] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" after=8.031425ms iface="eth0" netns="/var/run/netns/cni-ed600688-fe17-e532-1577-2de2057fc9f5" Aug 13 01:30:08.064877 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.064877 containerd[1567]: 2025-08-13 01:30:07.965 [INFO][10439] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Aug 13 01:30:08.064877 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.064877 containerd[1567]: 2025-08-13 01:30:07.965 [INFO][10439] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Aug 13 01:30:08.064877 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.064877 containerd[1567]: 2025-08-13 01:30:08.011 [INFO][10449] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" HandleID="k8s-pod-network.62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:08.064877 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.064877 containerd[1567]: 2025-08-13 01:30:08.012 [INFO][10449] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:08.064877 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.064877 containerd[1567]: 2025-08-13 01:30:08.012 [INFO][10449] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:08.064877 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.064877 containerd[1567]: 2025-08-13 01:30:08.058 [INFO][10449] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" HandleID="k8s-pod-network.62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:08.064877 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.064877 containerd[1567]: 2025-08-13 01:30:08.058 [INFO][10449] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" HandleID="k8s-pod-network.62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:08.064877 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.064877 containerd[1567]: 2025-08-13 01:30:08.060 [INFO][10449] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:08.064877 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.064877 containerd[1567]: 2025-08-13 01:30:08.062 [INFO][10439] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b" Aug 13 01:30:08.064877 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.067634 containerd[1567]: time="2025-08-13T01:30:08.066523172Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b-lo: no space left on device" Aug 13 01:30:08.071787 kubelet[2720]: E0813 01:30:08.067783 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b-lo: no space left on device" Aug 13 01:30:08.071787 kubelet[2720]: E0813 01:30:08.067869 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:08.071787 kubelet[2720]: E0813 01:30:08.067890 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:08.071787 kubelet[2720]: E0813 01:30:08.067954 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:08.069396 systemd[1]: run-netns-cni\x2ded600688\x2dfe17\x2de532\x2d1577\x2d2de2057fc9f5.mount: Deactivated successfully. Aug 13 01:30:08.296811 kubelet[2720]: I0813 01:30:08.296764 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:30:08.296811 kubelet[2720]: I0813 01:30:08.296813 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:30:08.301269 kubelet[2720]: I0813 01:30:08.301235 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:30:08.317459 kubelet[2720]: I0813 01:30:08.317394 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:30:08.317580 kubelet[2720]: I0813 01:30:08.317543 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-b2ds8","kube-system/coredns-674b8bbfcf-jlpx4","calico-system/calico-kube-controllers-b57d47f84-lznkx","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:30:08.317580 kubelet[2720]: E0813 01:30:08.317572 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:30:08.317580 kubelet[2720]: E0813 01:30:08.317579 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:30:08.317684 kubelet[2720]: E0813 01:30:08.317586 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:08.317684 kubelet[2720]: E0813 01:30:08.317595 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:30:08.317684 kubelet[2720]: E0813 01:30:08.317605 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:30:08.317684 kubelet[2720]: E0813 01:30:08.317613 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:30:08.317684 kubelet[2720]: E0813 01:30:08.317622 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:30:08.317684 kubelet[2720]: E0813 01:30:08.317633 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:30:08.317684 kubelet[2720]: E0813 01:30:08.317640 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:30:08.317684 kubelet[2720]: E0813 01:30:08.317648 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:30:08.317684 kubelet[2720]: I0813 01:30:08.317656 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:30:08.751906 containerd[1567]: time="2025-08-13T01:30:08.751701730Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:08.864334 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:08.866167 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.786 [INFO][10465] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16382 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.786 [INFO][10465] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.818 [INFO][10478] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" HandleID="k8s-pod-network.6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.819 [INFO][10478] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" HandleID="k8s-pod-network.6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cd9b0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:08.818794635 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.819 [INFO][10478] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.819 [INFO][10478] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.819 [INFO][10478] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.824 [INFO][10478] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" host="172-234-25-215" Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.827 [INFO][10478] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.835 [INFO][10478] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.837 [INFO][10478] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.840 [INFO][10478] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.840 [INFO][10478] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" host="172-234-25-215" Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.841 [INFO][10478] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.846 [INFO][10478] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" host="172-234-25-215" Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.855 [INFO][10478] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.16/26] block=192.168.62.0/26 handle="k8s-pod-network.6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" host="172-234-25-215" Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.882540 containerd[1567]: 2025-08-13 01:30:08.855 [INFO][10478] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.16/26] handle="k8s-pod-network.6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" host="172-234-25-215" Aug 13 01:30:08.882540 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.885437 containerd[1567]: 2025-08-13 01:30:08.855 [INFO][10478] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:08.885437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.885437 containerd[1567]: 2025-08-13 01:30:08.855 [INFO][10478] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.16/26] IPv6=[] ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" HandleID="k8s-pod-network.6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:08.885437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.885437 containerd[1567]: 2025-08-13 01:30:08.858 [INFO][10465] cni-plugin/k8s.go 418: Populated endpoint ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16382", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"62b292506e0961ef4101ef7d70c1e92335f9c74f3b366dd7889537a6961c933b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.16/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:08.885437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.885437 containerd[1567]: 2025-08-13 01:30:08.858 [INFO][10465] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.16/32] ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:08.885437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.885437 containerd[1567]: 2025-08-13 01:30:08.858 [INFO][10465] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:08.885437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.885437 containerd[1567]: 2025-08-13 01:30:08.866 [INFO][10465] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:08.885437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.885437 containerd[1567]: 2025-08-13 01:30:08.867 [INFO][10465] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16382", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.16/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"8a:8e:b6:83:c6:1a", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:08.885437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.886200 containerd[1567]: 2025-08-13 01:30:08.878 [INFO][10465] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:08.886200 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:08.953996 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:08.954016 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:09.066838 containerd[1567]: 2025-08-13 01:30:08.951 [INFO][10501] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Aug 13 01:30:09.066838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.066838 containerd[1567]: 2025-08-13 01:30:08.952 [INFO][10501] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" iface="eth0" netns="/var/run/netns/cni-12d17efd-2acd-9417-5cd1-5bdb71cf3c40" Aug 13 01:30:09.066838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.066838 containerd[1567]: 2025-08-13 01:30:08.952 [INFO][10501] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" iface="eth0" netns="/var/run/netns/cni-12d17efd-2acd-9417-5cd1-5bdb71cf3c40" Aug 13 01:30:09.066838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.066838 containerd[1567]: 2025-08-13 01:30:08.962 [INFO][10501] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" after=10.59001ms iface="eth0" netns="/var/run/netns/cni-12d17efd-2acd-9417-5cd1-5bdb71cf3c40" Aug 13 01:30:09.066838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.066838 containerd[1567]: 2025-08-13 01:30:08.962 [INFO][10501] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Aug 13 01:30:09.066838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.066838 containerd[1567]: 2025-08-13 01:30:08.963 [INFO][10501] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Aug 13 01:30:09.066838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.066838 containerd[1567]: 2025-08-13 01:30:09.019 [INFO][10510] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" HandleID="k8s-pod-network.6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:09.066838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.066838 containerd[1567]: 2025-08-13 01:30:09.019 [INFO][10510] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:09.066838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.066838 containerd[1567]: 2025-08-13 01:30:09.019 [INFO][10510] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:09.066838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.066838 containerd[1567]: 2025-08-13 01:30:09.058 [INFO][10510] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" HandleID="k8s-pod-network.6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:09.066838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.066838 containerd[1567]: 2025-08-13 01:30:09.058 [INFO][10510] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" HandleID="k8s-pod-network.6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:09.066838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.066838 containerd[1567]: 2025-08-13 01:30:09.060 [INFO][10510] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:09.066838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.066838 containerd[1567]: 2025-08-13 01:30:09.064 [INFO][10501] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed" Aug 13 01:30:09.066838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.070772 containerd[1567]: time="2025-08-13T01:30:09.069733089Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed-lo: no space left on device" Aug 13 01:30:09.070852 kubelet[2720]: E0813 01:30:09.070003 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed-lo: no space left on device" Aug 13 01:30:09.070852 kubelet[2720]: E0813 01:30:09.070054 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:09.070852 kubelet[2720]: E0813 01:30:09.070076 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:09.070852 kubelet[2720]: E0813 01:30:09.070124 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:09.072784 systemd[1]: run-netns-cni\x2d12d17efd\x2d2acd\x2d9417\x2d5cd1\x2d5bdb71cf3c40.mount: Deactivated successfully. Aug 13 01:30:09.360328 containerd[1567]: time="2025-08-13T01:30:09.360223810Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:09.755437 containerd[1567]: time="2025-08-13T01:30:09.755402866Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:09.869486 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:09.869707 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.794 [INFO][10521] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16461 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.794 [INFO][10521] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.826 [INFO][10534] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" HandleID="k8s-pod-network.d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.826 [INFO][10534] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" HandleID="k8s-pod-network.d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f270), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:09.826811433 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.827 [INFO][10534] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.827 [INFO][10534] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.827 [INFO][10534] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.833 [INFO][10534] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" host="172-234-25-215" Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.837 [INFO][10534] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.842 [INFO][10534] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.843 [INFO][10534] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.848 [INFO][10534] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.848 [INFO][10534] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" host="172-234-25-215" Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.850 [INFO][10534] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3 Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.857 [INFO][10534] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" host="172-234-25-215" Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.863 [INFO][10534] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.17/26] block=192.168.62.0/26 handle="k8s-pod-network.d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" host="172-234-25-215" Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.894226 containerd[1567]: 2025-08-13 01:30:09.863 [INFO][10534] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.17/26] handle="k8s-pod-network.d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" host="172-234-25-215" Aug 13 01:30:09.894226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.895273 containerd[1567]: 2025-08-13 01:30:09.863 [INFO][10534] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:09.895273 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.895273 containerd[1567]: 2025-08-13 01:30:09.863 [INFO][10534] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.17/26] IPv6=[] ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" HandleID="k8s-pod-network.d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:09.895273 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.895273 containerd[1567]: 2025-08-13 01:30:09.866 [INFO][10521] cni-plugin/k8s.go 418: Populated endpoint ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16461", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"6ac2af7996add5448e877bec83a5a1617457ed0307d0c3b524907041567e02ed", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.17/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:09.895273 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.895273 containerd[1567]: 2025-08-13 01:30:09.866 [INFO][10521] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.17/32] ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:09.895273 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.895273 containerd[1567]: 2025-08-13 01:30:09.866 [INFO][10521] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:09.895273 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.895273 containerd[1567]: 2025-08-13 01:30:09.868 [INFO][10521] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:09.895273 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.895273 containerd[1567]: 2025-08-13 01:30:09.874 [INFO][10521] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16461", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.17/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ca:44:4e:32:63:84", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:09.895273 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.895647 containerd[1567]: 2025-08-13 01:30:09.885 [INFO][10521] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:09.895647 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:09.971656 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:09.971663 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:09.984079 systemd[1]: Started sshd@38-172.234.25.215:22-147.75.109.163:39778.service - OpenSSH per-connection server daemon (147.75.109.163:39778). Aug 13 01:30:10.078753 containerd[1567]: 2025-08-13 01:30:09.964 [INFO][10559] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Aug 13 01:30:10.078753 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.078753 containerd[1567]: 2025-08-13 01:30:09.965 [INFO][10559] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" iface="eth0" netns="/var/run/netns/cni-5085e5a3-b0ad-6d7c-ac0f-fd0702c5deef" Aug 13 01:30:10.078753 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.078753 containerd[1567]: 2025-08-13 01:30:09.966 [INFO][10559] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" iface="eth0" netns="/var/run/netns/cni-5085e5a3-b0ad-6d7c-ac0f-fd0702c5deef" Aug 13 01:30:10.078753 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.078753 containerd[1567]: 2025-08-13 01:30:09.975 [INFO][10559] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" after=9.391252ms iface="eth0" netns="/var/run/netns/cni-5085e5a3-b0ad-6d7c-ac0f-fd0702c5deef" Aug 13 01:30:10.078753 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.078753 containerd[1567]: 2025-08-13 01:30:09.975 [INFO][10559] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Aug 13 01:30:10.078753 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.078753 containerd[1567]: 2025-08-13 01:30:09.975 [INFO][10559] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Aug 13 01:30:10.078753 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.078753 containerd[1567]: 2025-08-13 01:30:10.014 [INFO][10568] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" HandleID="k8s-pod-network.d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:10.078753 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.078753 containerd[1567]: 2025-08-13 01:30:10.015 [INFO][10568] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:10.078753 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.078753 containerd[1567]: 2025-08-13 01:30:10.015 [INFO][10568] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:10.078753 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.078753 containerd[1567]: 2025-08-13 01:30:10.069 [INFO][10568] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" HandleID="k8s-pod-network.d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:10.078753 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.078753 containerd[1567]: 2025-08-13 01:30:10.069 [INFO][10568] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" HandleID="k8s-pod-network.d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:10.078753 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.078753 containerd[1567]: 2025-08-13 01:30:10.070 [INFO][10568] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:10.078753 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.078753 containerd[1567]: 2025-08-13 01:30:10.074 [INFO][10559] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3" Aug 13 01:30:10.078753 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.082424 systemd[1]: run-netns-cni\x2d5085e5a3\x2db0ad\x2d6d7c\x2dac0f\x2dfd0702c5deef.mount: Deactivated successfully. Aug 13 01:30:10.083085 containerd[1567]: time="2025-08-13T01:30:10.083042238Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3-lo: no space left on device" Aug 13 01:30:10.083811 kubelet[2720]: E0813 01:30:10.083764 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3-lo: no space left on device" Aug 13 01:30:10.084135 kubelet[2720]: E0813 01:30:10.083844 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:10.084135 kubelet[2720]: E0813 01:30:10.083866 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:10.084135 kubelet[2720]: E0813 01:30:10.083926 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:10.342511 sshd[10572]: Accepted publickey for core from 147.75.109.163 port 39778 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:30:10.344381 sshd-session[10572]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:30:10.353305 systemd-logind[1543]: New session 39 of user core. Aug 13 01:30:10.357960 systemd[1]: Started session-39.scope - Session 39 of User core. Aug 13 01:30:10.668957 sshd[10579]: Connection closed by 147.75.109.163 port 39778 Aug 13 01:30:10.669507 sshd-session[10572]: pam_unix(sshd:session): session closed for user core Aug 13 01:30:10.675196 systemd[1]: sshd@38-172.234.25.215:22-147.75.109.163:39778.service: Deactivated successfully. Aug 13 01:30:10.678149 systemd[1]: session-39.scope: Deactivated successfully. Aug 13 01:30:10.679783 systemd-logind[1543]: Session 39 logged out. Waiting for processes to exit. Aug 13 01:30:10.682398 systemd-logind[1543]: Removed session 39. Aug 13 01:30:10.759072 containerd[1567]: time="2025-08-13T01:30:10.758759954Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:10.882851 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:10.885294 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.800 [INFO][10590] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16527 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.800 [INFO][10590] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.834 [INFO][10602] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" HandleID="k8s-pod-network.e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.835 [INFO][10602] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" HandleID="k8s-pod-network.e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cd960), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:10.834865473 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.835 [INFO][10602] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.835 [INFO][10602] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.835 [INFO][10602] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.848 [INFO][10602] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" host="172-234-25-215" Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.852 [INFO][10602] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.856 [INFO][10602] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.859 [INFO][10602] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.863 [INFO][10602] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.863 [INFO][10602] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" host="172-234-25-215" Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.865 [INFO][10602] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139 Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.869 [INFO][10602] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" host="172-234-25-215" Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.876 [INFO][10602] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.18/26] block=192.168.62.0/26 handle="k8s-pod-network.e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" host="172-234-25-215" Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.906737 containerd[1567]: 2025-08-13 01:30:10.876 [INFO][10602] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.18/26] handle="k8s-pod-network.e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" host="172-234-25-215" Aug 13 01:30:10.906737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.908478 containerd[1567]: 2025-08-13 01:30:10.876 [INFO][10602] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:10.908478 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.908478 containerd[1567]: 2025-08-13 01:30:10.876 [INFO][10602] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.18/26] IPv6=[] ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" HandleID="k8s-pod-network.e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:10.908478 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.908478 containerd[1567]: 2025-08-13 01:30:10.878 [INFO][10590] cni-plugin/k8s.go 418: Populated endpoint ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16527", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"d23a2e527d57d49b0605b4345afe0cf71b5ef2ec3285dd5182e44a3dff8cdca3", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.18/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:10.908478 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.908478 containerd[1567]: 2025-08-13 01:30:10.879 [INFO][10590] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.18/32] ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:10.908478 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.908478 containerd[1567]: 2025-08-13 01:30:10.879 [INFO][10590] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:10.908478 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.908478 containerd[1567]: 2025-08-13 01:30:10.885 [INFO][10590] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:10.908478 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.908478 containerd[1567]: 2025-08-13 01:30:10.887 [INFO][10590] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16527", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.18/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"1e:7a:98:57:ed:03", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:10.908478 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.911707 containerd[1567]: 2025-08-13 01:30:10.897 [INFO][10590] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:10.911707 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:10.959341 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:10.959350 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:11.046052 containerd[1567]: 2025-08-13 01:30:10.957 [INFO][10625] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Aug 13 01:30:11.046052 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.046052 containerd[1567]: 2025-08-13 01:30:10.958 [INFO][10625] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" iface="eth0" netns="/var/run/netns/cni-782d3c38-b675-99ed-0d19-0f0a15040962" Aug 13 01:30:11.046052 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.046052 containerd[1567]: 2025-08-13 01:30:10.958 [INFO][10625] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" iface="eth0" netns="/var/run/netns/cni-782d3c38-b675-99ed-0d19-0f0a15040962" Aug 13 01:30:11.046052 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.046052 containerd[1567]: 2025-08-13 01:30:10.965 [INFO][10625] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" after=6.960048ms iface="eth0" netns="/var/run/netns/cni-782d3c38-b675-99ed-0d19-0f0a15040962" Aug 13 01:30:11.046052 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.046052 containerd[1567]: 2025-08-13 01:30:10.965 [INFO][10625] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Aug 13 01:30:11.046052 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.046052 containerd[1567]: 2025-08-13 01:30:10.965 [INFO][10625] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Aug 13 01:30:11.046052 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.046052 containerd[1567]: 2025-08-13 01:30:11.007 [INFO][10634] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" HandleID="k8s-pod-network.e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:11.046052 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.046052 containerd[1567]: 2025-08-13 01:30:11.007 [INFO][10634] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:11.046052 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.046052 containerd[1567]: 2025-08-13 01:30:11.007 [INFO][10634] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:11.046052 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.046052 containerd[1567]: 2025-08-13 01:30:11.040 [INFO][10634] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" HandleID="k8s-pod-network.e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:11.046052 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.046052 containerd[1567]: 2025-08-13 01:30:11.040 [INFO][10634] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" HandleID="k8s-pod-network.e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:11.046052 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.046052 containerd[1567]: 2025-08-13 01:30:11.041 [INFO][10634] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:11.046052 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.046052 containerd[1567]: 2025-08-13 01:30:11.043 [INFO][10625] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139" Aug 13 01:30:11.046052 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.049205 systemd[1]: run-netns-cni\x2d782d3c38\x2db675\x2d99ed\x2d0d19\x2d0f0a15040962.mount: Deactivated successfully. Aug 13 01:30:11.049789 containerd[1567]: time="2025-08-13T01:30:11.049648084Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139-lo: no space left on device" Aug 13 01:30:11.051207 kubelet[2720]: E0813 01:30:11.050771 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139-lo: no space left on device" Aug 13 01:30:11.052041 kubelet[2720]: E0813 01:30:11.051766 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:11.052041 kubelet[2720]: E0813 01:30:11.051821 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:11.052041 kubelet[2720]: E0813 01:30:11.051899 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:11.180883 containerd[1567]: time="2025-08-13T01:30:11.180826631Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:30:11.760474 containerd[1567]: time="2025-08-13T01:30:11.760427275Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:11.859928 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:11.861473 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.793 [INFO][10644] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16602 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.793 [INFO][10644] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.818 [INFO][10657] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" HandleID="k8s-pod-network.19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.818 [INFO][10657] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" HandleID="k8s-pod-network.19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f0a0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:11.818732346 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.819 [INFO][10657] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.819 [INFO][10657] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.819 [INFO][10657] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.828 [INFO][10657] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" host="172-234-25-215" Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.833 [INFO][10657] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.837 [INFO][10657] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.838 [INFO][10657] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.840 [INFO][10657] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.840 [INFO][10657] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" host="172-234-25-215" Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.842 [INFO][10657] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.847 [INFO][10657] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" host="172-234-25-215" Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.853 [INFO][10657] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.19/26] block=192.168.62.0/26 handle="k8s-pod-network.19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" host="172-234-25-215" Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876011 containerd[1567]: 2025-08-13 01:30:11.853 [INFO][10657] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.19/26] handle="k8s-pod-network.19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" host="172-234-25-215" Aug 13 01:30:11.876011 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876878 containerd[1567]: 2025-08-13 01:30:11.853 [INFO][10657] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:11.876878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876878 containerd[1567]: 2025-08-13 01:30:11.853 [INFO][10657] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.19/26] IPv6=[] ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" HandleID="k8s-pod-network.19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:11.876878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876878 containerd[1567]: 2025-08-13 01:30:11.856 [INFO][10644] cni-plugin/k8s.go 418: Populated endpoint ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16602", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e71a099606f88996842929072fbb86385ec1897a840babe47e8b7ce06457c139", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.19/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:11.876878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876878 containerd[1567]: 2025-08-13 01:30:11.856 [INFO][10644] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.19/32] ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:11.876878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876878 containerd[1567]: 2025-08-13 01:30:11.856 [INFO][10644] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:11.876878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876878 containerd[1567]: 2025-08-13 01:30:11.861 [INFO][10644] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:11.876878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.876878 containerd[1567]: 2025-08-13 01:30:11.861 [INFO][10644] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16602", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.19/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"fe:4f:78:99:a0:a9", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:11.876878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.877194 containerd[1567]: 2025-08-13 01:30:11.870 [INFO][10644] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:11.877194 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:11.977289 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:11.977299 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:12.069501 containerd[1567]: 2025-08-13 01:30:11.973 [INFO][10679] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Aug 13 01:30:12.069501 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.069501 containerd[1567]: 2025-08-13 01:30:11.973 [INFO][10679] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" iface="eth0" netns="/var/run/netns/cni-3c563078-f524-2d48-0e99-57ee53e2069e" Aug 13 01:30:12.069501 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.069501 containerd[1567]: 2025-08-13 01:30:11.974 [INFO][10679] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" iface="eth0" netns="/var/run/netns/cni-3c563078-f524-2d48-0e99-57ee53e2069e" Aug 13 01:30:12.069501 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.069501 containerd[1567]: 2025-08-13 01:30:11.985 [INFO][10679] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" after=11.371129ms iface="eth0" netns="/var/run/netns/cni-3c563078-f524-2d48-0e99-57ee53e2069e" Aug 13 01:30:12.069501 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.069501 containerd[1567]: 2025-08-13 01:30:11.985 [INFO][10679] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Aug 13 01:30:12.069501 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.069501 containerd[1567]: 2025-08-13 01:30:11.985 [INFO][10679] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Aug 13 01:30:12.069501 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.069501 containerd[1567]: 2025-08-13 01:30:12.022 [INFO][10690] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" HandleID="k8s-pod-network.19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:12.069501 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.069501 containerd[1567]: 2025-08-13 01:30:12.022 [INFO][10690] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:12.069501 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.069501 containerd[1567]: 2025-08-13 01:30:12.022 [INFO][10690] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:12.069501 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.069501 containerd[1567]: 2025-08-13 01:30:12.062 [INFO][10690] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" HandleID="k8s-pod-network.19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:12.069501 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.069501 containerd[1567]: 2025-08-13 01:30:12.062 [INFO][10690] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" HandleID="k8s-pod-network.19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:12.069501 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.069501 containerd[1567]: 2025-08-13 01:30:12.064 [INFO][10690] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:12.069501 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.069501 containerd[1567]: 2025-08-13 01:30:12.067 [INFO][10679] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d" Aug 13 01:30:12.069501 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.073898 containerd[1567]: time="2025-08-13T01:30:12.073783954Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d-lo: no space left on device" Aug 13 01:30:12.074587 systemd[1]: run-netns-cni\x2d3c563078\x2df524\x2d2d48\x2d0e99\x2d57ee53e2069e.mount: Deactivated successfully. Aug 13 01:30:12.077761 kubelet[2720]: E0813 01:30:12.077439 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d-lo: no space left on device" Aug 13 01:30:12.078639 kubelet[2720]: E0813 01:30:12.078009 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:12.078639 kubelet[2720]: E0813 01:30:12.078035 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:12.078639 kubelet[2720]: E0813 01:30:12.078305 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:12.360455 containerd[1567]: time="2025-08-13T01:30:12.360350272Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:12.762539 containerd[1567]: time="2025-08-13T01:30:12.762283107Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:12.873266 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:12.874798 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.797 [INFO][10700] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16665 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.798 [INFO][10700] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.831 [INFO][10713] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" HandleID="k8s-pod-network.f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.831 [INFO][10713] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" HandleID="k8s-pod-network.f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f210), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:12.831192298 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.831 [INFO][10713] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.831 [INFO][10713] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.831 [INFO][10713] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.839 [INFO][10713] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" host="172-234-25-215" Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.843 [INFO][10713] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.848 [INFO][10713] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.850 [INFO][10713] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.853 [INFO][10713] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.854 [INFO][10713] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" host="172-234-25-215" Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.855 [INFO][10713] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96 Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.860 [INFO][10713] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" host="172-234-25-215" Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.866 [INFO][10713] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.20/26] block=192.168.62.0/26 handle="k8s-pod-network.f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" host="172-234-25-215" Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.892545 containerd[1567]: 2025-08-13 01:30:12.866 [INFO][10713] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.20/26] handle="k8s-pod-network.f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" host="172-234-25-215" Aug 13 01:30:12.892545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.894517 containerd[1567]: 2025-08-13 01:30:12.866 [INFO][10713] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:12.894517 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.894517 containerd[1567]: 2025-08-13 01:30:12.867 [INFO][10713] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.20/26] IPv6=[] ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" HandleID="k8s-pod-network.f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:12.894517 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.894517 containerd[1567]: 2025-08-13 01:30:12.869 [INFO][10700] cni-plugin/k8s.go 418: Populated endpoint ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16665", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"19348db2f3da6761528d2e8536ab6a0a16235fd777f057e23ca6f1bf0fbdc78d", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.20/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:12.894517 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.894517 containerd[1567]: 2025-08-13 01:30:12.869 [INFO][10700] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.20/32] ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:12.894517 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.894517 containerd[1567]: 2025-08-13 01:30:12.869 [INFO][10700] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:12.894517 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.894517 containerd[1567]: 2025-08-13 01:30:12.873 [INFO][10700] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:12.894517 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.894517 containerd[1567]: 2025-08-13 01:30:12.874 [INFO][10700] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16665", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.20/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"6e:e2:35:87:83:ed", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:12.894517 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.895893 containerd[1567]: 2025-08-13 01:30:12.884 [INFO][10700] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:12.895893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:12.963578 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:12.964312 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:13.048884 containerd[1567]: 2025-08-13 01:30:12.959 [INFO][10737] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Aug 13 01:30:13.048884 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.048884 containerd[1567]: 2025-08-13 01:30:12.960 [INFO][10737] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" iface="eth0" netns="/var/run/netns/cni-989ce0aa-1291-ab0d-9507-8a6659272d20" Aug 13 01:30:13.048884 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.048884 containerd[1567]: 2025-08-13 01:30:12.960 [INFO][10737] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" iface="eth0" netns="/var/run/netns/cni-989ce0aa-1291-ab0d-9507-8a6659272d20" Aug 13 01:30:13.048884 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.048884 containerd[1567]: 2025-08-13 01:30:12.972 [INFO][10737] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" after=12.010378ms iface="eth0" netns="/var/run/netns/cni-989ce0aa-1291-ab0d-9507-8a6659272d20" Aug 13 01:30:13.048884 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.048884 containerd[1567]: 2025-08-13 01:30:12.972 [INFO][10737] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Aug 13 01:30:13.048884 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.048884 containerd[1567]: 2025-08-13 01:30:12.972 [INFO][10737] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Aug 13 01:30:13.048884 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.048884 containerd[1567]: 2025-08-13 01:30:13.011 [INFO][10746] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" HandleID="k8s-pod-network.f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:13.048884 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.048884 containerd[1567]: 2025-08-13 01:30:13.012 [INFO][10746] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:13.048884 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.048884 containerd[1567]: 2025-08-13 01:30:13.012 [INFO][10746] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:13.048884 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.048884 containerd[1567]: 2025-08-13 01:30:13.041 [INFO][10746] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" HandleID="k8s-pod-network.f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:13.048884 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.048884 containerd[1567]: 2025-08-13 01:30:13.041 [INFO][10746] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" HandleID="k8s-pod-network.f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:13.048884 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.048884 containerd[1567]: 2025-08-13 01:30:13.043 [INFO][10746] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:13.048884 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.048884 containerd[1567]: 2025-08-13 01:30:13.045 [INFO][10737] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96" Aug 13 01:30:13.048884 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.054620 containerd[1567]: time="2025-08-13T01:30:13.054532785Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96-lo: no space left on device" Aug 13 01:30:13.054994 kubelet[2720]: E0813 01:30:13.054957 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96-lo: no space left on device" Aug 13 01:30:13.055055 kubelet[2720]: E0813 01:30:13.055014 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:13.055055 kubelet[2720]: E0813 01:30:13.055034 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:13.055119 kubelet[2720]: E0813 01:30:13.055090 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:13.055829 systemd[1]: run-netns-cni\x2d989ce0aa\x2d1291\x2dab0d\x2d9507\x2d8a6659272d20.mount: Deactivated successfully. Aug 13 01:30:13.764506 containerd[1567]: time="2025-08-13T01:30:13.764427368Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:13.922492 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:13.924999 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.806 [INFO][10756] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16741 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.806 [INFO][10756] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.854 [INFO][10770] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" HandleID="k8s-pod-network.ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.857 [INFO][10770] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" HandleID="k8s-pod-network.ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f270), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:13.854527741 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.857 [INFO][10770] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.857 [INFO][10770] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.857 [INFO][10770] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.869 [INFO][10770] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" host="172-234-25-215" Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.873 [INFO][10770] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.883 [INFO][10770] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.885 [INFO][10770] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.888 [INFO][10770] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.888 [INFO][10770] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" host="172-234-25-215" Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.889 [INFO][10770] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775 Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.901 [INFO][10770] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" host="172-234-25-215" Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.911 [INFO][10770] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.21/26] block=192.168.62.0/26 handle="k8s-pod-network.ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" host="172-234-25-215" Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.948766 containerd[1567]: 2025-08-13 01:30:13.912 [INFO][10770] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.21/26] handle="k8s-pod-network.ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" host="172-234-25-215" Aug 13 01:30:13.948766 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.949545 containerd[1567]: 2025-08-13 01:30:13.912 [INFO][10770] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:13.949545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.949545 containerd[1567]: 2025-08-13 01:30:13.912 [INFO][10770] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.21/26] IPv6=[] ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" HandleID="k8s-pod-network.ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:13.949545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.949545 containerd[1567]: 2025-08-13 01:30:13.916 [INFO][10756] cni-plugin/k8s.go 418: Populated endpoint ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16741", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"f737f2222b157176c6ab62f8abab38a5d4b0bdc9028c7554f7fc8d21b148de96", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.21/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:13.949545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.949545 containerd[1567]: 2025-08-13 01:30:13.917 [INFO][10756] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.21/32] ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:13.949545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.949545 containerd[1567]: 2025-08-13 01:30:13.917 [INFO][10756] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:13.949545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.949545 containerd[1567]: 2025-08-13 01:30:13.927 [INFO][10756] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:13.949545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.949545 containerd[1567]: 2025-08-13 01:30:13.928 [INFO][10756] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16741", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.21/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"e2:4f:b1:6b:37:45", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:13.949545 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:13.949923 containerd[1567]: 2025-08-13 01:30:13.943 [INFO][10756] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:13.949923 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.043301 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:14.043562 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:14.207263 containerd[1567]: 2025-08-13 01:30:14.040 [INFO][10791] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Aug 13 01:30:14.207263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.207263 containerd[1567]: 2025-08-13 01:30:14.041 [INFO][10791] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" iface="eth0" netns="/var/run/netns/cni-b1c420b1-5773-9804-1bee-d7f929bb1c58" Aug 13 01:30:14.207263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.207263 containerd[1567]: 2025-08-13 01:30:14.042 [INFO][10791] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" iface="eth0" netns="/var/run/netns/cni-b1c420b1-5773-9804-1bee-d7f929bb1c58" Aug 13 01:30:14.207263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.207263 containerd[1567]: 2025-08-13 01:30:14.052 [INFO][10791] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" after=10.21707ms iface="eth0" netns="/var/run/netns/cni-b1c420b1-5773-9804-1bee-d7f929bb1c58" Aug 13 01:30:14.207263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.207263 containerd[1567]: 2025-08-13 01:30:14.052 [INFO][10791] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Aug 13 01:30:14.207263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.207263 containerd[1567]: 2025-08-13 01:30:14.052 [INFO][10791] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Aug 13 01:30:14.207263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.207263 containerd[1567]: 2025-08-13 01:30:14.149 [INFO][10798] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" HandleID="k8s-pod-network.ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:14.207263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.207263 containerd[1567]: 2025-08-13 01:30:14.149 [INFO][10798] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:14.207263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.207263 containerd[1567]: 2025-08-13 01:30:14.149 [INFO][10798] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:14.207263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.207263 containerd[1567]: 2025-08-13 01:30:14.200 [INFO][10798] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" HandleID="k8s-pod-network.ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:14.207263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.207263 containerd[1567]: 2025-08-13 01:30:14.200 [INFO][10798] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" HandleID="k8s-pod-network.ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:14.207263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.207263 containerd[1567]: 2025-08-13 01:30:14.202 [INFO][10798] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:14.207263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.207263 containerd[1567]: 2025-08-13 01:30:14.204 [INFO][10791] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775" Aug 13 01:30:14.207263 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.212868 containerd[1567]: time="2025-08-13T01:30:14.212788996Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775-lo: no space left on device" Aug 13 01:30:14.213161 kubelet[2720]: E0813 01:30:14.213116 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775-lo: no space left on device" Aug 13 01:30:14.213468 kubelet[2720]: E0813 01:30:14.213216 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:14.213468 kubelet[2720]: E0813 01:30:14.213238 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:14.213468 kubelet[2720]: E0813 01:30:14.213285 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:14.213810 systemd[1]: run-netns-cni\x2db1c420b1\x2d5773\x2d9804\x2d1bee\x2dd7f929bb1c58.mount: Deactivated successfully. Aug 13 01:30:14.766071 containerd[1567]: time="2025-08-13T01:30:14.766021501Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:14.864786 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:14.865012 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.799 [INFO][10813] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16814 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.801 [INFO][10813] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.829 [INFO][10824] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" HandleID="k8s-pod-network.93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.829 [INFO][10824] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" HandleID="k8s-pod-network.93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:14.829301954 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.830 [INFO][10824] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.830 [INFO][10824] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.830 [INFO][10824] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.836 [INFO][10824] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" host="172-234-25-215" Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.839 [INFO][10824] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.843 [INFO][10824] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.845 [INFO][10824] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.847 [INFO][10824] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.847 [INFO][10824] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" host="172-234-25-215" Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.848 [INFO][10824] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.852 [INFO][10824] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" host="172-234-25-215" Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.858 [INFO][10824] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.22/26] block=192.168.62.0/26 handle="k8s-pod-network.93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" host="172-234-25-215" Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.876546 containerd[1567]: 2025-08-13 01:30:14.858 [INFO][10824] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.22/26] handle="k8s-pod-network.93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" host="172-234-25-215" Aug 13 01:30:14.876546 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.878160 containerd[1567]: 2025-08-13 01:30:14.858 [INFO][10824] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:14.878160 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.878160 containerd[1567]: 2025-08-13 01:30:14.858 [INFO][10824] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.22/26] IPv6=[] ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" HandleID="k8s-pod-network.93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:14.878160 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.878160 containerd[1567]: 2025-08-13 01:30:14.861 [INFO][10813] cni-plugin/k8s.go 418: Populated endpoint ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16814", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ab9cad2992c3167539896f27ac5ef1f7de53bfe1b457fcf34960b9e6eb6f8775", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.22/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:14.878160 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.878160 containerd[1567]: 2025-08-13 01:30:14.861 [INFO][10813] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.22/32] ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:14.878160 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.878160 containerd[1567]: 2025-08-13 01:30:14.861 [INFO][10813] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:14.878160 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.878160 containerd[1567]: 2025-08-13 01:30:14.864 [INFO][10813] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:14.878160 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.878160 containerd[1567]: 2025-08-13 01:30:14.865 [INFO][10813] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16814", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.22/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"6a:4c:fb:63:19:2f", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:14.878160 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.878467 containerd[1567]: 2025-08-13 01:30:14.873 [INFO][10813] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:14.878467 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:14.933562 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:14.933571 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:15.021761 containerd[1567]: 2025-08-13 01:30:14.931 [INFO][10848] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Aug 13 01:30:15.021761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.021761 containerd[1567]: 2025-08-13 01:30:14.932 [INFO][10848] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" iface="eth0" netns="/var/run/netns/cni-649ccb31-fada-0ce9-9bd0-c929ddc85cc9" Aug 13 01:30:15.021761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.021761 containerd[1567]: 2025-08-13 01:30:14.932 [INFO][10848] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" iface="eth0" netns="/var/run/netns/cni-649ccb31-fada-0ce9-9bd0-c929ddc85cc9" Aug 13 01:30:15.021761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.021761 containerd[1567]: 2025-08-13 01:30:14.939 [INFO][10848] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" after=7.687036ms iface="eth0" netns="/var/run/netns/cni-649ccb31-fada-0ce9-9bd0-c929ddc85cc9" Aug 13 01:30:15.021761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.021761 containerd[1567]: 2025-08-13 01:30:14.940 [INFO][10848] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Aug 13 01:30:15.021761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.021761 containerd[1567]: 2025-08-13 01:30:14.940 [INFO][10848] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Aug 13 01:30:15.021761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.021761 containerd[1567]: 2025-08-13 01:30:14.977 [INFO][10858] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" HandleID="k8s-pod-network.93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:15.021761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.021761 containerd[1567]: 2025-08-13 01:30:14.977 [INFO][10858] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:15.021761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.021761 containerd[1567]: 2025-08-13 01:30:14.978 [INFO][10858] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:15.021761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.021761 containerd[1567]: 2025-08-13 01:30:15.015 [INFO][10858] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" HandleID="k8s-pod-network.93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:15.021761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.021761 containerd[1567]: 2025-08-13 01:30:15.015 [INFO][10858] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" HandleID="k8s-pod-network.93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:15.021761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.021761 containerd[1567]: 2025-08-13 01:30:15.016 [INFO][10858] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:15.021761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.021761 containerd[1567]: 2025-08-13 01:30:15.019 [INFO][10848] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a" Aug 13 01:30:15.021761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.023939 containerd[1567]: time="2025-08-13T01:30:15.023804924Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a-lo: no space left on device" Aug 13 01:30:15.025385 kubelet[2720]: E0813 01:30:15.024101 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a-lo: no space left on device" Aug 13 01:30:15.025385 kubelet[2720]: E0813 01:30:15.024151 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:15.025385 kubelet[2720]: E0813 01:30:15.024170 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:15.025385 kubelet[2720]: E0813 01:30:15.024211 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:15.028265 systemd[1]: run-netns-cni\x2d649ccb31\x2dfada\x2d0ce9\x2d9bd0\x2dc929ddc85cc9.mount: Deactivated successfully. Aug 13 01:30:15.360601 containerd[1567]: time="2025-08-13T01:30:15.360485210Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:15.729857 systemd[1]: Started sshd@39-172.234.25.215:22-147.75.109.163:39786.service - OpenSSH per-connection server daemon (147.75.109.163:39786). Aug 13 01:30:15.768551 containerd[1567]: time="2025-08-13T01:30:15.768253174Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:15.871968 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:15.872996 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.805 [INFO][10871] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16868 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.805 [INFO][10871] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.830 [INFO][10884] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" HandleID="k8s-pod-network.786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.831 [INFO][10884] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" HandleID="k8s-pod-network.786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024eff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:15.830846408 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.831 [INFO][10884] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.831 [INFO][10884] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.831 [INFO][10884] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.838 [INFO][10884] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" host="172-234-25-215" Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.842 [INFO][10884] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.846 [INFO][10884] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.849 [INFO][10884] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.854 [INFO][10884] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.854 [INFO][10884] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" host="172-234-25-215" Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.856 [INFO][10884] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6 Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.859 [INFO][10884] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" host="172-234-25-215" Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.865 [INFO][10884] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.23/26] block=192.168.62.0/26 handle="k8s-pod-network.786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" host="172-234-25-215" Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.891493 containerd[1567]: 2025-08-13 01:30:15.866 [INFO][10884] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.23/26] handle="k8s-pod-network.786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" host="172-234-25-215" Aug 13 01:30:15.891493 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.893285 containerd[1567]: 2025-08-13 01:30:15.866 [INFO][10884] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:15.893285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.893285 containerd[1567]: 2025-08-13 01:30:15.866 [INFO][10884] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.23/26] IPv6=[] ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" HandleID="k8s-pod-network.786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:15.893285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.893285 containerd[1567]: 2025-08-13 01:30:15.868 [INFO][10871] cni-plugin/k8s.go 418: Populated endpoint ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16868", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"93853586e93c0416ef5b54d2d40a3b40bb070e110b79f42ad109f864d167551a", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.23/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:15.893285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.893285 containerd[1567]: 2025-08-13 01:30:15.868 [INFO][10871] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.23/32] ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:15.893285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.893285 containerd[1567]: 2025-08-13 01:30:15.868 [INFO][10871] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:15.893285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.893285 containerd[1567]: 2025-08-13 01:30:15.872 [INFO][10871] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:15.893285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.893285 containerd[1567]: 2025-08-13 01:30:15.873 [INFO][10871] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16868", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.23/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"72:16:bf:02:8f:2e", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:15.893285 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.893588 containerd[1567]: 2025-08-13 01:30:15.887 [INFO][10871] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:15.893588 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:15.989899 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:15.989908 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:16.077151 sshd[10867]: Accepted publickey for core from 147.75.109.163 port 39786 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:30:16.079649 sshd-session[10867]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:30:16.083779 containerd[1567]: 2025-08-13 01:30:15.981 [INFO][10910] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Aug 13 01:30:16.083779 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.083779 containerd[1567]: 2025-08-13 01:30:15.983 [INFO][10910] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" iface="eth0" netns="/var/run/netns/cni-f58bf5bb-e661-7a51-d3ea-4b072c46c2f9" Aug 13 01:30:16.083779 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.083779 containerd[1567]: 2025-08-13 01:30:15.983 [INFO][10910] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" iface="eth0" netns="/var/run/netns/cni-f58bf5bb-e661-7a51-d3ea-4b072c46c2f9" Aug 13 01:30:16.083779 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.083779 containerd[1567]: 2025-08-13 01:30:15.994 [INFO][10910] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" after=11.743428ms iface="eth0" netns="/var/run/netns/cni-f58bf5bb-e661-7a51-d3ea-4b072c46c2f9" Aug 13 01:30:16.083779 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.083779 containerd[1567]: 2025-08-13 01:30:15.994 [INFO][10910] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Aug 13 01:30:16.083779 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.083779 containerd[1567]: 2025-08-13 01:30:15.995 [INFO][10910] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Aug 13 01:30:16.083779 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.083779 containerd[1567]: 2025-08-13 01:30:16.034 [INFO][10922] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" HandleID="k8s-pod-network.786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:16.083779 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.083779 containerd[1567]: 2025-08-13 01:30:16.035 [INFO][10922] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:16.083779 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.083779 containerd[1567]: 2025-08-13 01:30:16.035 [INFO][10922] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:16.083779 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.083779 containerd[1567]: 2025-08-13 01:30:16.072 [INFO][10922] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" HandleID="k8s-pod-network.786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:16.083779 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.083779 containerd[1567]: 2025-08-13 01:30:16.072 [INFO][10922] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" HandleID="k8s-pod-network.786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:16.083779 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.083779 containerd[1567]: 2025-08-13 01:30:16.074 [INFO][10922] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:16.083779 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.083779 containerd[1567]: 2025-08-13 01:30:16.080 [INFO][10910] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6" Aug 13 01:30:16.083779 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.086768 systemd-logind[1543]: New session 40 of user core. Aug 13 01:30:16.087500 containerd[1567]: time="2025-08-13T01:30:16.087416972Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6-lo: no space left on device" Aug 13 01:30:16.088620 kubelet[2720]: E0813 01:30:16.088544 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6-lo: no space left on device" Aug 13 01:30:16.089317 kubelet[2720]: E0813 01:30:16.088618 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:16.089317 kubelet[2720]: E0813 01:30:16.088637 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:16.089791 systemd[1]: run-netns-cni\x2df58bf5bb\x2de661\x2d7a51\x2dd3ea\x2d4b072c46c2f9.mount: Deactivated successfully. Aug 13 01:30:16.089955 kubelet[2720]: E0813 01:30:16.089473 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:16.098046 systemd[1]: Started session-40.scope - Session 40 of User core. Aug 13 01:30:16.382491 sshd[10930]: Connection closed by 147.75.109.163 port 39786 Aug 13 01:30:16.383187 sshd-session[10867]: pam_unix(sshd:session): session closed for user core Aug 13 01:30:16.388105 systemd[1]: sshd@39-172.234.25.215:22-147.75.109.163:39786.service: Deactivated successfully. Aug 13 01:30:16.390665 systemd[1]: session-40.scope: Deactivated successfully. Aug 13 01:30:16.393917 systemd-logind[1543]: Session 40 logged out. Waiting for processes to exit. Aug 13 01:30:16.395561 systemd-logind[1543]: Removed session 40. Aug 13 01:30:16.770256 containerd[1567]: time="2025-08-13T01:30:16.769970968Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:16.876308 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:16.878045 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.807 [INFO][10942] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16938 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.807 [INFO][10942] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.832 [INFO][10955] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" HandleID="k8s-pod-network.face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.832 [INFO][10955] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" HandleID="k8s-pod-network.face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:16.832273693 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.832 [INFO][10955] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.832 [INFO][10955] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.832 [INFO][10955] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.838 [INFO][10955] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" host="172-234-25-215" Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.841 [INFO][10955] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.847 [INFO][10955] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.850 [INFO][10955] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.855 [INFO][10955] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.856 [INFO][10955] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" host="172-234-25-215" Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.857 [INFO][10955] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.860 [INFO][10955] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" host="172-234-25-215" Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.866 [INFO][10955] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.24/26] block=192.168.62.0/26 handle="k8s-pod-network.face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" host="172-234-25-215" Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.891135 containerd[1567]: 2025-08-13 01:30:16.866 [INFO][10955] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.24/26] handle="k8s-pod-network.face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" host="172-234-25-215" Aug 13 01:30:16.891135 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.893440 containerd[1567]: 2025-08-13 01:30:16.866 [INFO][10955] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:16.893440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.893440 containerd[1567]: 2025-08-13 01:30:16.866 [INFO][10955] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.24/26] IPv6=[] ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" HandleID="k8s-pod-network.face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:16.893440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.893440 containerd[1567]: 2025-08-13 01:30:16.869 [INFO][10942] cni-plugin/k8s.go 418: Populated endpoint ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16938", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"786e494f132af4b764ce025a8cb2443011ebda3f7340d9fafb6ad1d6acef96d6", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.24/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:16.893440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.893440 containerd[1567]: 2025-08-13 01:30:16.870 [INFO][10942] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.24/32] ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:16.893440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.893440 containerd[1567]: 2025-08-13 01:30:16.870 [INFO][10942] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:16.893440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.893440 containerd[1567]: 2025-08-13 01:30:16.873 [INFO][10942] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:16.893440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.893440 containerd[1567]: 2025-08-13 01:30:16.873 [INFO][10942] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16938", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.24/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"5a:22:8e:c1:2d:70", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:16.893440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.894816 containerd[1567]: 2025-08-13 01:30:16.885 [INFO][10942] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:16.894816 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:16.965120 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:16.965132 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:17.052532 containerd[1567]: 2025-08-13 01:30:16.962 [INFO][10978] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Aug 13 01:30:17.052532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.052532 containerd[1567]: 2025-08-13 01:30:16.963 [INFO][10978] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" iface="eth0" netns="/var/run/netns/cni-1cac41be-3865-418d-f61f-44d06c2a404d" Aug 13 01:30:17.052532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.052532 containerd[1567]: 2025-08-13 01:30:16.963 [INFO][10978] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" iface="eth0" netns="/var/run/netns/cni-1cac41be-3865-418d-f61f-44d06c2a404d" Aug 13 01:30:17.052532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.052532 containerd[1567]: 2025-08-13 01:30:16.969 [INFO][10978] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" after=6.145929ms iface="eth0" netns="/var/run/netns/cni-1cac41be-3865-418d-f61f-44d06c2a404d" Aug 13 01:30:17.052532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.052532 containerd[1567]: 2025-08-13 01:30:16.969 [INFO][10978] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Aug 13 01:30:17.052532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.052532 containerd[1567]: 2025-08-13 01:30:16.970 [INFO][10978] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Aug 13 01:30:17.052532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.052532 containerd[1567]: 2025-08-13 01:30:17.008 [INFO][10988] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" HandleID="k8s-pod-network.face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:17.052532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.052532 containerd[1567]: 2025-08-13 01:30:17.008 [INFO][10988] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:17.052532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.052532 containerd[1567]: 2025-08-13 01:30:17.009 [INFO][10988] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:17.052532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.052532 containerd[1567]: 2025-08-13 01:30:17.043 [INFO][10988] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" HandleID="k8s-pod-network.face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:17.052532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.052532 containerd[1567]: 2025-08-13 01:30:17.043 [INFO][10988] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" HandleID="k8s-pod-network.face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:17.052532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.052532 containerd[1567]: 2025-08-13 01:30:17.044 [INFO][10988] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:17.052532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.052532 containerd[1567]: 2025-08-13 01:30:17.049 [INFO][10978] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f" Aug 13 01:30:17.052532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.058105 containerd[1567]: time="2025-08-13T01:30:17.055944868Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f-lo: no space left on device" Aug 13 01:30:17.059532 systemd[1]: run-netns-cni\x2d1cac41be\x2d3865\x2d418d\x2df61f\x2d44d06c2a404d.mount: Deactivated successfully. Aug 13 01:30:17.059983 kubelet[2720]: E0813 01:30:17.059938 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f-lo: no space left on device" Aug 13 01:30:17.060027 kubelet[2720]: E0813 01:30:17.059994 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:17.060027 kubelet[2720]: E0813 01:30:17.060015 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:17.060136 kubelet[2720]: E0813 01:30:17.060066 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:17.074648 kubelet[2720]: E0813 01:30:17.074214 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:30:17.075141 kubelet[2720]: E0813 01:30:17.075121 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:30:17.075408 kubelet[2720]: E0813 01:30:17.075385 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-307916557-_eRC sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-3587324838: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:30:17.076143 kubelet[2720]: E0813 01:30:17.076120 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-294491888-fYh9 sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4258209367: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:30:17.470553 containerd[1567]: time="2025-08-13T01:30:17.470466971Z" level=warning msg="container event discarded" container=7d84ed886c0b1326d79ecfd0bcd1de121c3247f9cff6c203229e46496e35dcdb type=CONTAINER_CREATED_EVENT Aug 13 01:30:17.470553 containerd[1567]: time="2025-08-13T01:30:17.470534571Z" level=warning msg="container event discarded" container=7d84ed886c0b1326d79ecfd0bcd1de121c3247f9cff6c203229e46496e35dcdb type=CONTAINER_STARTED_EVENT Aug 13 01:30:17.491773 containerd[1567]: time="2025-08-13T01:30:17.491740682Z" level=warning msg="container event discarded" container=6df0c80144a1579c42fd874e4fdabc1ad503793034fc23e7d723fce185be0263 type=CONTAINER_CREATED_EVENT Aug 13 01:30:17.491773 containerd[1567]: time="2025-08-13T01:30:17.491765462Z" level=warning msg="container event discarded" container=6df0c80144a1579c42fd874e4fdabc1ad503793034fc23e7d723fce185be0263 type=CONTAINER_STARTED_EVENT Aug 13 01:30:17.491773 containerd[1567]: time="2025-08-13T01:30:17.491774622Z" level=warning msg="container event discarded" container=43b994a67168ff22c568e118bfdf83cbf8755570148c8488e918964ae398e1ce type=CONTAINER_CREATED_EVENT Aug 13 01:30:17.492111 containerd[1567]: time="2025-08-13T01:30:17.491781701Z" level=warning msg="container event discarded" container=43b994a67168ff22c568e118bfdf83cbf8755570148c8488e918964ae398e1ce type=CONTAINER_STARTED_EVENT Aug 13 01:30:17.492111 containerd[1567]: time="2025-08-13T01:30:17.491788311Z" level=warning msg="container event discarded" container=e175264c33fcc3cbfc4ab096894b87893d82a9339dff54de92520cdc084ea674 type=CONTAINER_CREATED_EVENT Aug 13 01:30:17.511011 containerd[1567]: time="2025-08-13T01:30:17.510971775Z" level=warning msg="container event discarded" container=d8abadf258f601795311223ac256947ee2260f53a2d3f5c3b4649ca912a66738 type=CONTAINER_CREATED_EVENT Aug 13 01:30:17.511011 containerd[1567]: time="2025-08-13T01:30:17.511004855Z" level=warning msg="container event discarded" container=b2a7ee63e6d55f3bec9cbaaad91d3d9cd19ee86000685da949918d72785c9953 type=CONTAINER_CREATED_EVENT Aug 13 01:30:17.621888 containerd[1567]: time="2025-08-13T01:30:17.621831451Z" level=warning msg="container event discarded" container=e175264c33fcc3cbfc4ab096894b87893d82a9339dff54de92520cdc084ea674 type=CONTAINER_STARTED_EVENT Aug 13 01:30:17.621888 containerd[1567]: time="2025-08-13T01:30:17.621873880Z" level=warning msg="container event discarded" container=d8abadf258f601795311223ac256947ee2260f53a2d3f5c3b4649ca912a66738 type=CONTAINER_STARTED_EVENT Aug 13 01:30:17.640247 containerd[1567]: time="2025-08-13T01:30:17.640196546Z" level=warning msg="container event discarded" container=b2a7ee63e6d55f3bec9cbaaad91d3d9cd19ee86000685da949918d72785c9953 type=CONTAINER_STARTED_EVENT Aug 13 01:30:17.657278 containerd[1567]: time="2025-08-13T01:30:17.657229905Z" level=info msg="TaskExit event in podsandbox handler container_id:\"05a7e6943f56eff710d015cc4d373aa0238d7506c1cf61f23b36bc5ebcb7ce84\" id:\"c757a0592bedae416db0d0d36b95b098a462cfbd1be0e78fe31bc71f5774be15\" pid:11010 exited_at:{seconds:1755048617 nanos:656816115}" Aug 13 01:30:17.771321 containerd[1567]: time="2025-08-13T01:30:17.771228103Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:17.887188 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:17.888983 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.813 [INFO][11026] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 16996 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.814 [INFO][11026] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.847 [INFO][11035] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" HandleID="k8s-pod-network.c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.847 [INFO][11035] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" HandleID="k8s-pod-network.c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d57a0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:17.847510583 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.847 [INFO][11035] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.848 [INFO][11035] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.848 [INFO][11035] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.856 [INFO][11035] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" host="172-234-25-215" Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.860 [INFO][11035] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.865 [INFO][11035] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.867 [INFO][11035] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.869 [INFO][11035] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.870 [INFO][11035] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" host="172-234-25-215" Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.871 [INFO][11035] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.874 [INFO][11035] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" host="172-234-25-215" Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.880 [INFO][11035] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.25/26] block=192.168.62.0/26 handle="k8s-pod-network.c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" host="172-234-25-215" Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.907770 containerd[1567]: 2025-08-13 01:30:17.880 [INFO][11035] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.25/26] handle="k8s-pod-network.c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" host="172-234-25-215" Aug 13 01:30:17.907770 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.909161 containerd[1567]: 2025-08-13 01:30:17.881 [INFO][11035] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:17.909161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.909161 containerd[1567]: 2025-08-13 01:30:17.881 [INFO][11035] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.25/26] IPv6=[] ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" HandleID="k8s-pod-network.c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:17.909161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.909161 containerd[1567]: 2025-08-13 01:30:17.883 [INFO][11026] cni-plugin/k8s.go 418: Populated endpoint ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16996", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"face25f2a23e0d340f38b5a19fc14e7b64be2a27f62af2100a5df67dc664324f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.25/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:17.909161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.909161 containerd[1567]: 2025-08-13 01:30:17.884 [INFO][11026] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.25/32] ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:17.909161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.909161 containerd[1567]: 2025-08-13 01:30:17.884 [INFO][11026] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:17.909161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.909161 containerd[1567]: 2025-08-13 01:30:17.888 [INFO][11026] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:17.909161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.909161 containerd[1567]: 2025-08-13 01:30:17.889 [INFO][11026] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"16996", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.25/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ce:a7:bd:36:68:ee", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:17.909161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.909469 containerd[1567]: 2025-08-13 01:30:17.897 [INFO][11026] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:17.909469 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:17.989911 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:17.989919 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:18.093655 containerd[1567]: 2025-08-13 01:30:17.987 [INFO][11058] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Aug 13 01:30:18.093655 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.093655 containerd[1567]: 2025-08-13 01:30:17.988 [INFO][11058] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" iface="eth0" netns="/var/run/netns/cni-b90f3dc8-1cdd-7065-a5d8-b299c81a8692" Aug 13 01:30:18.093655 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.093655 containerd[1567]: 2025-08-13 01:30:17.988 [INFO][11058] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" iface="eth0" netns="/var/run/netns/cni-b90f3dc8-1cdd-7065-a5d8-b299c81a8692" Aug 13 01:30:18.093655 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.093655 containerd[1567]: 2025-08-13 01:30:17.995 [INFO][11058] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" after=6.923156ms iface="eth0" netns="/var/run/netns/cni-b90f3dc8-1cdd-7065-a5d8-b299c81a8692" Aug 13 01:30:18.093655 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.093655 containerd[1567]: 2025-08-13 01:30:17.995 [INFO][11058] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Aug 13 01:30:18.093655 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.093655 containerd[1567]: 2025-08-13 01:30:17.995 [INFO][11058] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Aug 13 01:30:18.093655 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.093655 containerd[1567]: 2025-08-13 01:30:18.043 [INFO][11066] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" HandleID="k8s-pod-network.c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:18.093655 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.093655 containerd[1567]: 2025-08-13 01:30:18.043 [INFO][11066] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:18.093655 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.093655 containerd[1567]: 2025-08-13 01:30:18.043 [INFO][11066] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:18.093655 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.093655 containerd[1567]: 2025-08-13 01:30:18.087 [INFO][11066] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" HandleID="k8s-pod-network.c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:18.093655 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.093655 containerd[1567]: 2025-08-13 01:30:18.087 [INFO][11066] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" HandleID="k8s-pod-network.c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:18.093655 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.093655 containerd[1567]: 2025-08-13 01:30:18.089 [INFO][11066] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:18.093655 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.093655 containerd[1567]: 2025-08-13 01:30:18.091 [INFO][11058] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a" Aug 13 01:30:18.093655 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.096630 systemd[1]: run-netns-cni\x2db90f3dc8\x2d1cdd\x2d7065\x2da5d8\x2db299c81a8692.mount: Deactivated successfully. Aug 13 01:30:18.097921 containerd[1567]: time="2025-08-13T01:30:18.097832319Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a-lo: no space left on device" Aug 13 01:30:18.099602 kubelet[2720]: E0813 01:30:18.099546 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a-lo: no space left on device" Aug 13 01:30:18.099876 kubelet[2720]: E0813 01:30:18.099600 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:18.099876 kubelet[2720]: E0813 01:30:18.099656 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:18.099876 kubelet[2720]: E0813 01:30:18.099736 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:18.336044 kubelet[2720]: I0813 01:30:18.336013 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:30:18.336044 kubelet[2720]: I0813 01:30:18.336055 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:30:18.337580 kubelet[2720]: I0813 01:30:18.337541 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:30:18.352379 kubelet[2720]: I0813 01:30:18.352319 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:30:18.352468 kubelet[2720]: I0813 01:30:18.352446 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-b2ds8","kube-system/coredns-674b8bbfcf-jlpx4","calico-system/calico-kube-controllers-b57d47f84-lznkx","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:30:18.352544 kubelet[2720]: E0813 01:30:18.352481 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:30:18.352544 kubelet[2720]: E0813 01:30:18.352490 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:30:18.352544 kubelet[2720]: E0813 01:30:18.352498 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:18.352544 kubelet[2720]: E0813 01:30:18.352509 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:30:18.352544 kubelet[2720]: E0813 01:30:18.352518 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:30:18.352544 kubelet[2720]: E0813 01:30:18.352525 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:30:18.352544 kubelet[2720]: E0813 01:30:18.352533 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:30:18.352544 kubelet[2720]: E0813 01:30:18.352543 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:30:18.352544 kubelet[2720]: E0813 01:30:18.352550 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:30:18.352544 kubelet[2720]: E0813 01:30:18.352559 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:30:18.353500 kubelet[2720]: I0813 01:30:18.352568 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:30:18.360630 containerd[1567]: time="2025-08-13T01:30:18.360563292Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:18.773036 containerd[1567]: time="2025-08-13T01:30:18.772993898Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:18.888769 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:18.891543 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.816 [INFO][11077] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 17067 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.816 [INFO][11077] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.843 [INFO][11091] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" HandleID="k8s-pod-network.9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.844 [INFO][11091] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" HandleID="k8s-pod-network.9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5820), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:18.843651147 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.844 [INFO][11091] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.845 [INFO][11091] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.845 [INFO][11091] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.856 [INFO][11091] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" host="172-234-25-215" Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.862 [INFO][11091] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.866 [INFO][11091] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.868 [INFO][11091] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.871 [INFO][11091] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.871 [INFO][11091] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" host="172-234-25-215" Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.872 [INFO][11091] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.876 [INFO][11091] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" host="172-234-25-215" Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.882 [INFO][11091] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.26/26] block=192.168.62.0/26 handle="k8s-pod-network.9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" host="172-234-25-215" Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910214 containerd[1567]: 2025-08-13 01:30:18.882 [INFO][11091] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.26/26] handle="k8s-pod-network.9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" host="172-234-25-215" Aug 13 01:30:18.910214 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910962 containerd[1567]: 2025-08-13 01:30:18.882 [INFO][11091] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:18.910962 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910962 containerd[1567]: 2025-08-13 01:30:18.882 [INFO][11091] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.26/26] IPv6=[] ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" HandleID="k8s-pod-network.9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:18.910962 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910962 containerd[1567]: 2025-08-13 01:30:18.884 [INFO][11077] cni-plugin/k8s.go 418: Populated endpoint ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17067", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c90d76bd0253e093302fc91f1619c18be8aeeffd2a08f94b49d250be176f609a", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.26/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:18.910962 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910962 containerd[1567]: 2025-08-13 01:30:18.885 [INFO][11077] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.26/32] ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:18.910962 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910962 containerd[1567]: 2025-08-13 01:30:18.885 [INFO][11077] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:18.910962 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910962 containerd[1567]: 2025-08-13 01:30:18.892 [INFO][11077] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:18.910962 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.910962 containerd[1567]: 2025-08-13 01:30:18.893 [INFO][11077] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17067", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.26/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"2a:41:b5:46:53:da", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:18.910962 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.911268 containerd[1567]: 2025-08-13 01:30:18.903 [INFO][11077] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:18.911268 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:18.966614 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:18.966622 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:19.066014 containerd[1567]: 2025-08-13 01:30:18.965 [INFO][11114] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Aug 13 01:30:19.066014 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.066014 containerd[1567]: 2025-08-13 01:30:18.965 [INFO][11114] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" iface="eth0" netns="/var/run/netns/cni-c899828c-3d73-b1cf-12c2-ebe672e553b5" Aug 13 01:30:19.066014 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.066014 containerd[1567]: 2025-08-13 01:30:18.966 [INFO][11114] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" iface="eth0" netns="/var/run/netns/cni-c899828c-3d73-b1cf-12c2-ebe672e553b5" Aug 13 01:30:19.066014 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.066014 containerd[1567]: 2025-08-13 01:30:18.976 [INFO][11114] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" after=10.436201ms iface="eth0" netns="/var/run/netns/cni-c899828c-3d73-b1cf-12c2-ebe672e553b5" Aug 13 01:30:19.066014 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.066014 containerd[1567]: 2025-08-13 01:30:18.977 [INFO][11114] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Aug 13 01:30:19.066014 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.066014 containerd[1567]: 2025-08-13 01:30:18.977 [INFO][11114] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Aug 13 01:30:19.066014 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.066014 containerd[1567]: 2025-08-13 01:30:19.026 [INFO][11124] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" HandleID="k8s-pod-network.9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:19.066014 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.066014 containerd[1567]: 2025-08-13 01:30:19.026 [INFO][11124] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:19.066014 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.066014 containerd[1567]: 2025-08-13 01:30:19.026 [INFO][11124] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:19.066014 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.066014 containerd[1567]: 2025-08-13 01:30:19.059 [INFO][11124] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" HandleID="k8s-pod-network.9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:19.066014 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.066014 containerd[1567]: 2025-08-13 01:30:19.059 [INFO][11124] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" HandleID="k8s-pod-network.9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:19.066014 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.066014 containerd[1567]: 2025-08-13 01:30:19.060 [INFO][11124] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:19.066014 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.066014 containerd[1567]: 2025-08-13 01:30:19.063 [INFO][11114] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b" Aug 13 01:30:19.066014 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.069510 systemd[1]: run-netns-cni\x2dc899828c\x2d3d73\x2db1cf\x2d12c2\x2debe672e553b5.mount: Deactivated successfully. Aug 13 01:30:19.070961 containerd[1567]: time="2025-08-13T01:30:19.070836997Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b-lo: no space left on device" Aug 13 01:30:19.072111 kubelet[2720]: E0813 01:30:19.071790 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b-lo: no space left on device" Aug 13 01:30:19.072111 kubelet[2720]: E0813 01:30:19.071856 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:19.072111 kubelet[2720]: E0813 01:30:19.071877 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:19.072111 kubelet[2720]: E0813 01:30:19.071924 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:19.774612 containerd[1567]: time="2025-08-13T01:30:19.774547863Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:19.889746 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:19.892904 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.816 [INFO][11134] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 17135 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.817 [INFO][11134] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.844 [INFO][11145] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" HandleID="k8s-pod-network.a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.845 [INFO][11145] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" HandleID="k8s-pod-network.a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5010), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:19.844836303 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.845 [INFO][11145] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.845 [INFO][11145] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.845 [INFO][11145] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.853 [INFO][11145] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" host="172-234-25-215" Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.859 [INFO][11145] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.863 [INFO][11145] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.867 [INFO][11145] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.870 [INFO][11145] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.870 [INFO][11145] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" host="172-234-25-215" Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.871 [INFO][11145] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90 Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.874 [INFO][11145] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" host="172-234-25-215" Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.881 [INFO][11145] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.27/26] block=192.168.62.0/26 handle="k8s-pod-network.a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" host="172-234-25-215" Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.912911 containerd[1567]: 2025-08-13 01:30:19.881 [INFO][11145] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.27/26] handle="k8s-pod-network.a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" host="172-234-25-215" Aug 13 01:30:19.912911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.915520 containerd[1567]: 2025-08-13 01:30:19.881 [INFO][11145] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:19.915520 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.915520 containerd[1567]: 2025-08-13 01:30:19.881 [INFO][11145] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.27/26] IPv6=[] ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" HandleID="k8s-pod-network.a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:19.915520 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.915520 containerd[1567]: 2025-08-13 01:30:19.884 [INFO][11134] cni-plugin/k8s.go 418: Populated endpoint ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17135", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"9dc121ebfed1b6854cb14e6921d71dc31a76c8d8ed05e7b7e00f403f76aab92b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.27/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:19.915520 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.915520 containerd[1567]: 2025-08-13 01:30:19.884 [INFO][11134] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.27/32] ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:19.915520 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.915520 containerd[1567]: 2025-08-13 01:30:19.884 [INFO][11134] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:19.915520 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.915520 containerd[1567]: 2025-08-13 01:30:19.891 [INFO][11134] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:19.915520 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.915520 containerd[1567]: 2025-08-13 01:30:19.894 [INFO][11134] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17135", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.27/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"3a:de:11:2a:0a:d0", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:19.915520 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.917936 containerd[1567]: 2025-08-13 01:30:19.905 [INFO][11134] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:19.917936 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:19.997677 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:19.997687 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:20.099413 containerd[1567]: 2025-08-13 01:30:19.994 [INFO][11167] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Aug 13 01:30:20.099413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.099413 containerd[1567]: 2025-08-13 01:30:19.995 [INFO][11167] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" iface="eth0" netns="/var/run/netns/cni-c8fe8f5b-efac-87e8-5373-1b288c6748b1" Aug 13 01:30:20.099413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.099413 containerd[1567]: 2025-08-13 01:30:19.996 [INFO][11167] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" iface="eth0" netns="/var/run/netns/cni-c8fe8f5b-efac-87e8-5373-1b288c6748b1" Aug 13 01:30:20.099413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.099413 containerd[1567]: 2025-08-13 01:30:20.002 [INFO][11167] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" after=6.964638ms iface="eth0" netns="/var/run/netns/cni-c8fe8f5b-efac-87e8-5373-1b288c6748b1" Aug 13 01:30:20.099413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.099413 containerd[1567]: 2025-08-13 01:30:20.003 [INFO][11167] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Aug 13 01:30:20.099413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.099413 containerd[1567]: 2025-08-13 01:30:20.003 [INFO][11167] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Aug 13 01:30:20.099413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.099413 containerd[1567]: 2025-08-13 01:30:20.048 [INFO][11176] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" HandleID="k8s-pod-network.a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:20.099413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.099413 containerd[1567]: 2025-08-13 01:30:20.048 [INFO][11176] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:20.099413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.099413 containerd[1567]: 2025-08-13 01:30:20.048 [INFO][11176] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:20.099413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.099413 containerd[1567]: 2025-08-13 01:30:20.092 [INFO][11176] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" HandleID="k8s-pod-network.a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:20.099413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.099413 containerd[1567]: 2025-08-13 01:30:20.092 [INFO][11176] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" HandleID="k8s-pod-network.a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:20.099413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.099413 containerd[1567]: 2025-08-13 01:30:20.093 [INFO][11176] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:20.099413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.099413 containerd[1567]: 2025-08-13 01:30:20.096 [INFO][11167] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90" Aug 13 01:30:20.099413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.103101 containerd[1567]: time="2025-08-13T01:30:20.103031545Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90-lo: no space left on device" Aug 13 01:30:20.103663 kubelet[2720]: E0813 01:30:20.103451 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90-lo: no space left on device" Aug 13 01:30:20.103663 kubelet[2720]: E0813 01:30:20.103529 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:20.103663 kubelet[2720]: E0813 01:30:20.103549 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:20.104572 kubelet[2720]: E0813 01:30:20.104271 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:20.104200 systemd[1]: run-netns-cni\x2dc8fe8f5b\x2defac\x2d87e8\x2d5373\x2d1b288c6748b1.mount: Deactivated successfully. Aug 13 01:30:20.328500 containerd[1567]: time="2025-08-13T01:30:20.328396058Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:30:20.328500 containerd[1567]: time="2025-08-13T01:30:20.328487118Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:30:20.328640 containerd[1567]: time="2025-08-13T01:30:20.328530267Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:30:20.328640 containerd[1567]: time="2025-08-13T01:30:20.328553587Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:30:20.328640 containerd[1567]: time="2025-08-13T01:30:20.328574927Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:30:20.328640 containerd[1567]: time="2025-08-13T01:30:20.328595287Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:30:20.329532 containerd[1567]: time="2025-08-13T01:30:20.329484226Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:30:20.329532 containerd[1567]: time="2025-08-13T01:30:20.329536226Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:30:20.776815 containerd[1567]: time="2025-08-13T01:30:20.776770998Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:20.893151 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:20.894513 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.823 [INFO][11189] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 17210 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.824 [INFO][11189] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.859 [INFO][11202] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" HandleID="k8s-pod-network.8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.859 [INFO][11202] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" HandleID="k8s-pod-network.8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5080), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:20.859029686 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.859 [INFO][11202] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.859 [INFO][11202] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.859 [INFO][11202] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.865 [INFO][11202] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" host="172-234-25-215" Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.869 [INFO][11202] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.872 [INFO][11202] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.873 [INFO][11202] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.876 [INFO][11202] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.876 [INFO][11202] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" host="172-234-25-215" Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.877 [INFO][11202] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396 Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.880 [INFO][11202] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" host="172-234-25-215" Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.885 [INFO][11202] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.28/26] block=192.168.62.0/26 handle="k8s-pod-network.8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" host="172-234-25-215" Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.913708 containerd[1567]: 2025-08-13 01:30:20.885 [INFO][11202] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.28/26] handle="k8s-pod-network.8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" host="172-234-25-215" Aug 13 01:30:20.913708 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.917111 containerd[1567]: 2025-08-13 01:30:20.885 [INFO][11202] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:20.917111 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.917111 containerd[1567]: 2025-08-13 01:30:20.885 [INFO][11202] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.28/26] IPv6=[] ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" HandleID="k8s-pod-network.8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:20.917111 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.917111 containerd[1567]: 2025-08-13 01:30:20.888 [INFO][11189] cni-plugin/k8s.go 418: Populated endpoint ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17210", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"a826ea7f9635b06986d02d31e0d2a7f7bb4dea2c7288492e86680d6113069b90", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.28/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:20.917111 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.917111 containerd[1567]: 2025-08-13 01:30:20.888 [INFO][11189] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.28/32] ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:20.917111 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.917111 containerd[1567]: 2025-08-13 01:30:20.888 [INFO][11189] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:20.917111 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.917111 containerd[1567]: 2025-08-13 01:30:20.891 [INFO][11189] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:20.917111 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.917111 containerd[1567]: 2025-08-13 01:30:20.892 [INFO][11189] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17210", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.28/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"56:24:20:0c:68:f3", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:20.917111 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.917418 containerd[1567]: 2025-08-13 01:30:20.909 [INFO][11189] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:20.917418 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:20.975921 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:20.976093 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:21.065824 containerd[1567]: 2025-08-13 01:30:20.974 [INFO][11227] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Aug 13 01:30:21.065824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.065824 containerd[1567]: 2025-08-13 01:30:20.974 [INFO][11227] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" iface="eth0" netns="/var/run/netns/cni-10ec74d4-5a87-df23-d03f-88f81b44f9d5" Aug 13 01:30:21.065824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.065824 containerd[1567]: 2025-08-13 01:30:20.974 [INFO][11227] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" iface="eth0" netns="/var/run/netns/cni-10ec74d4-5a87-df23-d03f-88f81b44f9d5" Aug 13 01:30:21.065824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.065824 containerd[1567]: 2025-08-13 01:30:20.980 [INFO][11227] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" after=5.729179ms iface="eth0" netns="/var/run/netns/cni-10ec74d4-5a87-df23-d03f-88f81b44f9d5" Aug 13 01:30:21.065824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.065824 containerd[1567]: 2025-08-13 01:30:20.980 [INFO][11227] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Aug 13 01:30:21.065824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.065824 containerd[1567]: 2025-08-13 01:30:20.980 [INFO][11227] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Aug 13 01:30:21.065824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.065824 containerd[1567]: 2025-08-13 01:30:21.021 [INFO][11236] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" HandleID="k8s-pod-network.8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:21.065824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.065824 containerd[1567]: 2025-08-13 01:30:21.022 [INFO][11236] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:21.065824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.065824 containerd[1567]: 2025-08-13 01:30:21.022 [INFO][11236] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:21.065824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.065824 containerd[1567]: 2025-08-13 01:30:21.055 [INFO][11236] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" HandleID="k8s-pod-network.8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:21.065824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.065824 containerd[1567]: 2025-08-13 01:30:21.055 [INFO][11236] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" HandleID="k8s-pod-network.8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:21.065824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.065824 containerd[1567]: 2025-08-13 01:30:21.059 [INFO][11236] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:21.065824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.065824 containerd[1567]: 2025-08-13 01:30:21.062 [INFO][11227] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396" Aug 13 01:30:21.065824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.067809 systemd[1]: run-netns-cni\x2d10ec74d4\x2d5a87\x2ddf23\x2dd03f\x2d88f81b44f9d5.mount: Deactivated successfully. Aug 13 01:30:21.069461 containerd[1567]: time="2025-08-13T01:30:21.068818717Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396-lo: no space left on device" Aug 13 01:30:21.069807 kubelet[2720]: E0813 01:30:21.069769 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396-lo: no space left on device" Aug 13 01:30:21.069862 kubelet[2720]: E0813 01:30:21.069826 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:21.069862 kubelet[2720]: E0813 01:30:21.069848 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:21.069924 kubelet[2720]: E0813 01:30:21.069900 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:21.360869 containerd[1567]: time="2025-08-13T01:30:21.360772197Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:21.454848 systemd[1]: Started sshd@40-172.234.25.215:22-147.75.109.163:47942.service - OpenSSH per-connection server daemon (147.75.109.163:47942). Aug 13 01:30:21.778020 containerd[1567]: time="2025-08-13T01:30:21.777969835Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:21.811318 sshd[11246]: Accepted publickey for core from 147.75.109.163 port 47942 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:30:21.814887 sshd-session[11246]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:30:21.823201 systemd-logind[1543]: New session 41 of user core. Aug 13 01:30:21.829914 systemd[1]: Started session-41.scope - Session 41 of User core. Aug 13 01:30:21.901929 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:21.904237 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.826 [INFO][11249] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 17276 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.826 [INFO][11249] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.859 [INFO][11262] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" HandleID="k8s-pod-network.97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.859 [INFO][11262] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" HandleID="k8s-pod-network.97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024eff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:21.859616074 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.859 [INFO][11262] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.860 [INFO][11262] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.860 [INFO][11262] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.865 [INFO][11262] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" host="172-234-25-215" Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.869 [INFO][11262] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.873 [INFO][11262] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.875 [INFO][11262] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.877 [INFO][11262] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.877 [INFO][11262] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" host="172-234-25-215" Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.879 [INFO][11262] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79 Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.882 [INFO][11262] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" host="172-234-25-215" Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.894 [INFO][11262] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.29/26] block=192.168.62.0/26 handle="k8s-pod-network.97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" host="172-234-25-215" Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.925682 containerd[1567]: 2025-08-13 01:30:21.894 [INFO][11262] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.29/26] handle="k8s-pod-network.97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" host="172-234-25-215" Aug 13 01:30:21.925682 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.927112 containerd[1567]: 2025-08-13 01:30:21.894 [INFO][11262] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:21.927112 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.927112 containerd[1567]: 2025-08-13 01:30:21.894 [INFO][11262] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.29/26] IPv6=[] ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" HandleID="k8s-pod-network.97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:21.927112 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.927112 containerd[1567]: 2025-08-13 01:30:21.897 [INFO][11249] cni-plugin/k8s.go 418: Populated endpoint ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17276", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"8ddbdce86247c65cf49c57cccc59dda4e5f8b5a9dc354dc9fa9114a9d7eec396", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.29/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:21.927112 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.927112 containerd[1567]: 2025-08-13 01:30:21.897 [INFO][11249] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.29/32] ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:21.927112 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.927112 containerd[1567]: 2025-08-13 01:30:21.897 [INFO][11249] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:21.927112 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.927112 containerd[1567]: 2025-08-13 01:30:21.905 [INFO][11249] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:21.927112 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.927112 containerd[1567]: 2025-08-13 01:30:21.907 [INFO][11249] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17276", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.29/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"5e:40:64:39:80:8b", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:21.927112 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:21.927426 containerd[1567]: 2025-08-13 01:30:21.918 [INFO][11249] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:21.927426 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.020301 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:22.020311 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:22.065447 kubelet[2720]: I0813 01:30:22.064867 2720 image_gc_manager.go:391] "Disk usage on image filesystem is over the high threshold, trying to free bytes down to the low threshold" usage=100 highThreshold=85 amountToFree=411531673 lowThreshold=80 Aug 13 01:30:22.067756 kubelet[2720]: E0813 01:30:22.066195 2720 kubelet.go:1596] "Image garbage collection failed multiple times in a row" err="Failed to garbage collect required amount of images. Attempted to free 411531673 bytes, but only found 0 bytes eligible to free." Aug 13 01:30:22.176543 containerd[1567]: 2025-08-13 01:30:22.015 [INFO][11287] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Aug 13 01:30:22.176543 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.176543 containerd[1567]: 2025-08-13 01:30:22.016 [INFO][11287] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" iface="eth0" netns="/var/run/netns/cni-1223be09-8091-7411-48c9-842befcb4453" Aug 13 01:30:22.176543 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.176543 containerd[1567]: 2025-08-13 01:30:22.016 [INFO][11287] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" iface="eth0" netns="/var/run/netns/cni-1223be09-8091-7411-48c9-842befcb4453" Aug 13 01:30:22.176543 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.176543 containerd[1567]: 2025-08-13 01:30:22.023 [INFO][11287] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" after=7.737236ms iface="eth0" netns="/var/run/netns/cni-1223be09-8091-7411-48c9-842befcb4453" Aug 13 01:30:22.176543 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.176543 containerd[1567]: 2025-08-13 01:30:22.023 [INFO][11287] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Aug 13 01:30:22.176543 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.176543 containerd[1567]: 2025-08-13 01:30:22.024 [INFO][11287] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Aug 13 01:30:22.176543 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.176543 containerd[1567]: 2025-08-13 01:30:22.075 [INFO][11297] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" HandleID="k8s-pod-network.97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:22.176543 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.176543 containerd[1567]: 2025-08-13 01:30:22.077 [INFO][11297] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:22.176543 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.176543 containerd[1567]: 2025-08-13 01:30:22.077 [INFO][11297] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:22.176543 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.176543 containerd[1567]: 2025-08-13 01:30:22.167 [INFO][11297] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" HandleID="k8s-pod-network.97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:22.176543 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.176543 containerd[1567]: 2025-08-13 01:30:22.168 [INFO][11297] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" HandleID="k8s-pod-network.97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:22.176543 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.176543 containerd[1567]: 2025-08-13 01:30:22.171 [INFO][11297] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:22.176543 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.176543 containerd[1567]: 2025-08-13 01:30:22.174 [INFO][11287] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79" Aug 13 01:30:22.176543 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.179846 containerd[1567]: time="2025-08-13T01:30:22.179793941Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79-lo: no space left on device" Aug 13 01:30:22.180135 kubelet[2720]: E0813 01:30:22.180096 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79-lo: no space left on device" Aug 13 01:30:22.180320 kubelet[2720]: E0813 01:30:22.180278 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:22.180389 kubelet[2720]: E0813 01:30:22.180372 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:22.180500 kubelet[2720]: E0813 01:30:22.180476 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:22.181427 systemd[1]: run-netns-cni\x2d1223be09\x2d8091\x2d7411\x2d48c9\x2d842befcb4453.mount: Deactivated successfully. Aug 13 01:30:22.209802 sshd[11263]: Connection closed by 147.75.109.163 port 47942 Aug 13 01:30:22.210452 sshd-session[11246]: pam_unix(sshd:session): session closed for user core Aug 13 01:30:22.215630 systemd-logind[1543]: Session 41 logged out. Waiting for processes to exit. Aug 13 01:30:22.216336 systemd[1]: sshd@40-172.234.25.215:22-147.75.109.163:47942.service: Deactivated successfully. Aug 13 01:30:22.219187 systemd[1]: session-41.scope: Deactivated successfully. Aug 13 01:30:22.220861 systemd-logind[1543]: Removed session 41. Aug 13 01:30:22.779690 containerd[1567]: time="2025-08-13T01:30:22.779644972Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:22.896596 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:22.898281 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.819 [INFO][11319] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 17337 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.819 [INFO][11319] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.852 [INFO][11332] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" HandleID="k8s-pod-network.bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.853 [INFO][11332] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" HandleID="k8s-pod-network.bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f210), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:22.852439027 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.853 [INFO][11332] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.853 [INFO][11332] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.853 [INFO][11332] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.863 [INFO][11332] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" host="172-234-25-215" Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.868 [INFO][11332] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.872 [INFO][11332] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.873 [INFO][11332] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.875 [INFO][11332] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.875 [INFO][11332] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" host="172-234-25-215" Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.877 [INFO][11332] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.880 [INFO][11332] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" host="172-234-25-215" Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.888 [INFO][11332] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.30/26] block=192.168.62.0/26 handle="k8s-pod-network.bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" host="172-234-25-215" Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.918465 containerd[1567]: 2025-08-13 01:30:22.888 [INFO][11332] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.30/26] handle="k8s-pod-network.bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" host="172-234-25-215" Aug 13 01:30:22.918465 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.920372 containerd[1567]: 2025-08-13 01:30:22.888 [INFO][11332] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:22.920372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.920372 containerd[1567]: 2025-08-13 01:30:22.888 [INFO][11332] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.30/26] IPv6=[] ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" HandleID="k8s-pod-network.bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:22.920372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.920372 containerd[1567]: 2025-08-13 01:30:22.891 [INFO][11319] cni-plugin/k8s.go 418: Populated endpoint ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17337", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"97fa0327f1a676e94644e036630ea63962c4ca15bc2c39df668ca23641228b79", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.30/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:22.920372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.920372 containerd[1567]: 2025-08-13 01:30:22.891 [INFO][11319] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.30/32] ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:22.920372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.920372 containerd[1567]: 2025-08-13 01:30:22.891 [INFO][11319] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:22.920372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.920372 containerd[1567]: 2025-08-13 01:30:22.899 [INFO][11319] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:22.920372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.920372 containerd[1567]: 2025-08-13 01:30:22.900 [INFO][11319] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17337", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.30/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"1e:c8:12:7a:2f:67", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:22.920372 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.920962 containerd[1567]: 2025-08-13 01:30:22.913 [INFO][11319] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:22.920962 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:22.984149 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:22.984165 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:23.081558 containerd[1567]: 2025-08-13 01:30:22.981 [INFO][11355] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Aug 13 01:30:23.081558 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.081558 containerd[1567]: 2025-08-13 01:30:22.981 [INFO][11355] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" iface="eth0" netns="/var/run/netns/cni-376c2a9d-e5c0-8015-303f-082542816b72" Aug 13 01:30:23.081558 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.081558 containerd[1567]: 2025-08-13 01:30:22.982 [INFO][11355] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" iface="eth0" netns="/var/run/netns/cni-376c2a9d-e5c0-8015-303f-082542816b72" Aug 13 01:30:23.081558 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.081558 containerd[1567]: 2025-08-13 01:30:22.990 [INFO][11355] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" after=8.655454ms iface="eth0" netns="/var/run/netns/cni-376c2a9d-e5c0-8015-303f-082542816b72" Aug 13 01:30:23.081558 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.081558 containerd[1567]: 2025-08-13 01:30:22.990 [INFO][11355] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Aug 13 01:30:23.081558 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.081558 containerd[1567]: 2025-08-13 01:30:22.990 [INFO][11355] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Aug 13 01:30:23.081558 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.081558 containerd[1567]: 2025-08-13 01:30:23.024 [INFO][11364] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" HandleID="k8s-pod-network.bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:23.081558 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.081558 containerd[1567]: 2025-08-13 01:30:23.024 [INFO][11364] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:23.081558 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.081558 containerd[1567]: 2025-08-13 01:30:23.024 [INFO][11364] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:23.081558 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.081558 containerd[1567]: 2025-08-13 01:30:23.074 [INFO][11364] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" HandleID="k8s-pod-network.bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:23.081558 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.081558 containerd[1567]: 2025-08-13 01:30:23.075 [INFO][11364] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" HandleID="k8s-pod-network.bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:23.081558 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.081558 containerd[1567]: 2025-08-13 01:30:23.076 [INFO][11364] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:23.081558 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.081558 containerd[1567]: 2025-08-13 01:30:23.079 [INFO][11355] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f" Aug 13 01:30:23.081558 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.085951 containerd[1567]: time="2025-08-13T01:30:23.085894655Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f-lo: no space left on device" Aug 13 01:30:23.086298 systemd[1]: run-netns-cni\x2d376c2a9d\x2de5c0\x2d8015\x2d303f\x2d082542816b72.mount: Deactivated successfully. Aug 13 01:30:23.088545 kubelet[2720]: E0813 01:30:23.086709 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f-lo: no space left on device" Aug 13 01:30:23.088545 kubelet[2720]: E0813 01:30:23.087007 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:23.088545 kubelet[2720]: E0813 01:30:23.087037 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:23.088545 kubelet[2720]: E0813 01:30:23.087207 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:23.782016 containerd[1567]: time="2025-08-13T01:30:23.781957837Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:23.886006 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:23.888532 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.820 [INFO][11374] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 17403 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.821 [INFO][11374] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.844 [INFO][11387] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" HandleID="k8s-pod-network.3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.845 [INFO][11387] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" HandleID="k8s-pod-network.3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:23.84492986 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.845 [INFO][11387] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.845 [INFO][11387] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.846 [INFO][11387] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.854 [INFO][11387] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" host="172-234-25-215" Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.859 [INFO][11387] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.866 [INFO][11387] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.867 [INFO][11387] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.869 [INFO][11387] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.869 [INFO][11387] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" host="172-234-25-215" Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.871 [INFO][11387] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936 Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.874 [INFO][11387] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" host="172-234-25-215" Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.879 [INFO][11387] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.31/26] block=192.168.62.0/26 handle="k8s-pod-network.3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" host="172-234-25-215" Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.906675 containerd[1567]: 2025-08-13 01:30:23.879 [INFO][11387] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.31/26] handle="k8s-pod-network.3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" host="172-234-25-215" Aug 13 01:30:23.906675 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.908393 containerd[1567]: 2025-08-13 01:30:23.879 [INFO][11387] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:23.908393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.908393 containerd[1567]: 2025-08-13 01:30:23.879 [INFO][11387] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.31/26] IPv6=[] ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" HandleID="k8s-pod-network.3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:23.908393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.908393 containerd[1567]: 2025-08-13 01:30:23.882 [INFO][11374] cni-plugin/k8s.go 418: Populated endpoint ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17403", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"bfe22029b1672383850122a0d4392d0abe813a7189d254491f4d82a87eb3261f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.31/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:23.908393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.908393 containerd[1567]: 2025-08-13 01:30:23.882 [INFO][11374] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.31/32] ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:23.908393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.908393 containerd[1567]: 2025-08-13 01:30:23.882 [INFO][11374] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:23.908393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.908393 containerd[1567]: 2025-08-13 01:30:23.889 [INFO][11374] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:23.908393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.908393 containerd[1567]: 2025-08-13 01:30:23.889 [INFO][11374] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17403", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.31/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"3a:0d:c8:24:fe:96", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:23.908393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.908704 containerd[1567]: 2025-08-13 01:30:23.899 [INFO][11374] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:23.908704 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:23.976302 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:23.976314 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:24.059707 containerd[1567]: 2025-08-13 01:30:23.974 [INFO][11411] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Aug 13 01:30:24.059707 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.059707 containerd[1567]: 2025-08-13 01:30:23.974 [INFO][11411] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" iface="eth0" netns="/var/run/netns/cni-57ce22b6-ac6a-ac5f-a7fd-53479ebe492d" Aug 13 01:30:24.059707 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.059707 containerd[1567]: 2025-08-13 01:30:23.974 [INFO][11411] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" iface="eth0" netns="/var/run/netns/cni-57ce22b6-ac6a-ac5f-a7fd-53479ebe492d" Aug 13 01:30:24.059707 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.059707 containerd[1567]: 2025-08-13 01:30:23.983 [INFO][11411] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" after=8.573644ms iface="eth0" netns="/var/run/netns/cni-57ce22b6-ac6a-ac5f-a7fd-53479ebe492d" Aug 13 01:30:24.059707 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.059707 containerd[1567]: 2025-08-13 01:30:23.983 [INFO][11411] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Aug 13 01:30:24.059707 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.059707 containerd[1567]: 2025-08-13 01:30:23.983 [INFO][11411] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Aug 13 01:30:24.059707 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.059707 containerd[1567]: 2025-08-13 01:30:24.016 [INFO][11418] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" HandleID="k8s-pod-network.3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:24.059707 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.059707 containerd[1567]: 2025-08-13 01:30:24.017 [INFO][11418] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:24.059707 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.059707 containerd[1567]: 2025-08-13 01:30:24.018 [INFO][11418] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:24.059707 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.059707 containerd[1567]: 2025-08-13 01:30:24.052 [INFO][11418] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" HandleID="k8s-pod-network.3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:24.059707 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.059707 containerd[1567]: 2025-08-13 01:30:24.052 [INFO][11418] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" HandleID="k8s-pod-network.3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:24.059707 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.059707 containerd[1567]: 2025-08-13 01:30:24.053 [INFO][11418] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:24.059707 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.059707 containerd[1567]: 2025-08-13 01:30:24.056 [INFO][11411] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936" Aug 13 01:30:24.059707 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.064375 systemd[1]: run-netns-cni\x2d57ce22b6\x2dac6a\x2dac5f\x2da7fd\x2d53479ebe492d.mount: Deactivated successfully. Aug 13 01:30:24.064752 containerd[1567]: time="2025-08-13T01:30:24.064604614Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936-lo: no space left on device" Aug 13 01:30:24.066368 kubelet[2720]: E0813 01:30:24.066322 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936-lo: no space left on device" Aug 13 01:30:24.066450 kubelet[2720]: E0813 01:30:24.066414 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:24.066450 kubelet[2720]: E0813 01:30:24.066441 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:24.066506 kubelet[2720]: E0813 01:30:24.066484 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:24.148035 containerd[1567]: time="2025-08-13T01:30:24.147956670Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:30:24.361035 containerd[1567]: time="2025-08-13T01:30:24.360920416Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:24.783342 containerd[1567]: time="2025-08-13T01:30:24.783307584Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:24.931992 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:24.933233 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.821 [INFO][11434] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 17472 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.822 [INFO][11434] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.843 [INFO][11445] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" HandleID="k8s-pod-network.c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.843 [INFO][11445] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" HandleID="k8s-pod-network.c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:24.843681952 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.843 [INFO][11445] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.843 [INFO][11445] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.844 [INFO][11445] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.860 [INFO][11445] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" host="172-234-25-215" Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.866 [INFO][11445] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.873 [INFO][11445] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.877 [INFO][11445] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.886 [INFO][11445] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.886 [INFO][11445] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" host="172-234-25-215" Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.887 [INFO][11445] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185 Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.895 [INFO][11445] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" host="172-234-25-215" Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.906 [INFO][11445] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.32/26] block=192.168.62.0/26 handle="k8s-pod-network.c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" host="172-234-25-215" Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948085 containerd[1567]: 2025-08-13 01:30:24.906 [INFO][11445] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.32/26] handle="k8s-pod-network.c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" host="172-234-25-215" Aug 13 01:30:24.948085 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948868 containerd[1567]: 2025-08-13 01:30:24.906 [INFO][11445] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:24.948868 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948868 containerd[1567]: 2025-08-13 01:30:24.906 [INFO][11445] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.32/26] IPv6=[] ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" HandleID="k8s-pod-network.c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:24.948868 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948868 containerd[1567]: 2025-08-13 01:30:24.910 [INFO][11434] cni-plugin/k8s.go 418: Populated endpoint ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17472", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"3f7051e3c7b8715dea5852e71594823f503dde5dd8a9fbf02bef9b70b92c1936", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.32/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:24.948868 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948868 containerd[1567]: 2025-08-13 01:30:24.910 [INFO][11434] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.32/32] ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:24.948868 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948868 containerd[1567]: 2025-08-13 01:30:24.910 [INFO][11434] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:24.948868 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948868 containerd[1567]: 2025-08-13 01:30:24.916 [INFO][11434] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:24.948868 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.948868 containerd[1567]: 2025-08-13 01:30:24.916 [INFO][11434] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17472", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.32/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"1e:75:11:e3:f5:ca", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:24.948868 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:24.949177 containerd[1567]: 2025-08-13 01:30:24.943 [INFO][11434] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:24.949177 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.038403 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:25.038416 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:25.218440 containerd[1567]: 2025-08-13 01:30:25.035 [INFO][11468] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Aug 13 01:30:25.218440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.218440 containerd[1567]: 2025-08-13 01:30:25.036 [INFO][11468] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" iface="eth0" netns="/var/run/netns/cni-05dd0650-0af5-ffb7-d55b-26eb0c497838" Aug 13 01:30:25.218440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.218440 containerd[1567]: 2025-08-13 01:30:25.037 [INFO][11468] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" iface="eth0" netns="/var/run/netns/cni-05dd0650-0af5-ffb7-d55b-26eb0c497838" Aug 13 01:30:25.218440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.218440 containerd[1567]: 2025-08-13 01:30:25.046 [INFO][11468] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" after=9.782812ms iface="eth0" netns="/var/run/netns/cni-05dd0650-0af5-ffb7-d55b-26eb0c497838" Aug 13 01:30:25.218440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.218440 containerd[1567]: 2025-08-13 01:30:25.046 [INFO][11468] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Aug 13 01:30:25.218440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.218440 containerd[1567]: 2025-08-13 01:30:25.047 [INFO][11468] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Aug 13 01:30:25.218440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.218440 containerd[1567]: 2025-08-13 01:30:25.133 [INFO][11476] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" HandleID="k8s-pod-network.c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:25.218440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.218440 containerd[1567]: 2025-08-13 01:30:25.134 [INFO][11476] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:25.218440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.218440 containerd[1567]: 2025-08-13 01:30:25.134 [INFO][11476] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:25.218440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.218440 containerd[1567]: 2025-08-13 01:30:25.207 [INFO][11476] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" HandleID="k8s-pod-network.c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:25.218440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.218440 containerd[1567]: 2025-08-13 01:30:25.207 [INFO][11476] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" HandleID="k8s-pod-network.c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:25.218440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.218440 containerd[1567]: 2025-08-13 01:30:25.209 [INFO][11476] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:25.218440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.218440 containerd[1567]: 2025-08-13 01:30:25.213 [INFO][11468] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185" Aug 13 01:30:25.218440 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.223899 systemd[1]: run-netns-cni\x2d05dd0650\x2d0af5\x2dffb7\x2dd55b\x2d26eb0c497838.mount: Deactivated successfully. Aug 13 01:30:25.225299 containerd[1567]: time="2025-08-13T01:30:25.225233297Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185-lo: no space left on device" Aug 13 01:30:25.227315 kubelet[2720]: E0813 01:30:25.227010 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185-lo: no space left on device" Aug 13 01:30:25.227315 kubelet[2720]: E0813 01:30:25.227062 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:25.227315 kubelet[2720]: E0813 01:30:25.227081 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:25.227315 kubelet[2720]: E0813 01:30:25.227127 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:25.785233 containerd[1567]: time="2025-08-13T01:30:25.785194892Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:25.911052 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:25.911308 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.833 [INFO][11489] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 17536 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.833 [INFO][11489] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.863 [INFO][11503] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" HandleID="k8s-pod-network.bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.864 [INFO][11503] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" HandleID="k8s-pod-network.bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc000332b00), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:25.863869336 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.864 [INFO][11503] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.864 [INFO][11503] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.864 [INFO][11503] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.870 [INFO][11503] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" host="172-234-25-215" Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.875 [INFO][11503] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.880 [INFO][11503] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.882 [INFO][11503] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.884 [INFO][11503] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.884 [INFO][11503] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" host="172-234-25-215" Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.886 [INFO][11503] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.892 [INFO][11503] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" host="172-234-25-215" Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.900 [INFO][11503] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.33/26] block=192.168.62.0/26 handle="k8s-pod-network.bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" host="172-234-25-215" Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.929512 containerd[1567]: 2025-08-13 01:30:25.900 [INFO][11503] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.33/26] handle="k8s-pod-network.bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" host="172-234-25-215" Aug 13 01:30:25.929512 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.930346 containerd[1567]: 2025-08-13 01:30:25.901 [INFO][11503] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:25.930346 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.930346 containerd[1567]: 2025-08-13 01:30:25.901 [INFO][11503] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.33/26] IPv6=[] ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" HandleID="k8s-pod-network.bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:25.930346 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.930346 containerd[1567]: 2025-08-13 01:30:25.905 [INFO][11489] cni-plugin/k8s.go 418: Populated endpoint ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17536", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c9ffbf6e49022570d3b3873d3f5871bb243fee5187cafdefd5265f065c6e0185", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.33/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:25.930346 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.930346 containerd[1567]: 2025-08-13 01:30:25.906 [INFO][11489] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.33/32] ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:25.930346 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.930346 containerd[1567]: 2025-08-13 01:30:25.906 [INFO][11489] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:25.930346 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.930346 containerd[1567]: 2025-08-13 01:30:25.909 [INFO][11489] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:25.930346 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.930346 containerd[1567]: 2025-08-13 01:30:25.910 [INFO][11489] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17536", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.33/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"1a:a5:67:9d:33:94", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:25.930346 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.930649 containerd[1567]: 2025-08-13 01:30:25.924 [INFO][11489] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:25.930649 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:25.987945 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:25.987970 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:26.069800 containerd[1567]: 2025-08-13 01:30:25.986 [INFO][11527] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Aug 13 01:30:26.069800 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.069800 containerd[1567]: 2025-08-13 01:30:25.986 [INFO][11527] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" iface="eth0" netns="/var/run/netns/cni-77885530-8184-4a83-ddc7-7a6c1f8dc683" Aug 13 01:30:26.069800 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.069800 containerd[1567]: 2025-08-13 01:30:25.987 [INFO][11527] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" iface="eth0" netns="/var/run/netns/cni-77885530-8184-4a83-ddc7-7a6c1f8dc683" Aug 13 01:30:26.069800 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.069800 containerd[1567]: 2025-08-13 01:30:25.991 [INFO][11527] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" after=5.047781ms iface="eth0" netns="/var/run/netns/cni-77885530-8184-4a83-ddc7-7a6c1f8dc683" Aug 13 01:30:26.069800 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.069800 containerd[1567]: 2025-08-13 01:30:25.992 [INFO][11527] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Aug 13 01:30:26.069800 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.069800 containerd[1567]: 2025-08-13 01:30:25.992 [INFO][11527] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Aug 13 01:30:26.069800 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.069800 containerd[1567]: 2025-08-13 01:30:26.032 [INFO][11535] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" HandleID="k8s-pod-network.bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:26.069800 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.069800 containerd[1567]: 2025-08-13 01:30:26.032 [INFO][11535] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:26.069800 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.069800 containerd[1567]: 2025-08-13 01:30:26.033 [INFO][11535] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:26.069800 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.069800 containerd[1567]: 2025-08-13 01:30:26.063 [INFO][11535] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" HandleID="k8s-pod-network.bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:26.069800 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.069800 containerd[1567]: 2025-08-13 01:30:26.063 [INFO][11535] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" HandleID="k8s-pod-network.bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:26.069800 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.069800 containerd[1567]: 2025-08-13 01:30:26.065 [INFO][11535] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:26.069800 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.069800 containerd[1567]: 2025-08-13 01:30:26.067 [INFO][11527] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb" Aug 13 01:30:26.069800 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.073659 containerd[1567]: time="2025-08-13T01:30:26.073616648Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb-lo: no space left on device" Aug 13 01:30:26.073879 systemd[1]: run-netns-cni\x2d77885530\x2d8184\x2d4a83\x2dddc7\x2d7a6c1f8dc683.mount: Deactivated successfully. Aug 13 01:30:26.076838 kubelet[2720]: E0813 01:30:26.076443 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb-lo: no space left on device" Aug 13 01:30:26.077031 kubelet[2720]: E0813 01:30:26.076947 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:26.077207 kubelet[2720]: E0813 01:30:26.076975 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:26.077469 kubelet[2720]: E0813 01:30:26.077299 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:26.787652 containerd[1567]: time="2025-08-13T01:30:26.787593618Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:26.915016 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:26.916298 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.833 [INFO][11555] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 17612 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.833 [INFO][11555] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.867 [INFO][11568] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" HandleID="k8s-pod-network.77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.868 [INFO][11568] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" HandleID="k8s-pod-network.77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:26.8678653 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.868 [INFO][11568] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.868 [INFO][11568] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.868 [INFO][11568] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.874 [INFO][11568] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" host="172-234-25-215" Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.883 [INFO][11568] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.887 [INFO][11568] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.890 [INFO][11568] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.892 [INFO][11568] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.892 [INFO][11568] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" host="172-234-25-215" Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.893 [INFO][11568] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43 Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.898 [INFO][11568] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" host="172-234-25-215" Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.905 [INFO][11568] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.34/26] block=192.168.62.0/26 handle="k8s-pod-network.77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" host="172-234-25-215" Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.935775 containerd[1567]: 2025-08-13 01:30:26.906 [INFO][11568] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.34/26] handle="k8s-pod-network.77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" host="172-234-25-215" Aug 13 01:30:26.935775 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.936464 containerd[1567]: 2025-08-13 01:30:26.906 [INFO][11568] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:26.936464 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.936464 containerd[1567]: 2025-08-13 01:30:26.906 [INFO][11568] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.34/26] IPv6=[] ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" HandleID="k8s-pod-network.77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:26.936464 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.936464 containerd[1567]: 2025-08-13 01:30:26.909 [INFO][11555] cni-plugin/k8s.go 418: Populated endpoint ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17612", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"bf446bbd8e07932d9021abdad89187d8d63798b1206d42e46ce636d137cf48fb", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.34/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:26.936464 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.936464 containerd[1567]: 2025-08-13 01:30:26.909 [INFO][11555] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.34/32] ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:26.936464 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.936464 containerd[1567]: 2025-08-13 01:30:26.909 [INFO][11555] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:26.936464 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.936464 containerd[1567]: 2025-08-13 01:30:26.916 [INFO][11555] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:26.936464 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.936464 containerd[1567]: 2025-08-13 01:30:26.917 [INFO][11555] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17612", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.34/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"16:6c:02:61:5e:31", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:26.936464 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:26.936779 containerd[1567]: 2025-08-13 01:30:26.930 [INFO][11555] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:26.936779 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.021929 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:27.021938 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:27.110586 containerd[1567]: 2025-08-13 01:30:27.016 [INFO][11591] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Aug 13 01:30:27.110586 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.110586 containerd[1567]: 2025-08-13 01:30:27.016 [INFO][11591] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" iface="eth0" netns="/var/run/netns/cni-046d30fe-59d3-b12b-ef74-dcbfdd0bec60" Aug 13 01:30:27.110586 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.110586 containerd[1567]: 2025-08-13 01:30:27.016 [INFO][11591] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" iface="eth0" netns="/var/run/netns/cni-046d30fe-59d3-b12b-ef74-dcbfdd0bec60" Aug 13 01:30:27.110586 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.110586 containerd[1567]: 2025-08-13 01:30:27.028 [INFO][11591] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" after=12.402097ms iface="eth0" netns="/var/run/netns/cni-046d30fe-59d3-b12b-ef74-dcbfdd0bec60" Aug 13 01:30:27.110586 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.110586 containerd[1567]: 2025-08-13 01:30:27.028 [INFO][11591] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Aug 13 01:30:27.110586 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.110586 containerd[1567]: 2025-08-13 01:30:27.029 [INFO][11591] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Aug 13 01:30:27.110586 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.110586 containerd[1567]: 2025-08-13 01:30:27.066 [INFO][11602] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" HandleID="k8s-pod-network.77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:27.110586 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.110586 containerd[1567]: 2025-08-13 01:30:27.066 [INFO][11602] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:27.110586 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.110586 containerd[1567]: 2025-08-13 01:30:27.067 [INFO][11602] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:27.110586 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.110586 containerd[1567]: 2025-08-13 01:30:27.100 [INFO][11602] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" HandleID="k8s-pod-network.77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:27.110586 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.110586 containerd[1567]: 2025-08-13 01:30:27.100 [INFO][11602] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" HandleID="k8s-pod-network.77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:27.110586 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.110586 containerd[1567]: 2025-08-13 01:30:27.102 [INFO][11602] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:27.110586 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.110586 containerd[1567]: 2025-08-13 01:30:27.106 [INFO][11591] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43" Aug 13 01:30:27.110586 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.113592 systemd[1]: run-netns-cni\x2d046d30fe\x2d59d3\x2db12b\x2def74\x2ddcbfdd0bec60.mount: Deactivated successfully. Aug 13 01:30:27.117984 containerd[1567]: time="2025-08-13T01:30:27.117877518Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43-lo: no space left on device" Aug 13 01:30:27.118202 kubelet[2720]: E0813 01:30:27.118154 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43-lo: no space left on device" Aug 13 01:30:27.118796 kubelet[2720]: E0813 01:30:27.118471 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:27.118796 kubelet[2720]: E0813 01:30:27.118504 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:27.118944 kubelet[2720]: E0813 01:30:27.118566 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:27.278350 systemd[1]: Started sshd@41-172.234.25.215:22-147.75.109.163:47952.service - OpenSSH per-connection server daemon (147.75.109.163:47952). Aug 13 01:30:27.361171 containerd[1567]: time="2025-08-13T01:30:27.361070088Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:27.620197 sshd[11611]: Accepted publickey for core from 147.75.109.163 port 47952 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:30:27.621959 sshd-session[11611]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:30:27.627004 systemd-logind[1543]: New session 42 of user core. Aug 13 01:30:27.637880 systemd[1]: Started session-42.scope - Session 42 of User core. Aug 13 01:30:27.789307 containerd[1567]: time="2025-08-13T01:30:27.789260547Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:27.933608 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:27.934976 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.836 [INFO][11615] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 17685 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.837 [INFO][11615] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.875 [INFO][11633] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" HandleID="k8s-pod-network.114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.875 [INFO][11633] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" HandleID="k8s-pod-network.114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5a60), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:27.875561267 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.875 [INFO][11633] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.876 [INFO][11633] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.876 [INFO][11633] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.885 [INFO][11633] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" host="172-234-25-215" Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.889 [INFO][11633] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.898 [INFO][11633] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.903 [INFO][11633] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.909 [INFO][11633] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.910 [INFO][11633] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" host="172-234-25-215" Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.914 [INFO][11633] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916 Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.920 [INFO][11633] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" host="172-234-25-215" Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.926 [INFO][11633] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.35/26] block=192.168.62.0/26 handle="k8s-pod-network.114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" host="172-234-25-215" Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.950387 containerd[1567]: 2025-08-13 01:30:27.926 [INFO][11633] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.35/26] handle="k8s-pod-network.114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" host="172-234-25-215" Aug 13 01:30:27.950387 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.951615 containerd[1567]: 2025-08-13 01:30:27.926 [INFO][11633] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:27.951615 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.951615 containerd[1567]: 2025-08-13 01:30:27.926 [INFO][11633] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.35/26] IPv6=[] ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" HandleID="k8s-pod-network.114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:27.951615 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.951615 containerd[1567]: 2025-08-13 01:30:27.929 [INFO][11615] cni-plugin/k8s.go 418: Populated endpoint ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17685", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"77986ebdbc42d9c6f18a8e7f61741a5059787584de6df78f1ab038b0b25cab43", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.35/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:27.951615 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.951615 containerd[1567]: 2025-08-13 01:30:27.929 [INFO][11615] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.35/32] ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:27.951615 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.951615 containerd[1567]: 2025-08-13 01:30:27.929 [INFO][11615] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:27.951615 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.951615 containerd[1567]: 2025-08-13 01:30:27.935 [INFO][11615] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:27.951615 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.951615 containerd[1567]: 2025-08-13 01:30:27.936 [INFO][11615] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17685", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.35/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"c2:dc:39:82:76:e1", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:27.951615 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.953526 containerd[1567]: 2025-08-13 01:30:27.945 [INFO][11615] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:27.953526 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:27.964431 sshd[11613]: Connection closed by 147.75.109.163 port 47952 Aug 13 01:30:27.966316 sshd-session[11611]: pam_unix(sshd:session): session closed for user core Aug 13 01:30:27.972391 systemd-logind[1543]: Session 42 logged out. Waiting for processes to exit. Aug 13 01:30:27.973022 systemd[1]: sshd@41-172.234.25.215:22-147.75.109.163:47952.service: Deactivated successfully. Aug 13 01:30:27.977342 systemd[1]: session-42.scope: Deactivated successfully. Aug 13 01:30:27.980989 containerd[1567]: time="2025-08-13T01:30:27.980918013Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-controller-manager-172-234-25-215_96c354cf60eb29e83ce001f635cc0212/kube-controller-manager/0.log\"" error="write /var/log/pods/kube-system_kube-controller-manager-172-234-25-215_96c354cf60eb29e83ce001f635cc0212/kube-controller-manager/0.log: no space left on device" Aug 13 01:30:27.984882 systemd-logind[1543]: Removed session 42. Aug 13 01:30:28.051119 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:28.051507 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:28.165585 containerd[1567]: 2025-08-13 01:30:28.049 [INFO][11659] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Aug 13 01:30:28.165585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.165585 containerd[1567]: 2025-08-13 01:30:28.049 [INFO][11659] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" iface="eth0" netns="/var/run/netns/cni-991467ae-1747-d289-51d5-463127f93719" Aug 13 01:30:28.165585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.165585 containerd[1567]: 2025-08-13 01:30:28.050 [INFO][11659] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" iface="eth0" netns="/var/run/netns/cni-991467ae-1747-d289-51d5-463127f93719" Aug 13 01:30:28.165585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.165585 containerd[1567]: 2025-08-13 01:30:28.059 [INFO][11659] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" after=10.083421ms iface="eth0" netns="/var/run/netns/cni-991467ae-1747-d289-51d5-463127f93719" Aug 13 01:30:28.165585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.165585 containerd[1567]: 2025-08-13 01:30:28.060 [INFO][11659] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Aug 13 01:30:28.165585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.165585 containerd[1567]: 2025-08-13 01:30:28.060 [INFO][11659] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Aug 13 01:30:28.165585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.165585 containerd[1567]: 2025-08-13 01:30:28.097 [INFO][11670] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" HandleID="k8s-pod-network.114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:28.165585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.165585 containerd[1567]: 2025-08-13 01:30:28.097 [INFO][11670] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:28.165585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.165585 containerd[1567]: 2025-08-13 01:30:28.097 [INFO][11670] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:28.165585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.165585 containerd[1567]: 2025-08-13 01:30:28.152 [INFO][11670] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" HandleID="k8s-pod-network.114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:28.165585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.165585 containerd[1567]: 2025-08-13 01:30:28.153 [INFO][11670] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" HandleID="k8s-pod-network.114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:28.165585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.165585 containerd[1567]: 2025-08-13 01:30:28.156 [INFO][11670] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:28.165585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.165585 containerd[1567]: 2025-08-13 01:30:28.159 [INFO][11659] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916" Aug 13 01:30:28.165585 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.172441 containerd[1567]: time="2025-08-13T01:30:28.170662092Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916-lo: no space left on device" Aug 13 01:30:28.173139 systemd[1]: run-netns-cni\x2d991467ae\x2d1747\x2dd289\x2d51d5\x2d463127f93719.mount: Deactivated successfully. Aug 13 01:30:28.175091 kubelet[2720]: E0813 01:30:28.174345 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916-lo: no space left on device" Aug 13 01:30:28.175091 kubelet[2720]: E0813 01:30:28.174399 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:28.175091 kubelet[2720]: E0813 01:30:28.174422 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:28.175091 kubelet[2720]: E0813 01:30:28.174465 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:28.372697 kubelet[2720]: I0813 01:30:28.372649 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:30:28.372697 kubelet[2720]: I0813 01:30:28.372692 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:30:28.374331 kubelet[2720]: I0813 01:30:28.374309 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:30:28.388114 kubelet[2720]: I0813 01:30:28.388091 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:30:28.388372 kubelet[2720]: I0813 01:30:28.388354 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-kube-controllers-b57d47f84-lznkx","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:30:28.388483 kubelet[2720]: E0813 01:30:28.388470 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:30:28.388545 kubelet[2720]: E0813 01:30:28.388536 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:30:28.388597 kubelet[2720]: E0813 01:30:28.388589 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:28.388646 kubelet[2720]: E0813 01:30:28.388637 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:30:28.388700 kubelet[2720]: E0813 01:30:28.388691 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:30:28.389766 kubelet[2720]: E0813 01:30:28.388801 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:30:28.389766 kubelet[2720]: E0813 01:30:28.388812 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:30:28.389766 kubelet[2720]: E0813 01:30:28.388830 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:30:28.389766 kubelet[2720]: E0813 01:30:28.388837 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:30:28.389766 kubelet[2720]: E0813 01:30:28.388849 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:30:28.389766 kubelet[2720]: I0813 01:30:28.388860 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:30:28.484270 containerd[1567]: time="2025-08-13T01:30:28.484173672Z" level=warning msg="container event discarded" container=23e526e680e61ae7e941cc509509cd39acc7d69ce570bad007f6f667be120c7c type=CONTAINER_CREATED_EVENT Aug 13 01:30:28.484270 containerd[1567]: time="2025-08-13T01:30:28.484240922Z" level=warning msg="container event discarded" container=23e526e680e61ae7e941cc509509cd39acc7d69ce570bad007f6f667be120c7c type=CONTAINER_STARTED_EVENT Aug 13 01:30:28.509526 containerd[1567]: time="2025-08-13T01:30:28.509486345Z" level=warning msg="container event discarded" container=6369622dc3b5e4883e8463ceb04a4d4c703a3350ba099bf1c41d2444aaba6313 type=CONTAINER_CREATED_EVENT Aug 13 01:30:28.589951 containerd[1567]: time="2025-08-13T01:30:28.589892316Z" level=warning msg="container event discarded" container=6369622dc3b5e4883e8463ceb04a4d4c703a3350ba099bf1c41d2444aaba6313 type=CONTAINER_STARTED_EVENT Aug 13 01:30:28.677292 containerd[1567]: time="2025-08-13T01:30:28.677185425Z" level=warning msg="container event discarded" container=19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec type=CONTAINER_CREATED_EVENT Aug 13 01:30:28.677292 containerd[1567]: time="2025-08-13T01:30:28.677231715Z" level=warning msg="container event discarded" container=19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec type=CONTAINER_STARTED_EVENT Aug 13 01:30:28.791513 containerd[1567]: time="2025-08-13T01:30:28.791474254Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:28.908328 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:28.909757 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.838 [INFO][11684] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 17765 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.838 [INFO][11684] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.869 [INFO][11697] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" HandleID="k8s-pod-network.e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.869 [INFO][11697] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" HandleID="k8s-pod-network.e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024efe0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:28.86901413 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.869 [INFO][11697] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.869 [INFO][11697] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.869 [INFO][11697] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.875 [INFO][11697] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" host="172-234-25-215" Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.879 [INFO][11697] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.883 [INFO][11697] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.884 [INFO][11697] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.887 [INFO][11697] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.887 [INFO][11697] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" host="172-234-25-215" Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.888 [INFO][11697] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.892 [INFO][11697] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" host="172-234-25-215" Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.901 [INFO][11697] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.36/26] block=192.168.62.0/26 handle="k8s-pod-network.e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" host="172-234-25-215" Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.928161 containerd[1567]: 2025-08-13 01:30:28.901 [INFO][11697] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.36/26] handle="k8s-pod-network.e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" host="172-234-25-215" Aug 13 01:30:28.928161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.929371 containerd[1567]: 2025-08-13 01:30:28.901 [INFO][11697] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:28.929371 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.929371 containerd[1567]: 2025-08-13 01:30:28.901 [INFO][11697] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.36/26] IPv6=[] ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" HandleID="k8s-pod-network.e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:28.929371 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.929371 containerd[1567]: 2025-08-13 01:30:28.903 [INFO][11684] cni-plugin/k8s.go 418: Populated endpoint ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17765", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"114b56864ed11fe76ded773f8ebddf785df000e0124df14e07ae6fbd4ce1b916", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.36/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:28.929371 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.929371 containerd[1567]: 2025-08-13 01:30:28.903 [INFO][11684] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.36/32] ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:28.929371 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.929371 containerd[1567]: 2025-08-13 01:30:28.903 [INFO][11684] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:28.929371 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.929371 containerd[1567]: 2025-08-13 01:30:28.910 [INFO][11684] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:28.929371 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.929371 containerd[1567]: 2025-08-13 01:30:28.910 [INFO][11684] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17765", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.36/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"76:2d:f3:1d:48:63", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:28.929371 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.929928 containerd[1567]: 2025-08-13 01:30:28.922 [INFO][11684] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:28.929928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:28.978527 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:28.978538 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:29.049101 containerd[1567]: 2025-08-13 01:30:28.976 [INFO][11720] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Aug 13 01:30:29.049101 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.049101 containerd[1567]: 2025-08-13 01:30:28.976 [INFO][11720] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" iface="eth0" netns="/var/run/netns/cni-12fcafb0-9fbc-6be3-e257-6de81f04a0be" Aug 13 01:30:29.049101 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.049101 containerd[1567]: 2025-08-13 01:30:28.977 [INFO][11720] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" iface="eth0" netns="/var/run/netns/cni-12fcafb0-9fbc-6be3-e257-6de81f04a0be" Aug 13 01:30:29.049101 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.049101 containerd[1567]: 2025-08-13 01:30:28.983 [INFO][11720] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" after=7.273757ms iface="eth0" netns="/var/run/netns/cni-12fcafb0-9fbc-6be3-e257-6de81f04a0be" Aug 13 01:30:29.049101 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.049101 containerd[1567]: 2025-08-13 01:30:28.984 [INFO][11720] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Aug 13 01:30:29.049101 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.049101 containerd[1567]: 2025-08-13 01:30:28.984 [INFO][11720] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Aug 13 01:30:29.049101 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.049101 containerd[1567]: 2025-08-13 01:30:29.010 [INFO][11730] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" HandleID="k8s-pod-network.e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:29.049101 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.049101 containerd[1567]: 2025-08-13 01:30:29.011 [INFO][11730] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:29.049101 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.049101 containerd[1567]: 2025-08-13 01:30:29.011 [INFO][11730] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:29.049101 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.049101 containerd[1567]: 2025-08-13 01:30:29.042 [INFO][11730] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" HandleID="k8s-pod-network.e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:29.049101 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.049101 containerd[1567]: 2025-08-13 01:30:29.042 [INFO][11730] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" HandleID="k8s-pod-network.e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:29.049101 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.049101 containerd[1567]: 2025-08-13 01:30:29.043 [INFO][11730] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:29.049101 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.049101 containerd[1567]: 2025-08-13 01:30:29.046 [INFO][11720] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e" Aug 13 01:30:29.049101 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.054827 systemd[1]: run-netns-cni\x2d12fcafb0\x2d9fbc\x2d6be3\x2de257\x2d6de81f04a0be.mount: Deactivated successfully. Aug 13 01:30:29.055446 containerd[1567]: time="2025-08-13T01:30:29.055372746Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e-lo: no space left on device" Aug 13 01:30:29.058202 kubelet[2720]: E0813 01:30:29.058110 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e-lo: no space left on device" Aug 13 01:30:29.058269 kubelet[2720]: E0813 01:30:29.058255 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:29.058315 kubelet[2720]: E0813 01:30:29.058279 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:29.058536 kubelet[2720]: E0813 01:30:29.058495 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:29.073102 kubelet[2720]: E0813 01:30:29.073064 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:30:29.074498 containerd[1567]: time="2025-08-13T01:30:29.074456150Z" level=info msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\"" Aug 13 01:30:29.301145 containerd[1567]: time="2025-08-13T01:30:29.301077931Z" level=error msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\" failed" error="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-299933554-aTh- sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2568257827: no space left on device" Aug 13 01:30:29.301464 containerd[1567]: time="2025-08-13T01:30:29.301130901Z" level=info msg="stop pulling image registry.k8s.io/coredns/coredns:v1.12.0: active requests=0, bytes read=0" Aug 13 01:30:29.301743 kubelet[2720]: E0813 01:30:29.301651 2720 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-299933554-aTh- sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2568257827: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:30:29.302985 kubelet[2720]: E0813 01:30:29.301834 2720 kuberuntime_image.go:42] "Failed to pull image" err="failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-299933554-aTh- sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2568257827: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:30:29.302985 kubelet[2720]: E0813 01:30:29.302369 2720 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:coredns,Image:registry.k8s.io/coredns/coredns:v1.12.0,Command:[],Args:[-conf /etc/coredns/Corefile],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:dns,HostPort:0,ContainerPort:53,Protocol:UDP,HostIP:,},ContainerPort{Name:dns-tcp,HostPort:0,ContainerPort:53,Protocol:TCP,HostIP:,},ContainerPort{Name:metrics,HostPort:0,ContainerPort:9153,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{memory: {{178257920 0} {} 170Mi BinarySI},},Requests:ResourceList{cpu: {{100 -3} {} 100m DecimalSI},memory: {{73400320 0} {} 70Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:config-volume,ReadOnly:true,MountPath:/etc/coredns,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-lncr6,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{0 8080 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:60,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:5,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/ready,Port:{0 8181 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[NET_BIND_SERVICE],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-674b8bbfcf-b2ds8_kube-system(567c4932-37b3-4053-b427-2472bc4c195b): ErrImagePull: failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-299933554-aTh- sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2568257827: no space left on device" logger="UnhandledError" Aug 13 01:30:29.304091 kubelet[2720]: E0813 01:30:29.304051 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ErrImagePull: \"failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-299933554-aTh- sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2568257827: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:30:29.793075 containerd[1567]: time="2025-08-13T01:30:29.793035592Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:29.897170 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:29.899353 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.833 [INFO][11739] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 17823 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.833 [INFO][11739] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.860 [INFO][11753] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" HandleID="k8s-pod-network.d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.860 [INFO][11753] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" HandleID="k8s-pod-network.d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:29.860394648 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.860 [INFO][11753] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.860 [INFO][11753] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.860 [INFO][11753] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.866 [INFO][11753] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" host="172-234-25-215" Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.870 [INFO][11753] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.874 [INFO][11753] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.875 [INFO][11753] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.877 [INFO][11753] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.877 [INFO][11753] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" host="172-234-25-215" Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.879 [INFO][11753] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.883 [INFO][11753] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" host="172-234-25-215" Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.889 [INFO][11753] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.37/26] block=192.168.62.0/26 handle="k8s-pod-network.d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" host="172-234-25-215" Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.923838 containerd[1567]: 2025-08-13 01:30:29.889 [INFO][11753] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.37/26] handle="k8s-pod-network.d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" host="172-234-25-215" Aug 13 01:30:29.923838 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.924979 containerd[1567]: 2025-08-13 01:30:29.889 [INFO][11753] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:29.924979 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.924979 containerd[1567]: 2025-08-13 01:30:29.889 [INFO][11753] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.37/26] IPv6=[] ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" HandleID="k8s-pod-network.d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:29.924979 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.924979 containerd[1567]: 2025-08-13 01:30:29.893 [INFO][11739] cni-plugin/k8s.go 418: Populated endpoint ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17823", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e589e51e6e0673d27e6e09ea566d878e794d97c68f73b8e5fa261381a388e46e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.37/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:29.924979 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.924979 containerd[1567]: 2025-08-13 01:30:29.893 [INFO][11739] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.37/32] ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:29.924979 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.924979 containerd[1567]: 2025-08-13 01:30:29.893 [INFO][11739] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:29.924979 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.924979 containerd[1567]: 2025-08-13 01:30:29.900 [INFO][11739] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:29.924979 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.924979 containerd[1567]: 2025-08-13 01:30:29.901 [INFO][11739] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17823", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.37/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"d6:e2:bd:ae:4f:1d", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:29.924979 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.926119 containerd[1567]: 2025-08-13 01:30:29.913 [INFO][11739] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:29.926119 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:29.985815 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:29.985945 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:30.064293 containerd[1567]: 2025-08-13 01:30:29.981 [INFO][11776] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Aug 13 01:30:30.064293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.064293 containerd[1567]: 2025-08-13 01:30:29.982 [INFO][11776] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" iface="eth0" netns="/var/run/netns/cni-27cb5fb7-119a-03cf-8d9c-9abbde2963b9" Aug 13 01:30:30.064293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.064293 containerd[1567]: 2025-08-13 01:30:29.982 [INFO][11776] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" iface="eth0" netns="/var/run/netns/cni-27cb5fb7-119a-03cf-8d9c-9abbde2963b9" Aug 13 01:30:30.064293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.064293 containerd[1567]: 2025-08-13 01:30:29.989 [INFO][11776] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" after=7.755466ms iface="eth0" netns="/var/run/netns/cni-27cb5fb7-119a-03cf-8d9c-9abbde2963b9" Aug 13 01:30:30.064293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.064293 containerd[1567]: 2025-08-13 01:30:29.989 [INFO][11776] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Aug 13 01:30:30.064293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.064293 containerd[1567]: 2025-08-13 01:30:29.989 [INFO][11776] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Aug 13 01:30:30.064293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.064293 containerd[1567]: 2025-08-13 01:30:30.025 [INFO][11785] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" HandleID="k8s-pod-network.d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:30.064293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.064293 containerd[1567]: 2025-08-13 01:30:30.025 [INFO][11785] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:30.064293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.064293 containerd[1567]: 2025-08-13 01:30:30.025 [INFO][11785] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:30.064293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.064293 containerd[1567]: 2025-08-13 01:30:30.058 [INFO][11785] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" HandleID="k8s-pod-network.d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:30.064293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.064293 containerd[1567]: 2025-08-13 01:30:30.058 [INFO][11785] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" HandleID="k8s-pod-network.d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:30.064293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.064293 containerd[1567]: 2025-08-13 01:30:30.059 [INFO][11785] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:30.064293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.064293 containerd[1567]: 2025-08-13 01:30:30.062 [INFO][11776] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac" Aug 13 01:30:30.064293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.067440 containerd[1567]: time="2025-08-13T01:30:30.067323235Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac-lo: no space left on device" Aug 13 01:30:30.068077 kubelet[2720]: E0813 01:30:30.067968 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac-lo: no space left on device" Aug 13 01:30:30.068077 kubelet[2720]: E0813 01:30:30.068069 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:30.068151 kubelet[2720]: E0813 01:30:30.068096 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:30.068276 kubelet[2720]: E0813 01:30:30.068182 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:30.069013 systemd[1]: run-netns-cni\x2d27cb5fb7\x2d119a\x2d03cf\x2d8d9c\x2d9abbde2963b9.mount: Deactivated successfully. Aug 13 01:30:30.361352 containerd[1567]: time="2025-08-13T01:30:30.361207972Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:30.445756 containerd[1567]: time="2025-08-13T01:30:30.445664646Z" level=warning msg="container event discarded" container=86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382 type=CONTAINER_CREATED_EVENT Aug 13 01:30:30.499981 containerd[1567]: time="2025-08-13T01:30:30.499921336Z" level=warning msg="container event discarded" container=86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382 type=CONTAINER_STARTED_EVENT Aug 13 01:30:30.795400 containerd[1567]: time="2025-08-13T01:30:30.795108581Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:30.915285 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:30.916631 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.834 [INFO][11796] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 17886 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.834 [INFO][11796] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.866 [INFO][11810] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" HandleID="k8s-pod-network.ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.867 [INFO][11810] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" HandleID="k8s-pod-network.ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f1d0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:30.866828638 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.867 [INFO][11810] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.867 [INFO][11810] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.867 [INFO][11810] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.875 [INFO][11810] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" host="172-234-25-215" Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.879 [INFO][11810] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.884 [INFO][11810] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.886 [INFO][11810] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.888 [INFO][11810] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.888 [INFO][11810] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" host="172-234-25-215" Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.889 [INFO][11810] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.893 [INFO][11810] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" host="172-234-25-215" Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.901 [INFO][11810] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.38/26] block=192.168.62.0/26 handle="k8s-pod-network.ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" host="172-234-25-215" Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937009 containerd[1567]: 2025-08-13 01:30:30.901 [INFO][11810] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.38/26] handle="k8s-pod-network.ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" host="172-234-25-215" Aug 13 01:30:30.937009 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937700 containerd[1567]: 2025-08-13 01:30:30.901 [INFO][11810] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:30.937700 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937700 containerd[1567]: 2025-08-13 01:30:30.901 [INFO][11810] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.38/26] IPv6=[] ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" HandleID="k8s-pod-network.ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:30.937700 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937700 containerd[1567]: 2025-08-13 01:30:30.905 [INFO][11796] cni-plugin/k8s.go 418: Populated endpoint ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17886", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"d1d4ab74ad86d4e5e51e5af81f800f38e0f559b38e49f055f2a8c4c71700bbac", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.38/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:30.937700 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937700 containerd[1567]: 2025-08-13 01:30:30.905 [INFO][11796] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.38/32] ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:30.937700 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937700 containerd[1567]: 2025-08-13 01:30:30.905 [INFO][11796] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:30.937700 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937700 containerd[1567]: 2025-08-13 01:30:30.916 [INFO][11796] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:30.937700 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.937700 containerd[1567]: 2025-08-13 01:30:30.917 [INFO][11796] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17886", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.38/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"96:08:35:2f:2b:c2", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:30.937700 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:30.938042 containerd[1567]: 2025-08-13 01:30:30.930 [INFO][11796] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:30.938042 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.007815 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:31.008909 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:31.132878 containerd[1567]: 2025-08-13 01:30:31.003 [INFO][11834] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Aug 13 01:30:31.132878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.132878 containerd[1567]: 2025-08-13 01:30:31.004 [INFO][11834] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" iface="eth0" netns="/var/run/netns/cni-4f1a3b94-58a4-883d-055a-6531db8ae621" Aug 13 01:30:31.132878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.132878 containerd[1567]: 2025-08-13 01:30:31.006 [INFO][11834] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" iface="eth0" netns="/var/run/netns/cni-4f1a3b94-58a4-883d-055a-6531db8ae621" Aug 13 01:30:31.132878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.132878 containerd[1567]: 2025-08-13 01:30:31.017 [INFO][11834] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" after=13.088736ms iface="eth0" netns="/var/run/netns/cni-4f1a3b94-58a4-883d-055a-6531db8ae621" Aug 13 01:30:31.132878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.132878 containerd[1567]: 2025-08-13 01:30:31.018 [INFO][11834] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Aug 13 01:30:31.132878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.132878 containerd[1567]: 2025-08-13 01:30:31.018 [INFO][11834] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Aug 13 01:30:31.132878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.132878 containerd[1567]: 2025-08-13 01:30:31.049 [INFO][11842] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" HandleID="k8s-pod-network.ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:31.132878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.132878 containerd[1567]: 2025-08-13 01:30:31.054 [INFO][11842] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:31.132878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.132878 containerd[1567]: 2025-08-13 01:30:31.054 [INFO][11842] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:31.132878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.132878 containerd[1567]: 2025-08-13 01:30:31.121 [INFO][11842] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" HandleID="k8s-pod-network.ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:31.132878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.132878 containerd[1567]: 2025-08-13 01:30:31.121 [INFO][11842] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" HandleID="k8s-pod-network.ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:31.132878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.132878 containerd[1567]: 2025-08-13 01:30:31.124 [INFO][11842] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:31.132878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.132878 containerd[1567]: 2025-08-13 01:30:31.126 [INFO][11834] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb" Aug 13 01:30:31.132878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.135023 systemd[1]: run-netns-cni\x2d4f1a3b94\x2d58a4\x2d883d\x2d055a\x2d6531db8ae621.mount: Deactivated successfully. Aug 13 01:30:31.137657 containerd[1567]: time="2025-08-13T01:30:31.137310128Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb-lo: no space left on device" Aug 13 01:30:31.137869 kubelet[2720]: E0813 01:30:31.137828 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb-lo: no space left on device" Aug 13 01:30:31.138322 kubelet[2720]: E0813 01:30:31.137884 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:31.138322 kubelet[2720]: E0813 01:30:31.137906 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:31.138322 kubelet[2720]: E0813 01:30:31.137956 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:31.798093 containerd[1567]: time="2025-08-13T01:30:31.798047487Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:31.927960 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:31.928923 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.842 [INFO][11853] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 17962 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.842 [INFO][11853] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.869 [INFO][11864] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" HandleID="k8s-pod-network.e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.869 [INFO][11864] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" HandleID="k8s-pod-network.e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d58f0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:31.869633035 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.870 [INFO][11864] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.870 [INFO][11864] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.870 [INFO][11864] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.876 [INFO][11864] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" host="172-234-25-215" Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.880 [INFO][11864] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.885 [INFO][11864] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.887 [INFO][11864] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.889 [INFO][11864] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.889 [INFO][11864] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" host="172-234-25-215" Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.893 [INFO][11864] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.900 [INFO][11864] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" host="172-234-25-215" Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.913 [INFO][11864] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.39/26] block=192.168.62.0/26 handle="k8s-pod-network.e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" host="172-234-25-215" Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.948807 containerd[1567]: 2025-08-13 01:30:31.913 [INFO][11864] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.39/26] handle="k8s-pod-network.e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" host="172-234-25-215" Aug 13 01:30:31.948807 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.949515 containerd[1567]: 2025-08-13 01:30:31.913 [INFO][11864] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:31.949515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.949515 containerd[1567]: 2025-08-13 01:30:31.913 [INFO][11864] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.39/26] IPv6=[] ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" HandleID="k8s-pod-network.e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:31.949515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.949515 containerd[1567]: 2025-08-13 01:30:31.919 [INFO][11853] cni-plugin/k8s.go 418: Populated endpoint ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17962", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ab219d4d9a4a6a2e3a37401b7de5b1f8b6e1f7ea817bb337bdc626dce23ce3fb", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.39/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:31.949515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.949515 containerd[1567]: 2025-08-13 01:30:31.919 [INFO][11853] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.39/32] ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:31.949515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.949515 containerd[1567]: 2025-08-13 01:30:31.919 [INFO][11853] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:31.949515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.949515 containerd[1567]: 2025-08-13 01:30:31.928 [INFO][11853] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:31.949515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.949515 containerd[1567]: 2025-08-13 01:30:31.928 [INFO][11853] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"17962", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.39/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"b6:a1:eb:f8:c7:51", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:31.949515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:31.949861 containerd[1567]: 2025-08-13 01:30:31.937 [INFO][11853] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:31.949861 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.025425 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:32.025433 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:32.121785 kubelet[2720]: E0813 01:30:32.121577 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:30:32.125551 containerd[1567]: time="2025-08-13T01:30:32.125101252Z" level=info msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\"" Aug 13 01:30:32.140224 containerd[1567]: 2025-08-13 01:30:32.020 [INFO][11888] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Aug 13 01:30:32.140224 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.140224 containerd[1567]: 2025-08-13 01:30:32.021 [INFO][11888] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" iface="eth0" netns="/var/run/netns/cni-5cb55d47-9267-3ee2-a902-625cbcb49c19" Aug 13 01:30:32.140224 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.140224 containerd[1567]: 2025-08-13 01:30:32.022 [INFO][11888] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" iface="eth0" netns="/var/run/netns/cni-5cb55d47-9267-3ee2-a902-625cbcb49c19" Aug 13 01:30:32.140224 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.140224 containerd[1567]: 2025-08-13 01:30:32.026 [INFO][11888] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" after=4.99111ms iface="eth0" netns="/var/run/netns/cni-5cb55d47-9267-3ee2-a902-625cbcb49c19" Aug 13 01:30:32.140224 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.140224 containerd[1567]: 2025-08-13 01:30:32.026 [INFO][11888] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Aug 13 01:30:32.140224 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.140224 containerd[1567]: 2025-08-13 01:30:32.027 [INFO][11888] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Aug 13 01:30:32.140224 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.140224 containerd[1567]: 2025-08-13 01:30:32.067 [INFO][11898] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" HandleID="k8s-pod-network.e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:32.140224 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.140224 containerd[1567]: 2025-08-13 01:30:32.067 [INFO][11898] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:32.140224 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.140224 containerd[1567]: 2025-08-13 01:30:32.067 [INFO][11898] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:32.140224 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.140224 containerd[1567]: 2025-08-13 01:30:32.120 [INFO][11898] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" HandleID="k8s-pod-network.e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:32.140224 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.140224 containerd[1567]: 2025-08-13 01:30:32.120 [INFO][11898] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" HandleID="k8s-pod-network.e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:32.140224 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.140224 containerd[1567]: 2025-08-13 01:30:32.124 [INFO][11898] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:32.140224 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.140224 containerd[1567]: 2025-08-13 01:30:32.137 [INFO][11888] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b" Aug 13 01:30:32.140224 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.142826 containerd[1567]: time="2025-08-13T01:30:32.142772370Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b-lo: no space left on device" Aug 13 01:30:32.143973 kubelet[2720]: E0813 01:30:32.143935 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b-lo: no space left on device" Aug 13 01:30:32.145632 kubelet[2720]: E0813 01:30:32.143986 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:32.145632 kubelet[2720]: E0813 01:30:32.144006 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:32.145632 kubelet[2720]: E0813 01:30:32.144049 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:32.145694 systemd[1]: run-netns-cni\x2d5cb55d47\x2d9267\x2d3ee2\x2da902\x2d625cbcb49c19.mount: Deactivated successfully. Aug 13 01:30:32.350970 containerd[1567]: time="2025-08-13T01:30:32.350917425Z" level=error msg="PullImage \"registry.k8s.io/coredns/coredns:v1.12.0\" failed" error="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-349524588-hiNR sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2074644924: no space left on device" Aug 13 01:30:32.350970 containerd[1567]: time="2025-08-13T01:30:32.351009935Z" level=info msg="stop pulling image registry.k8s.io/coredns/coredns:v1.12.0: active requests=0, bytes read=0" Aug 13 01:30:32.351335 kubelet[2720]: E0813 01:30:32.351296 2720 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-349524588-hiNR sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2074644924: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:30:32.351405 kubelet[2720]: E0813 01:30:32.351345 2720 kuberuntime_image.go:42] "Failed to pull image" err="failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-349524588-hiNR sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2074644924: no space left on device" image="registry.k8s.io/coredns/coredns:v1.12.0" Aug 13 01:30:32.351516 kubelet[2720]: E0813 01:30:32.351472 2720 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:coredns,Image:registry.k8s.io/coredns/coredns:v1.12.0,Command:[],Args:[-conf /etc/coredns/Corefile],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:dns,HostPort:0,ContainerPort:53,Protocol:UDP,HostIP:,},ContainerPort{Name:dns-tcp,HostPort:0,ContainerPort:53,Protocol:TCP,HostIP:,},ContainerPort{Name:metrics,HostPort:0,ContainerPort:9153,Protocol:TCP,HostIP:,},},Env:[]EnvVar{},Resources:ResourceRequirements{Limits:ResourceList{memory: {{178257920 0} {} 170Mi BinarySI},},Requests:ResourceList{cpu: {{100 -3} {} 100m DecimalSI},memory: {{73400320 0} {} 70Mi BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:config-volume,ReadOnly:true,MountPath:/etc/coredns,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-jdpj9,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{0 8080 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:60,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:5,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/ready,Port:{0 8181 },Host:,Scheme:HTTP,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:0,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:IfNotPresent,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[NET_BIND_SERVICE],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod coredns-674b8bbfcf-jlpx4_kube-system(e3a3eaf4-0523-46b9-8d19-1303011f36f8): ErrImagePull: failed to pull and unpack image \"registry.k8s.io/coredns/coredns:v1.12.0\": failed to prepare extraction snapshot \"extract-349524588-hiNR sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2074644924: no space left on device" logger="UnhandledError" Aug 13 01:30:32.352829 kubelet[2720]: E0813 01:30:32.352781 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ErrImagePull: \"failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-349524588-hiNR sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2074644924: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:30:32.801647 containerd[1567]: time="2025-08-13T01:30:32.801324073Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:32.910994 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:32.911313 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.841 [INFO][11909] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18038 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.842 [INFO][11909] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.870 [INFO][11922] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" HandleID="k8s-pod-network.c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.870 [INFO][11922] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" HandleID="k8s-pod-network.c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002c5640), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:32.870297896 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.870 [INFO][11922] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.870 [INFO][11922] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.870 [INFO][11922] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.876 [INFO][11922] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" host="172-234-25-215" Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.879 [INFO][11922] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.884 [INFO][11922] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.886 [INFO][11922] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.888 [INFO][11922] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.888 [INFO][11922] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" host="172-234-25-215" Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.890 [INFO][11922] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693 Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.893 [INFO][11922] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" host="172-234-25-215" Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.899 [INFO][11922] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.40/26] block=192.168.62.0/26 handle="k8s-pod-network.c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" host="172-234-25-215" Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.923120 containerd[1567]: 2025-08-13 01:30:32.899 [INFO][11922] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.40/26] handle="k8s-pod-network.c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" host="172-234-25-215" Aug 13 01:30:32.923120 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.924904 containerd[1567]: 2025-08-13 01:30:32.899 [INFO][11922] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:32.924904 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.924904 containerd[1567]: 2025-08-13 01:30:32.899 [INFO][11922] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.40/26] IPv6=[] ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" HandleID="k8s-pod-network.c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:32.924904 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.924904 containerd[1567]: 2025-08-13 01:30:32.903 [INFO][11909] cni-plugin/k8s.go 418: Populated endpoint ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18038", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e5f74c63cd481e4bf81790f38bc1196b32fc7e6dd8c670ab352762c51a92b08b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.40/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:32.924904 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.924904 containerd[1567]: 2025-08-13 01:30:32.903 [INFO][11909] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.40/32] ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:32.924904 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.924904 containerd[1567]: 2025-08-13 01:30:32.903 [INFO][11909] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:32.924904 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.924904 containerd[1567]: 2025-08-13 01:30:32.911 [INFO][11909] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:32.924904 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.924904 containerd[1567]: 2025-08-13 01:30:32.911 [INFO][11909] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18038", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.40/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"02:8d:83:97:a6:bf", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:32.924904 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.925317 containerd[1567]: 2025-08-13 01:30:32.918 [INFO][11909] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:32.925317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:32.981963 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:32.982950 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:33.022793 systemd[1]: Started sshd@42-172.234.25.215:22-147.75.109.163:47338.service - OpenSSH per-connection server daemon (147.75.109.163:47338). Aug 13 01:30:33.073788 kubelet[2720]: E0813 01:30:33.073622 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:30:33.075883 containerd[1567]: 2025-08-13 01:30:32.979 [INFO][11946] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Aug 13 01:30:33.075883 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.075883 containerd[1567]: 2025-08-13 01:30:32.979 [INFO][11946] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" iface="eth0" netns="/var/run/netns/cni-4b3c3786-9266-5eb2-e2d2-fa5277afdb10" Aug 13 01:30:33.075883 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.075883 containerd[1567]: 2025-08-13 01:30:32.980 [INFO][11946] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" iface="eth0" netns="/var/run/netns/cni-4b3c3786-9266-5eb2-e2d2-fa5277afdb10" Aug 13 01:30:33.075883 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.075883 containerd[1567]: 2025-08-13 01:30:32.986 [INFO][11946] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" after=6.018889ms iface="eth0" netns="/var/run/netns/cni-4b3c3786-9266-5eb2-e2d2-fa5277afdb10" Aug 13 01:30:33.075883 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.075883 containerd[1567]: 2025-08-13 01:30:32.986 [INFO][11946] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Aug 13 01:30:33.075883 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.075883 containerd[1567]: 2025-08-13 01:30:32.986 [INFO][11946] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Aug 13 01:30:33.075883 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.075883 containerd[1567]: 2025-08-13 01:30:33.018 [INFO][11954] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" HandleID="k8s-pod-network.c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:33.075883 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.075883 containerd[1567]: 2025-08-13 01:30:33.019 [INFO][11954] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:33.075883 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.075883 containerd[1567]: 2025-08-13 01:30:33.019 [INFO][11954] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:33.075883 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.075883 containerd[1567]: 2025-08-13 01:30:33.068 [INFO][11954] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" HandleID="k8s-pod-network.c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:33.075883 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.075883 containerd[1567]: 2025-08-13 01:30:33.068 [INFO][11954] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" HandleID="k8s-pod-network.c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:33.075883 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.075883 containerd[1567]: 2025-08-13 01:30:33.070 [INFO][11954] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:33.075883 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.075883 containerd[1567]: 2025-08-13 01:30:33.072 [INFO][11946] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693" Aug 13 01:30:33.075883 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.078860 containerd[1567]: time="2025-08-13T01:30:33.078814981Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693-lo: no space left on device" Aug 13 01:30:33.079857 systemd[1]: run-netns-cni\x2d4b3c3786\x2d9266\x2d5eb2\x2de2d2\x2dfa5277afdb10.mount: Deactivated successfully. Aug 13 01:30:33.080506 kubelet[2720]: E0813 01:30:33.080459 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693-lo: no space left on device" Aug 13 01:30:33.080558 kubelet[2720]: E0813 01:30:33.080522 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:33.080558 kubelet[2720]: E0813 01:30:33.080541 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:33.081812 kubelet[2720]: E0813 01:30:33.080573 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:33.360966 sshd[11964]: Accepted publickey for core from 147.75.109.163 port 47338 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:30:33.361838 containerd[1567]: time="2025-08-13T01:30:33.361729278Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:33.362059 sshd-session[11964]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:30:33.366810 systemd-logind[1543]: New session 43 of user core. Aug 13 01:30:33.377942 systemd[1]: Started session-43.scope - Session 43 of User core. Aug 13 01:30:33.672121 sshd[11966]: Connection closed by 147.75.109.163 port 47338 Aug 13 01:30:33.673084 sshd-session[11964]: pam_unix(sshd:session): session closed for user core Aug 13 01:30:33.677428 systemd-logind[1543]: Session 43 logged out. Waiting for processes to exit. Aug 13 01:30:33.678368 systemd[1]: sshd@42-172.234.25.215:22-147.75.109.163:47338.service: Deactivated successfully. Aug 13 01:30:33.680841 systemd[1]: session-43.scope: Deactivated successfully. Aug 13 01:30:33.682765 systemd-logind[1543]: Removed session 43. Aug 13 01:30:33.804398 containerd[1567]: time="2025-08-13T01:30:33.804345841Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:33.915750 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:33.916841 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.842 [INFO][11977] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18101 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.843 [INFO][11977] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.873 [INFO][11990] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" HandleID="k8s-pod-network.fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.874 [INFO][11990] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" HandleID="k8s-pod-network.fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f210), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:33.873845782 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.874 [INFO][11990] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.874 [INFO][11990] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.874 [INFO][11990] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.879 [INFO][11990] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" host="172-234-25-215" Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.883 [INFO][11990] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.887 [INFO][11990] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.888 [INFO][11990] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.891 [INFO][11990] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.891 [INFO][11990] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" host="172-234-25-215" Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.893 [INFO][11990] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5 Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.898 [INFO][11990] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" host="172-234-25-215" Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.905 [INFO][11990] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.41/26] block=192.168.62.0/26 handle="k8s-pod-network.fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" host="172-234-25-215" Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.938830 containerd[1567]: 2025-08-13 01:30:33.905 [INFO][11990] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.41/26] handle="k8s-pod-network.fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" host="172-234-25-215" Aug 13 01:30:33.938830 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.939533 containerd[1567]: 2025-08-13 01:30:33.905 [INFO][11990] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:33.939533 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.939533 containerd[1567]: 2025-08-13 01:30:33.905 [INFO][11990] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.41/26] IPv6=[] ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" HandleID="k8s-pod-network.fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:33.939533 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.939533 containerd[1567]: 2025-08-13 01:30:33.909 [INFO][11977] cni-plugin/k8s.go 418: Populated endpoint ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18101", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c1abc6661fd45fa2b425b9e4b93007b5baf396ea1ce5cb2d037ac8f40bcbe693", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.41/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:33.939533 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.939533 containerd[1567]: 2025-08-13 01:30:33.910 [INFO][11977] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.41/32] ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:33.939533 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.939533 containerd[1567]: 2025-08-13 01:30:33.910 [INFO][11977] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:33.939533 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.939533 containerd[1567]: 2025-08-13 01:30:33.917 [INFO][11977] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:33.939533 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.939533 containerd[1567]: 2025-08-13 01:30:33.917 [INFO][11977] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18101", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.41/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"f2:6c:57:44:23:6e", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:33.939533 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:33.940628 containerd[1567]: 2025-08-13 01:30:33.928 [INFO][11977] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:33.940628 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.004550 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:34.004563 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:34.097047 containerd[1567]: 2025-08-13 01:30:33.995 [INFO][12014] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Aug 13 01:30:34.097047 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.097047 containerd[1567]: 2025-08-13 01:30:33.999 [INFO][12014] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" iface="eth0" netns="/var/run/netns/cni-f16691ed-f8e6-8c83-4ca0-bc12529cb861" Aug 13 01:30:34.097047 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.097047 containerd[1567]: 2025-08-13 01:30:34.000 [INFO][12014] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" iface="eth0" netns="/var/run/netns/cni-f16691ed-f8e6-8c83-4ca0-bc12529cb861" Aug 13 01:30:34.097047 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.097047 containerd[1567]: 2025-08-13 01:30:34.011 [INFO][12014] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" after=10.813301ms iface="eth0" netns="/var/run/netns/cni-f16691ed-f8e6-8c83-4ca0-bc12529cb861" Aug 13 01:30:34.097047 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.097047 containerd[1567]: 2025-08-13 01:30:34.011 [INFO][12014] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Aug 13 01:30:34.097047 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.097047 containerd[1567]: 2025-08-13 01:30:34.011 [INFO][12014] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Aug 13 01:30:34.097047 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.097047 containerd[1567]: 2025-08-13 01:30:34.054 [INFO][12024] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" HandleID="k8s-pod-network.fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:34.097047 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.097047 containerd[1567]: 2025-08-13 01:30:34.054 [INFO][12024] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:34.097047 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.097047 containerd[1567]: 2025-08-13 01:30:34.054 [INFO][12024] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:34.097047 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.097047 containerd[1567]: 2025-08-13 01:30:34.089 [INFO][12024] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" HandleID="k8s-pod-network.fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:34.097047 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.097047 containerd[1567]: 2025-08-13 01:30:34.089 [INFO][12024] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" HandleID="k8s-pod-network.fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:34.097047 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.097047 containerd[1567]: 2025-08-13 01:30:34.090 [INFO][12024] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:34.097047 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.097047 containerd[1567]: 2025-08-13 01:30:34.093 [INFO][12014] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5" Aug 13 01:30:34.097047 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.100694 containerd[1567]: time="2025-08-13T01:30:34.100609814Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5-lo: no space left on device" Aug 13 01:30:34.100989 systemd[1]: run-netns-cni\x2df16691ed\x2df8e6\x2d8c83\x2d4ca0\x2dbc12529cb861.mount: Deactivated successfully. Aug 13 01:30:34.101448 kubelet[2720]: E0813 01:30:34.101405 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5-lo: no space left on device" Aug 13 01:30:34.101704 kubelet[2720]: E0813 01:30:34.101454 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:34.101704 kubelet[2720]: E0813 01:30:34.101478 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:34.101704 kubelet[2720]: E0813 01:30:34.101519 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:34.805980 containerd[1567]: time="2025-08-13T01:30:34.805937391Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:34.916852 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:34.919236 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.842 [INFO][12035] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18173 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.842 [INFO][12035] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.876 [INFO][12047] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" HandleID="k8s-pod-network.ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.876 [INFO][12047] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" HandleID="k8s-pod-network.ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4f70), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:34.876058942 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.876 [INFO][12047] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.876 [INFO][12047] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.876 [INFO][12047] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.882 [INFO][12047] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" host="172-234-25-215" Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.886 [INFO][12047] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.890 [INFO][12047] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.892 [INFO][12047] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.896 [INFO][12047] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.896 [INFO][12047] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" host="172-234-25-215" Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.898 [INFO][12047] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.903 [INFO][12047] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" host="172-234-25-215" Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.910 [INFO][12047] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.42/26] block=192.168.62.0/26 handle="k8s-pod-network.ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" host="172-234-25-215" Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.939369 containerd[1567]: 2025-08-13 01:30:34.910 [INFO][12047] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.42/26] handle="k8s-pod-network.ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" host="172-234-25-215" Aug 13 01:30:34.939369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.940252 containerd[1567]: 2025-08-13 01:30:34.910 [INFO][12047] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:34.940252 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.940252 containerd[1567]: 2025-08-13 01:30:34.910 [INFO][12047] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.42/26] IPv6=[] ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" HandleID="k8s-pod-network.ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:34.940252 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.940252 containerd[1567]: 2025-08-13 01:30:34.913 [INFO][12035] cni-plugin/k8s.go 418: Populated endpoint ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18173", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"fe72558174fb1f25a17649c459531064e6fd646438cddfd681bffb79ae2319b5", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.42/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:34.940252 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.940252 containerd[1567]: 2025-08-13 01:30:34.913 [INFO][12035] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.42/32] ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:34.940252 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.940252 containerd[1567]: 2025-08-13 01:30:34.913 [INFO][12035] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:34.940252 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.940252 containerd[1567]: 2025-08-13 01:30:34.915 [INFO][12035] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:34.940252 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.940252 containerd[1567]: 2025-08-13 01:30:34.916 [INFO][12035] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18173", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.42/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"42:4a:5c:f0:69:50", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:34.940252 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:34.943235 containerd[1567]: 2025-08-13 01:30:34.930 [INFO][12035] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:34.943235 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.033333 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:35.034465 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:35.125513 containerd[1567]: 2025-08-13 01:30:35.029 [INFO][12072] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Aug 13 01:30:35.125513 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.125513 containerd[1567]: 2025-08-13 01:30:35.031 [INFO][12072] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" iface="eth0" netns="/var/run/netns/cni-63851df2-e935-606c-0672-322263543344" Aug 13 01:30:35.125513 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.125513 containerd[1567]: 2025-08-13 01:30:35.031 [INFO][12072] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" iface="eth0" netns="/var/run/netns/cni-63851df2-e935-606c-0672-322263543344" Aug 13 01:30:35.125513 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.125513 containerd[1567]: 2025-08-13 01:30:35.040 [INFO][12072] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" after=8.973063ms iface="eth0" netns="/var/run/netns/cni-63851df2-e935-606c-0672-322263543344" Aug 13 01:30:35.125513 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.125513 containerd[1567]: 2025-08-13 01:30:35.040 [INFO][12072] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Aug 13 01:30:35.125513 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.125513 containerd[1567]: 2025-08-13 01:30:35.040 [INFO][12072] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Aug 13 01:30:35.125513 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.125513 containerd[1567]: 2025-08-13 01:30:35.074 [INFO][12083] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" HandleID="k8s-pod-network.ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:35.125513 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.125513 containerd[1567]: 2025-08-13 01:30:35.075 [INFO][12083] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:35.125513 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.125513 containerd[1567]: 2025-08-13 01:30:35.075 [INFO][12083] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:35.125513 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.125513 containerd[1567]: 2025-08-13 01:30:35.118 [INFO][12083] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" HandleID="k8s-pod-network.ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:35.125513 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.125513 containerd[1567]: 2025-08-13 01:30:35.118 [INFO][12083] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" HandleID="k8s-pod-network.ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:35.125513 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.125513 containerd[1567]: 2025-08-13 01:30:35.120 [INFO][12083] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:35.125513 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.125513 containerd[1567]: 2025-08-13 01:30:35.122 [INFO][12072] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a" Aug 13 01:30:35.125513 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.130056 systemd[1]: run-netns-cni\x2d63851df2\x2de935\x2d606c\x2d0672\x2d322263543344.mount: Deactivated successfully. Aug 13 01:30:35.130869 containerd[1567]: time="2025-08-13T01:30:35.130814471Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a-lo: no space left on device" Aug 13 01:30:35.131682 kubelet[2720]: E0813 01:30:35.131638 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a-lo: no space left on device" Aug 13 01:30:35.131959 kubelet[2720]: E0813 01:30:35.131694 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:35.131959 kubelet[2720]: E0813 01:30:35.131744 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:35.131959 kubelet[2720]: E0813 01:30:35.131797 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:35.809149 containerd[1567]: time="2025-08-13T01:30:35.809105429Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:35.920199 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:35.921347 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.845 [INFO][12094] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18233 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.846 [INFO][12094] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.877 [INFO][12105] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" HandleID="k8s-pod-network.3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.877 [INFO][12105] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" HandleID="k8s-pod-network.3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:35.876999053 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.877 [INFO][12105] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.877 [INFO][12105] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.877 [INFO][12105] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.883 [INFO][12105] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" host="172-234-25-215" Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.887 [INFO][12105] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.890 [INFO][12105] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.891 [INFO][12105] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.894 [INFO][12105] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.894 [INFO][12105] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" host="172-234-25-215" Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.897 [INFO][12105] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95 Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.905 [INFO][12105] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" host="172-234-25-215" Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.911 [INFO][12105] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.43/26] block=192.168.62.0/26 handle="k8s-pod-network.3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" host="172-234-25-215" Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.941610 containerd[1567]: 2025-08-13 01:30:35.911 [INFO][12105] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.43/26] handle="k8s-pod-network.3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" host="172-234-25-215" Aug 13 01:30:35.941610 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.942919 containerd[1567]: 2025-08-13 01:30:35.911 [INFO][12105] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:35.942919 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.942919 containerd[1567]: 2025-08-13 01:30:35.911 [INFO][12105] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.43/26] IPv6=[] ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" HandleID="k8s-pod-network.3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:35.942919 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.942919 containerd[1567]: 2025-08-13 01:30:35.914 [INFO][12094] cni-plugin/k8s.go 418: Populated endpoint ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18233", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ced3d1539fc8fc540692997da3c87c4341c4c1b9ee57940d492bf6b6d518cd9a", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.43/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:35.942919 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.942919 containerd[1567]: 2025-08-13 01:30:35.914 [INFO][12094] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.43/32] ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:35.942919 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.942919 containerd[1567]: 2025-08-13 01:30:35.914 [INFO][12094] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:35.942919 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.942919 containerd[1567]: 2025-08-13 01:30:35.920 [INFO][12094] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:35.942919 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.942919 containerd[1567]: 2025-08-13 01:30:35.922 [INFO][12094] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18233", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.43/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"82:b7:d6:67:58:ab", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:35.942919 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:35.943232 containerd[1567]: 2025-08-13 01:30:35.932 [INFO][12094] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:35.943232 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.004070 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:36.004084 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:36.090333 containerd[1567]: 2025-08-13 01:30:35.999 [INFO][12130] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Aug 13 01:30:36.090333 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.090333 containerd[1567]: 2025-08-13 01:30:36.000 [INFO][12130] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" iface="eth0" netns="/var/run/netns/cni-5d7d0711-e3fb-92d6-a8c4-8c4e9e7d0840" Aug 13 01:30:36.090333 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.090333 containerd[1567]: 2025-08-13 01:30:36.001 [INFO][12130] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" iface="eth0" netns="/var/run/netns/cni-5d7d0711-e3fb-92d6-a8c4-8c4e9e7d0840" Aug 13 01:30:36.090333 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.090333 containerd[1567]: 2025-08-13 01:30:36.014 [INFO][12130] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" after=13.658375ms iface="eth0" netns="/var/run/netns/cni-5d7d0711-e3fb-92d6-a8c4-8c4e9e7d0840" Aug 13 01:30:36.090333 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.090333 containerd[1567]: 2025-08-13 01:30:36.015 [INFO][12130] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Aug 13 01:30:36.090333 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.090333 containerd[1567]: 2025-08-13 01:30:36.016 [INFO][12130] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Aug 13 01:30:36.090333 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.090333 containerd[1567]: 2025-08-13 01:30:36.043 [INFO][12138] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" HandleID="k8s-pod-network.3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:36.090333 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.090333 containerd[1567]: 2025-08-13 01:30:36.043 [INFO][12138] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:36.090333 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.090333 containerd[1567]: 2025-08-13 01:30:36.043 [INFO][12138] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:36.090333 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.090333 containerd[1567]: 2025-08-13 01:30:36.084 [INFO][12138] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" HandleID="k8s-pod-network.3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:36.090333 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.090333 containerd[1567]: 2025-08-13 01:30:36.084 [INFO][12138] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" HandleID="k8s-pod-network.3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:36.090333 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.090333 containerd[1567]: 2025-08-13 01:30:36.085 [INFO][12138] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:36.090333 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.090333 containerd[1567]: 2025-08-13 01:30:36.087 [INFO][12130] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95" Aug 13 01:30:36.090333 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.093923 containerd[1567]: time="2025-08-13T01:30:36.093875162Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95-lo: no space left on device" Aug 13 01:30:36.094994 kubelet[2720]: E0813 01:30:36.094956 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95-lo: no space left on device" Aug 13 01:30:36.095045 kubelet[2720]: E0813 01:30:36.095004 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:36.095045 kubelet[2720]: E0813 01:30:36.095023 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:36.095100 kubelet[2720]: E0813 01:30:36.095064 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:36.097310 systemd[1]: run-netns-cni\x2d5d7d0711\x2de3fb\x2d92d6\x2da8c4\x2d8c4e9e7d0840.mount: Deactivated successfully. Aug 13 01:30:36.362012 containerd[1567]: time="2025-08-13T01:30:36.361896737Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:36.810457 containerd[1567]: time="2025-08-13T01:30:36.810404430Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:36.933119 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:36.933333 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.859 [INFO][12148] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18306 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.860 [INFO][12148] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.889 [INFO][12161] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" HandleID="k8s-pod-network.d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.889 [INFO][12161] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" HandleID="k8s-pod-network.d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:36.889287974 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.889 [INFO][12161] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.889 [INFO][12161] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.890 [INFO][12161] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.898 [INFO][12161] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" host="172-234-25-215" Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.904 [INFO][12161] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.909 [INFO][12161] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.911 [INFO][12161] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.913 [INFO][12161] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.913 [INFO][12161] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" host="172-234-25-215" Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.915 [INFO][12161] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20 Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.918 [INFO][12161] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" host="172-234-25-215" Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.924 [INFO][12161] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.44/26] block=192.168.62.0/26 handle="k8s-pod-network.d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" host="172-234-25-215" Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.946107 containerd[1567]: 2025-08-13 01:30:36.924 [INFO][12161] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.44/26] handle="k8s-pod-network.d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" host="172-234-25-215" Aug 13 01:30:36.946107 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.949791 containerd[1567]: 2025-08-13 01:30:36.924 [INFO][12161] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:36.949791 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.949791 containerd[1567]: 2025-08-13 01:30:36.924 [INFO][12161] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.44/26] IPv6=[] ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" HandleID="k8s-pod-network.d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:36.949791 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.949791 containerd[1567]: 2025-08-13 01:30:36.928 [INFO][12148] cni-plugin/k8s.go 418: Populated endpoint ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18306", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"3a3d935d8f0bdd5e9d57dbdfd1776687134c42d32abd1a4da1130a7914ae8f95", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.44/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:36.949791 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.949791 containerd[1567]: 2025-08-13 01:30:36.928 [INFO][12148] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.44/32] ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:36.949791 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.949791 containerd[1567]: 2025-08-13 01:30:36.928 [INFO][12148] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:36.949791 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.949791 containerd[1567]: 2025-08-13 01:30:36.931 [INFO][12148] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:36.949791 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.949791 containerd[1567]: 2025-08-13 01:30:36.931 [INFO][12148] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18306", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.44/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"9e:eb:08:9f:6d:15", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:36.949791 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:36.952977 containerd[1567]: 2025-08-13 01:30:36.939 [INFO][12148] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:36.952977 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.049902 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:37.049912 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:37.203253 containerd[1567]: 2025-08-13 01:30:37.044 [INFO][12185] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Aug 13 01:30:37.203253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.203253 containerd[1567]: 2025-08-13 01:30:37.045 [INFO][12185] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" iface="eth0" netns="/var/run/netns/cni-933507d4-50be-dd38-aafe-901efc62309d" Aug 13 01:30:37.203253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.203253 containerd[1567]: 2025-08-13 01:30:37.045 [INFO][12185] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" iface="eth0" netns="/var/run/netns/cni-933507d4-50be-dd38-aafe-901efc62309d" Aug 13 01:30:37.203253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.203253 containerd[1567]: 2025-08-13 01:30:37.070 [INFO][12185] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" after=25.655252ms iface="eth0" netns="/var/run/netns/cni-933507d4-50be-dd38-aafe-901efc62309d" Aug 13 01:30:37.203253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.203253 containerd[1567]: 2025-08-13 01:30:37.071 [INFO][12185] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Aug 13 01:30:37.203253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.203253 containerd[1567]: 2025-08-13 01:30:37.071 [INFO][12185] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Aug 13 01:30:37.203253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.203253 containerd[1567]: 2025-08-13 01:30:37.118 [INFO][12194] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" HandleID="k8s-pod-network.d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:37.203253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.203253 containerd[1567]: 2025-08-13 01:30:37.122 [INFO][12194] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:37.203253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.203253 containerd[1567]: 2025-08-13 01:30:37.122 [INFO][12194] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:37.203253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.203253 containerd[1567]: 2025-08-13 01:30:37.195 [INFO][12194] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" HandleID="k8s-pod-network.d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:37.203253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.203253 containerd[1567]: 2025-08-13 01:30:37.195 [INFO][12194] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" HandleID="k8s-pod-network.d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:37.203253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.203253 containerd[1567]: 2025-08-13 01:30:37.196 [INFO][12194] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:37.203253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.203253 containerd[1567]: 2025-08-13 01:30:37.200 [INFO][12185] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20" Aug 13 01:30:37.203253 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.207486 containerd[1567]: time="2025-08-13T01:30:37.207399327Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20-lo: no space left on device" Aug 13 01:30:37.207922 kubelet[2720]: E0813 01:30:37.207877 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20-lo: no space left on device" Aug 13 01:30:37.208197 kubelet[2720]: E0813 01:30:37.207934 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:37.208197 kubelet[2720]: E0813 01:30:37.207957 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:37.208197 kubelet[2720]: E0813 01:30:37.208000 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:37.208850 systemd[1]: run-netns-cni\x2d933507d4\x2d50be\x2ddd38\x2daafe\x2d901efc62309d.mount: Deactivated successfully. Aug 13 01:30:37.811843 containerd[1567]: time="2025-08-13T01:30:37.811789441Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:37.914086 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:37.915878 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.850 [INFO][12205] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18366 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.850 [INFO][12205] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.876 [INFO][12219] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" HandleID="k8s-pod-network.ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.877 [INFO][12219] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" HandleID="k8s-pod-network.ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024eff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:37.876842681 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.877 [INFO][12219] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.877 [INFO][12219] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.877 [INFO][12219] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.884 [INFO][12219] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" host="172-234-25-215" Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.889 [INFO][12219] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.894 [INFO][12219] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.896 [INFO][12219] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.898 [INFO][12219] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.899 [INFO][12219] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" host="172-234-25-215" Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.900 [INFO][12219] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.902 [INFO][12219] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" host="172-234-25-215" Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.908 [INFO][12219] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.45/26] block=192.168.62.0/26 handle="k8s-pod-network.ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" host="172-234-25-215" Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.927165 containerd[1567]: 2025-08-13 01:30:37.908 [INFO][12219] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.45/26] handle="k8s-pod-network.ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" host="172-234-25-215" Aug 13 01:30:37.927165 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.928202 containerd[1567]: 2025-08-13 01:30:37.908 [INFO][12219] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:37.928202 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.928202 containerd[1567]: 2025-08-13 01:30:37.908 [INFO][12219] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.45/26] IPv6=[] ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" HandleID="k8s-pod-network.ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:37.928202 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.928202 containerd[1567]: 2025-08-13 01:30:37.910 [INFO][12205] cni-plugin/k8s.go 418: Populated endpoint ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18366", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"d9357449d35c3bea304db021b917ee115f9d000ef1bf9f3e770419d26968ad20", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.45/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:37.928202 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.928202 containerd[1567]: 2025-08-13 01:30:37.910 [INFO][12205] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.45/32] ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:37.928202 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.928202 containerd[1567]: 2025-08-13 01:30:37.911 [INFO][12205] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:37.928202 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.928202 containerd[1567]: 2025-08-13 01:30:37.915 [INFO][12205] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:37.928202 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.928202 containerd[1567]: 2025-08-13 01:30:37.916 [INFO][12205] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18366", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.45/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"0e:ff:40:ad:8c:89", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:37.928202 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.928515 containerd[1567]: 2025-08-13 01:30:37.923 [INFO][12205] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:37.928515 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:37.988996 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:37.989005 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:38.071861 containerd[1567]: 2025-08-13 01:30:37.987 [INFO][12241] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Aug 13 01:30:38.071861 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.071861 containerd[1567]: 2025-08-13 01:30:37.987 [INFO][12241] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" iface="eth0" netns="/var/run/netns/cni-2aced866-f3a5-18f3-49a0-e1272f5aa672" Aug 13 01:30:38.071861 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.071861 containerd[1567]: 2025-08-13 01:30:37.987 [INFO][12241] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" iface="eth0" netns="/var/run/netns/cni-2aced866-f3a5-18f3-49a0-e1272f5aa672" Aug 13 01:30:38.071861 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.071861 containerd[1567]: 2025-08-13 01:30:37.995 [INFO][12241] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" after=7.925695ms iface="eth0" netns="/var/run/netns/cni-2aced866-f3a5-18f3-49a0-e1272f5aa672" Aug 13 01:30:38.071861 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.071861 containerd[1567]: 2025-08-13 01:30:37.995 [INFO][12241] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Aug 13 01:30:38.071861 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.071861 containerd[1567]: 2025-08-13 01:30:37.995 [INFO][12241] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Aug 13 01:30:38.071861 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.071861 containerd[1567]: 2025-08-13 01:30:38.029 [INFO][12253] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" HandleID="k8s-pod-network.ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:38.071861 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.071861 containerd[1567]: 2025-08-13 01:30:38.029 [INFO][12253] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:38.071861 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.071861 containerd[1567]: 2025-08-13 01:30:38.029 [INFO][12253] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:38.071861 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.071861 containerd[1567]: 2025-08-13 01:30:38.066 [INFO][12253] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" HandleID="k8s-pod-network.ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:38.071861 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.071861 containerd[1567]: 2025-08-13 01:30:38.066 [INFO][12253] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" HandleID="k8s-pod-network.ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:38.071861 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.071861 containerd[1567]: 2025-08-13 01:30:38.067 [INFO][12253] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:38.071861 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.071861 containerd[1567]: 2025-08-13 01:30:38.069 [INFO][12241] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a" Aug 13 01:30:38.071861 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.074494 kubelet[2720]: E0813 01:30:38.074456 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:30:38.077534 containerd[1567]: time="2025-08-13T01:30:38.077461611Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a-lo: no space left on device" Aug 13 01:30:38.078073 kubelet[2720]: E0813 01:30:38.078052 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a-lo: no space left on device" Aug 13 01:30:38.078200 kubelet[2720]: E0813 01:30:38.078181 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:38.078288 kubelet[2720]: E0813 01:30:38.078264 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:38.078381 kubelet[2720]: E0813 01:30:38.078358 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:38.078688 systemd[1]: run-netns-cni\x2d2aced866\x2df3a5\x2d18f3\x2d49a0\x2de1272f5aa672.mount: Deactivated successfully. Aug 13 01:30:38.114644 containerd[1567]: time="2025-08-13T01:30:38.114594022Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:30:38.416597 kubelet[2720]: I0813 01:30:38.416519 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:30:38.416597 kubelet[2720]: I0813 01:30:38.416567 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:30:38.419028 kubelet[2720]: I0813 01:30:38.419004 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:30:38.439580 kubelet[2720]: I0813 01:30:38.439550 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:30:38.439734 kubelet[2720]: I0813 01:30:38.439687 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:30:38.440107 kubelet[2720]: E0813 01:30:38.440082 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:38.440107 kubelet[2720]: E0813 01:30:38.440102 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:30:38.440173 kubelet[2720]: E0813 01:30:38.440110 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:30:38.440173 kubelet[2720]: E0813 01:30:38.440119 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:30:38.440173 kubelet[2720]: E0813 01:30:38.440129 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:30:38.440173 kubelet[2720]: E0813 01:30:38.440136 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:30:38.440173 kubelet[2720]: E0813 01:30:38.440143 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:30:38.440173 kubelet[2720]: E0813 01:30:38.440165 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:30:38.440173 kubelet[2720]: E0813 01:30:38.440173 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:30:38.440314 kubelet[2720]: E0813 01:30:38.440181 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:30:38.440314 kubelet[2720]: I0813 01:30:38.440191 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:30:38.734796 systemd[1]: Started sshd@43-172.234.25.215:22-147.75.109.163:51236.service - OpenSSH per-connection server daemon (147.75.109.163:51236). Aug 13 01:30:38.814058 containerd[1567]: time="2025-08-13T01:30:38.813691372Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:38.927945 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:38.928889 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.851 [INFO][12265] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18432 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.852 [INFO][12265] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.879 [INFO][12279] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" HandleID="k8s-pod-network.c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.880 [INFO][12279] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" HandleID="k8s-pod-network.c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002ad4e0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:38.87968741 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.880 [INFO][12279] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.880 [INFO][12279] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.880 [INFO][12279] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.887 [INFO][12279] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" host="172-234-25-215" Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.892 [INFO][12279] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.901 [INFO][12279] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.904 [INFO][12279] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.906 [INFO][12279] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.906 [INFO][12279] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" host="172-234-25-215" Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.909 [INFO][12279] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67 Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.913 [INFO][12279] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" host="172-234-25-215" Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.920 [INFO][12279] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.46/26] block=192.168.62.0/26 handle="k8s-pod-network.c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" host="172-234-25-215" Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.943910 containerd[1567]: 2025-08-13 01:30:38.920 [INFO][12279] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.46/26] handle="k8s-pod-network.c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" host="172-234-25-215" Aug 13 01:30:38.943910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.945693 containerd[1567]: 2025-08-13 01:30:38.920 [INFO][12279] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:38.945693 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.945693 containerd[1567]: 2025-08-13 01:30:38.920 [INFO][12279] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.46/26] IPv6=[] ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" HandleID="k8s-pod-network.c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:38.945693 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.945693 containerd[1567]: 2025-08-13 01:30:38.923 [INFO][12265] cni-plugin/k8s.go 418: Populated endpoint ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18432", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ece7fa3b85614dec96c6884641d271ea8244a4262f40bb2dbde5f03ff01f538a", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.46/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:38.945693 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.945693 containerd[1567]: 2025-08-13 01:30:38.923 [INFO][12265] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.46/32] ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:38.945693 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.945693 containerd[1567]: 2025-08-13 01:30:38.923 [INFO][12265] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:38.945693 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.945693 containerd[1567]: 2025-08-13 01:30:38.929 [INFO][12265] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:38.945693 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.945693 containerd[1567]: 2025-08-13 01:30:38.929 [INFO][12265] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18432", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.46/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ba:7f:e3:f6:e2:73", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:38.945693 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.946514 containerd[1567]: 2025-08-13 01:30:38.940 [INFO][12265] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:38.946514 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:38.995956 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:38.995964 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:39.091380 sshd[12263]: Accepted publickey for core from 147.75.109.163 port 51236 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:30:39.096028 sshd-session[12263]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:30:39.104122 containerd[1567]: 2025-08-13 01:30:38.993 [INFO][12302] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Aug 13 01:30:39.104122 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.104122 containerd[1567]: 2025-08-13 01:30:38.994 [INFO][12302] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" iface="eth0" netns="/var/run/netns/cni-57ab0efa-c32f-4996-9406-e6d934e74481" Aug 13 01:30:39.104122 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.104122 containerd[1567]: 2025-08-13 01:30:38.994 [INFO][12302] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" iface="eth0" netns="/var/run/netns/cni-57ab0efa-c32f-4996-9406-e6d934e74481" Aug 13 01:30:39.104122 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.104122 containerd[1567]: 2025-08-13 01:30:39.005 [INFO][12302] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" after=11.601458ms iface="eth0" netns="/var/run/netns/cni-57ab0efa-c32f-4996-9406-e6d934e74481" Aug 13 01:30:39.104122 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.104122 containerd[1567]: 2025-08-13 01:30:39.005 [INFO][12302] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Aug 13 01:30:39.104122 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.104122 containerd[1567]: 2025-08-13 01:30:39.006 [INFO][12302] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Aug 13 01:30:39.104122 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.104122 containerd[1567]: 2025-08-13 01:30:39.036 [INFO][12312] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" HandleID="k8s-pod-network.c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:39.104122 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.104122 containerd[1567]: 2025-08-13 01:30:39.036 [INFO][12312] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:39.104122 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.104122 containerd[1567]: 2025-08-13 01:30:39.037 [INFO][12312] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:39.104122 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.104122 containerd[1567]: 2025-08-13 01:30:39.094 [INFO][12312] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" HandleID="k8s-pod-network.c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:39.104122 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.104122 containerd[1567]: 2025-08-13 01:30:39.094 [INFO][12312] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" HandleID="k8s-pod-network.c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:39.104122 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.104122 containerd[1567]: 2025-08-13 01:30:39.097 [INFO][12312] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:39.104122 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.104122 containerd[1567]: 2025-08-13 01:30:39.101 [INFO][12302] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67" Aug 13 01:30:39.104122 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.106340 systemd-logind[1543]: New session 44 of user core. Aug 13 01:30:39.108879 containerd[1567]: time="2025-08-13T01:30:39.108561018Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67-lo: no space left on device" Aug 13 01:30:39.109519 systemd[1]: run-netns-cni\x2d57ab0efa\x2dc32f\x2d4996\x2d9406\x2de6d934e74481.mount: Deactivated successfully. Aug 13 01:30:39.109672 kubelet[2720]: E0813 01:30:39.109508 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67-lo: no space left on device" Aug 13 01:30:39.109913 kubelet[2720]: E0813 01:30:39.109794 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:39.109913 kubelet[2720]: E0813 01:30:39.109819 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:39.110275 kubelet[2720]: E0813 01:30:39.110010 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:39.117164 systemd[1]: Started session-44.scope - Session 44 of User core. Aug 13 01:30:39.362078 containerd[1567]: time="2025-08-13T01:30:39.361985799Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:39.406865 sshd[12322]: Connection closed by 147.75.109.163 port 51236 Aug 13 01:30:39.409522 sshd-session[12263]: pam_unix(sshd:session): session closed for user core Aug 13 01:30:39.413968 systemd[1]: sshd@43-172.234.25.215:22-147.75.109.163:51236.service: Deactivated successfully. Aug 13 01:30:39.414299 systemd-logind[1543]: Session 44 logged out. Waiting for processes to exit. Aug 13 01:30:39.416975 systemd[1]: session-44.scope: Deactivated successfully. Aug 13 01:30:39.418926 systemd-logind[1543]: Removed session 44. Aug 13 01:30:39.552491 containerd[1567]: time="2025-08-13T01:30:39.552402679Z" level=warning msg="container event discarded" container=7091a88bf8ed2063aabec236166d769ff0994e69d63c60dc74e004d6f390e7e0 type=CONTAINER_CREATED_EVENT Aug 13 01:30:39.552491 containerd[1567]: time="2025-08-13T01:30:39.552481139Z" level=warning msg="container event discarded" container=7091a88bf8ed2063aabec236166d769ff0994e69d63c60dc74e004d6f390e7e0 type=CONTAINER_STARTED_EVENT Aug 13 01:30:39.586874 containerd[1567]: time="2025-08-13T01:30:39.586839795Z" level=warning msg="container event discarded" container=5105f19a4fa4aed619cbcaf7a7ad5ac97da1bec5f3e441a9c5640cdaf472ed9d type=CONTAINER_CREATED_EVENT Aug 13 01:30:39.586874 containerd[1567]: time="2025-08-13T01:30:39.586864295Z" level=warning msg="container event discarded" container=5105f19a4fa4aed619cbcaf7a7ad5ac97da1bec5f3e441a9c5640cdaf472ed9d type=CONTAINER_STARTED_EVENT Aug 13 01:30:39.816349 containerd[1567]: time="2025-08-13T01:30:39.816311401Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:39.922784 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:39.923792 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.856 [INFO][12334] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18500 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.856 [INFO][12334] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.882 [INFO][12347] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" HandleID="k8s-pod-network.35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.882 [INFO][12347] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" HandleID="k8s-pod-network.35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002c5920), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:39.88265452 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.882 [INFO][12347] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.883 [INFO][12347] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.883 [INFO][12347] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.888 [INFO][12347] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" host="172-234-25-215" Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.892 [INFO][12347] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.895 [INFO][12347] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.899 [INFO][12347] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.905 [INFO][12347] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.905 [INFO][12347] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" host="172-234-25-215" Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.907 [INFO][12347] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400 Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.910 [INFO][12347] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" host="172-234-25-215" Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.915 [INFO][12347] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.47/26] block=192.168.62.0/26 handle="k8s-pod-network.35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" host="172-234-25-215" Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.941275 containerd[1567]: 2025-08-13 01:30:39.915 [INFO][12347] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.47/26] handle="k8s-pod-network.35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" host="172-234-25-215" Aug 13 01:30:39.941275 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.945382 containerd[1567]: 2025-08-13 01:30:39.915 [INFO][12347] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:39.945382 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.945382 containerd[1567]: 2025-08-13 01:30:39.915 [INFO][12347] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.47/26] IPv6=[] ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" HandleID="k8s-pod-network.35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:39.945382 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.945382 containerd[1567]: 2025-08-13 01:30:39.919 [INFO][12334] cni-plugin/k8s.go 418: Populated endpoint ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18500", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c5d5ebca0705d292d295377db7c2b14cafd24fa4a08de48df72ab6ee039e0e67", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.47/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:39.945382 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.945382 containerd[1567]: 2025-08-13 01:30:39.919 [INFO][12334] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.47/32] ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:39.945382 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.945382 containerd[1567]: 2025-08-13 01:30:39.919 [INFO][12334] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:39.945382 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.945382 containerd[1567]: 2025-08-13 01:30:39.924 [INFO][12334] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:39.945382 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.945382 containerd[1567]: 2025-08-13 01:30:39.926 [INFO][12334] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18500", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.47/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"8e:97:90:1f:f6:ae", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:39.945382 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:39.946867 containerd[1567]: 2025-08-13 01:30:39.935 [INFO][12334] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:39.946867 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.008180 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:40.008194 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:40.117805 containerd[1567]: 2025-08-13 01:30:40.006 [INFO][12371] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Aug 13 01:30:40.117805 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.117805 containerd[1567]: 2025-08-13 01:30:40.006 [INFO][12371] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" iface="eth0" netns="/var/run/netns/cni-f361dffc-682a-1b28-b471-e69aa5e46416" Aug 13 01:30:40.117805 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.117805 containerd[1567]: 2025-08-13 01:30:40.007 [INFO][12371] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" iface="eth0" netns="/var/run/netns/cni-f361dffc-682a-1b28-b471-e69aa5e46416" Aug 13 01:30:40.117805 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.117805 containerd[1567]: 2025-08-13 01:30:40.012 [INFO][12371] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" after=5.928479ms iface="eth0" netns="/var/run/netns/cni-f361dffc-682a-1b28-b471-e69aa5e46416" Aug 13 01:30:40.117805 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.117805 containerd[1567]: 2025-08-13 01:30:40.013 [INFO][12371] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Aug 13 01:30:40.117805 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.117805 containerd[1567]: 2025-08-13 01:30:40.013 [INFO][12371] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Aug 13 01:30:40.117805 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.117805 containerd[1567]: 2025-08-13 01:30:40.053 [INFO][12379] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" HandleID="k8s-pod-network.35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:40.117805 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.117805 containerd[1567]: 2025-08-13 01:30:40.054 [INFO][12379] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:40.117805 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.117805 containerd[1567]: 2025-08-13 01:30:40.054 [INFO][12379] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:40.117805 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.117805 containerd[1567]: 2025-08-13 01:30:40.099 [INFO][12379] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" HandleID="k8s-pod-network.35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:40.117805 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.117805 containerd[1567]: 2025-08-13 01:30:40.099 [INFO][12379] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" HandleID="k8s-pod-network.35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:40.117805 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.117805 containerd[1567]: 2025-08-13 01:30:40.108 [INFO][12379] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:40.117805 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.117805 containerd[1567]: 2025-08-13 01:30:40.112 [INFO][12371] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400" Aug 13 01:30:40.117805 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.117805 containerd[1567]: time="2025-08-13T01:30:40.116219618Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400-lo: no space left on device" Aug 13 01:30:40.118516 kubelet[2720]: E0813 01:30:40.116518 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400-lo: no space left on device" Aug 13 01:30:40.118516 kubelet[2720]: E0813 01:30:40.116561 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:40.118516 kubelet[2720]: E0813 01:30:40.116580 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:40.118516 kubelet[2720]: E0813 01:30:40.116623 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:40.119098 systemd[1]: run-netns-cni\x2df361dffc\x2d682a\x2d1b28\x2db471\x2de69aa5e46416.mount: Deactivated successfully. Aug 13 01:30:40.817665 containerd[1567]: time="2025-08-13T01:30:40.817612014Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:40.923059 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:40.924010 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.856 [INFO][12388] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18572 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.857 [INFO][12388] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.883 [INFO][12401] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" HandleID="k8s-pod-network.3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.883 [INFO][12401] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" HandleID="k8s-pod-network.3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f100), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:40.883319273 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.884 [INFO][12401] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.884 [INFO][12401] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.884 [INFO][12401] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.891 [INFO][12401] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" host="172-234-25-215" Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.897 [INFO][12401] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.901 [INFO][12401] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.904 [INFO][12401] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.906 [INFO][12401] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.906 [INFO][12401] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" host="172-234-25-215" Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.907 [INFO][12401] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01 Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.910 [INFO][12401] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" host="172-234-25-215" Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.916 [INFO][12401] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.48/26] block=192.168.62.0/26 handle="k8s-pod-network.3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" host="172-234-25-215" Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.940446 containerd[1567]: 2025-08-13 01:30:40.917 [INFO][12401] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.48/26] handle="k8s-pod-network.3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" host="172-234-25-215" Aug 13 01:30:40.940446 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.941579 containerd[1567]: 2025-08-13 01:30:40.917 [INFO][12401] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:40.941579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.941579 containerd[1567]: 2025-08-13 01:30:40.917 [INFO][12401] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.48/26] IPv6=[] ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" HandleID="k8s-pod-network.3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:40.941579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.941579 containerd[1567]: 2025-08-13 01:30:40.919 [INFO][12388] cni-plugin/k8s.go 418: Populated endpoint ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18572", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"35465ac6fc64185ad2c46c8d3023ae0a9a6736bc4ea4a835b187d76b5d76b400", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.48/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:40.941579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.941579 containerd[1567]: 2025-08-13 01:30:40.919 [INFO][12388] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.48/32] ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:40.941579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.941579 containerd[1567]: 2025-08-13 01:30:40.919 [INFO][12388] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:40.941579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.941579 containerd[1567]: 2025-08-13 01:30:40.924 [INFO][12388] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:40.941579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.941579 containerd[1567]: 2025-08-13 01:30:40.925 [INFO][12388] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18572", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.48/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"56:54:a2:e0:97:f1", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:40.941579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.943594 containerd[1567]: 2025-08-13 01:30:40.933 [INFO][12388] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:40.943594 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:40.963947 containerd[1567]: time="2025-08-13T01:30:40.963891964Z" level=warning msg="container event discarded" container=a6a57ab1f557ef6f8b80fa519cc494d4ad4ce164e33088aa825a9cc7c2fe5f7b type=CONTAINER_CREATED_EVENT Aug 13 01:30:41.008640 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:41.008880 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:41.049977 containerd[1567]: time="2025-08-13T01:30:41.049586116Z" level=warning msg="container event discarded" container=a6a57ab1f557ef6f8b80fa519cc494d4ad4ce164e33088aa825a9cc7c2fe5f7b type=CONTAINER_STARTED_EVENT Aug 13 01:30:41.074635 kubelet[2720]: E0813 01:30:41.074549 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:30:41.079556 kubelet[2720]: E0813 01:30:41.079529 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-299933554-aTh- sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2568257827: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:30:41.136578 containerd[1567]: 2025-08-13 01:30:41.005 [INFO][12425] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Aug 13 01:30:41.136578 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.136578 containerd[1567]: 2025-08-13 01:30:41.006 [INFO][12425] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" iface="eth0" netns="/var/run/netns/cni-e7cf81f0-b95b-8d1a-4899-6199b11abbec" Aug 13 01:30:41.136578 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.136578 containerd[1567]: 2025-08-13 01:30:41.006 [INFO][12425] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" iface="eth0" netns="/var/run/netns/cni-e7cf81f0-b95b-8d1a-4899-6199b11abbec" Aug 13 01:30:41.136578 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.136578 containerd[1567]: 2025-08-13 01:30:41.013 [INFO][12425] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" after=6.669158ms iface="eth0" netns="/var/run/netns/cni-e7cf81f0-b95b-8d1a-4899-6199b11abbec" Aug 13 01:30:41.136578 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.136578 containerd[1567]: 2025-08-13 01:30:41.013 [INFO][12425] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Aug 13 01:30:41.136578 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.136578 containerd[1567]: 2025-08-13 01:30:41.013 [INFO][12425] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Aug 13 01:30:41.136578 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.136578 containerd[1567]: 2025-08-13 01:30:41.039 [INFO][12432] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" HandleID="k8s-pod-network.3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:41.136578 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.136578 containerd[1567]: 2025-08-13 01:30:41.040 [INFO][12432] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:41.136578 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.136578 containerd[1567]: 2025-08-13 01:30:41.040 [INFO][12432] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:41.136578 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.136578 containerd[1567]: 2025-08-13 01:30:41.126 [INFO][12432] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" HandleID="k8s-pod-network.3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:41.136578 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.136578 containerd[1567]: 2025-08-13 01:30:41.127 [INFO][12432] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" HandleID="k8s-pod-network.3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:41.136578 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.136578 containerd[1567]: 2025-08-13 01:30:41.128 [INFO][12432] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:41.136578 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.136578 containerd[1567]: 2025-08-13 01:30:41.134 [INFO][12425] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01" Aug 13 01:30:41.136578 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.140102 containerd[1567]: time="2025-08-13T01:30:41.140067269Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01-lo: no space left on device" Aug 13 01:30:41.140872 kubelet[2720]: E0813 01:30:41.140356 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01-lo: no space left on device" Aug 13 01:30:41.141760 systemd[1]: run-netns-cni\x2de7cf81f0\x2db95b\x2d8d1a\x2d4899\x2d6199b11abbec.mount: Deactivated successfully. Aug 13 01:30:41.143329 kubelet[2720]: E0813 01:30:41.142855 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:41.143329 kubelet[2720]: E0813 01:30:41.142881 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:41.143329 kubelet[2720]: E0813 01:30:41.142927 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:41.563663 containerd[1567]: time="2025-08-13T01:30:41.563584557Z" level=warning msg="container event discarded" container=d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6 type=CONTAINER_CREATED_EVENT Aug 13 01:30:41.626874 containerd[1567]: time="2025-08-13T01:30:41.626811951Z" level=warning msg="container event discarded" container=d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6 type=CONTAINER_STARTED_EVENT Aug 13 01:30:41.732171 containerd[1567]: time="2025-08-13T01:30:41.732120207Z" level=warning msg="container event discarded" container=d086f1637bbd72d47a1db8477734810362884fe6457db6ac2aa24e11df5a46f6 type=CONTAINER_STOPPED_EVENT Aug 13 01:30:41.819894 containerd[1567]: time="2025-08-13T01:30:41.819610014Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:41.929992 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:41.931084 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.860 [INFO][12442] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18635 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.860 [INFO][12442] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.885 [INFO][12456] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" HandleID="k8s-pod-network.1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.885 [INFO][12456] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" HandleID="k8s-pod-network.1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:41.885112324 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.885 [INFO][12456] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.885 [INFO][12456] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.885 [INFO][12456] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.891 [INFO][12456] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" host="172-234-25-215" Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.896 [INFO][12456] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.906 [INFO][12456] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.908 [INFO][12456] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.910 [INFO][12456] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.910 [INFO][12456] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" host="172-234-25-215" Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.912 [INFO][12456] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.915 [INFO][12456] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" host="172-234-25-215" Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.921 [INFO][12456] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.49/26] block=192.168.62.0/26 handle="k8s-pod-network.1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" host="172-234-25-215" Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.949504 containerd[1567]: 2025-08-13 01:30:41.922 [INFO][12456] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.49/26] handle="k8s-pod-network.1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" host="172-234-25-215" Aug 13 01:30:41.949504 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.953205 containerd[1567]: 2025-08-13 01:30:41.922 [INFO][12456] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:41.953205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.953205 containerd[1567]: 2025-08-13 01:30:41.922 [INFO][12456] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.49/26] IPv6=[] ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" HandleID="k8s-pod-network.1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:41.953205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.953205 containerd[1567]: 2025-08-13 01:30:41.924 [INFO][12442] cni-plugin/k8s.go 418: Populated endpoint ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18635", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"3ebd7b9577ba57cde0657a97dd0d9290415603e23db7f5778449b0ca7c343d01", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.49/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:41.953205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.953205 containerd[1567]: 2025-08-13 01:30:41.925 [INFO][12442] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.49/32] ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:41.953205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.953205 containerd[1567]: 2025-08-13 01:30:41.925 [INFO][12442] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:41.953205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.953205 containerd[1567]: 2025-08-13 01:30:41.931 [INFO][12442] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:41.953205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.953205 containerd[1567]: 2025-08-13 01:30:41.932 [INFO][12442] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18635", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.49/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ae:86:11:15:41:52", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:41.953205 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:41.953749 containerd[1567]: 2025-08-13 01:30:41.944 [INFO][12442] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:41.953749 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.022899 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:42.022917 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:42.140916 containerd[1567]: 2025-08-13 01:30:42.018 [INFO][12478] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Aug 13 01:30:42.140916 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.140916 containerd[1567]: 2025-08-13 01:30:42.019 [INFO][12478] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" iface="eth0" netns="/var/run/netns/cni-f584341d-3e3d-d6c0-b08f-160a26ccfdbc" Aug 13 01:30:42.140916 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.140916 containerd[1567]: 2025-08-13 01:30:42.019 [INFO][12478] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" iface="eth0" netns="/var/run/netns/cni-f584341d-3e3d-d6c0-b08f-160a26ccfdbc" Aug 13 01:30:42.140916 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.140916 containerd[1567]: 2025-08-13 01:30:42.029 [INFO][12478] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" after=9.981202ms iface="eth0" netns="/var/run/netns/cni-f584341d-3e3d-d6c0-b08f-160a26ccfdbc" Aug 13 01:30:42.140916 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.140916 containerd[1567]: 2025-08-13 01:30:42.029 [INFO][12478] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Aug 13 01:30:42.140916 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.140916 containerd[1567]: 2025-08-13 01:30:42.029 [INFO][12478] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Aug 13 01:30:42.140916 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.140916 containerd[1567]: 2025-08-13 01:30:42.070 [INFO][12489] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" HandleID="k8s-pod-network.1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:42.140916 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.140916 containerd[1567]: 2025-08-13 01:30:42.070 [INFO][12489] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:42.140916 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.140916 containerd[1567]: 2025-08-13 01:30:42.070 [INFO][12489] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:42.140916 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.140916 containerd[1567]: 2025-08-13 01:30:42.132 [INFO][12489] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" HandleID="k8s-pod-network.1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:42.140916 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.140916 containerd[1567]: 2025-08-13 01:30:42.133 [INFO][12489] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" HandleID="k8s-pod-network.1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:42.140916 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.140916 containerd[1567]: 2025-08-13 01:30:42.135 [INFO][12489] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:42.140916 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.140916 containerd[1567]: 2025-08-13 01:30:42.138 [INFO][12478] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad" Aug 13 01:30:42.140916 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.145952 systemd[1]: run-netns-cni\x2df584341d\x2d3e3d\x2dd6c0\x2db08f\x2d160a26ccfdbc.mount: Deactivated successfully. Aug 13 01:30:42.148954 containerd[1567]: time="2025-08-13T01:30:42.148847728Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad-lo: no space left on device" Aug 13 01:30:42.149313 kubelet[2720]: E0813 01:30:42.149232 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad-lo: no space left on device" Aug 13 01:30:42.149789 kubelet[2720]: E0813 01:30:42.149317 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:42.149789 kubelet[2720]: E0813 01:30:42.149340 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:42.149789 kubelet[2720]: E0813 01:30:42.149390 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:42.362211 containerd[1567]: time="2025-08-13T01:30:42.362155263Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:42.821423 containerd[1567]: time="2025-08-13T01:30:42.821375357Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:42.936470 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:42.938073 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.866 [INFO][12498] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18702 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.866 [INFO][12498] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.891 [INFO][12511] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" HandleID="k8s-pod-network.c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.892 [INFO][12511] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" HandleID="k8s-pod-network.c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f080), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:42.891634157 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.892 [INFO][12511] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.892 [INFO][12511] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.892 [INFO][12511] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.898 [INFO][12511] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" host="172-234-25-215" Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.903 [INFO][12511] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.909 [INFO][12511] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.911 [INFO][12511] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.915 [INFO][12511] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.915 [INFO][12511] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" host="172-234-25-215" Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.917 [INFO][12511] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4 Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.921 [INFO][12511] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" host="172-234-25-215" Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.929 [INFO][12511] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.50/26] block=192.168.62.0/26 handle="k8s-pod-network.c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" host="172-234-25-215" Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.948511 containerd[1567]: 2025-08-13 01:30:42.929 [INFO][12511] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.50/26] handle="k8s-pod-network.c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" host="172-234-25-215" Aug 13 01:30:42.948511 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.949408 containerd[1567]: 2025-08-13 01:30:42.929 [INFO][12511] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:42.949408 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.949408 containerd[1567]: 2025-08-13 01:30:42.929 [INFO][12511] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.50/26] IPv6=[] ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" HandleID="k8s-pod-network.c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:42.949408 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.949408 containerd[1567]: 2025-08-13 01:30:42.931 [INFO][12498] cni-plugin/k8s.go 418: Populated endpoint ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18702", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"1e13d657c0f2c564e863082264b1795dd7b4517f26c448a3329bdcc0b513f4ad", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.50/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:42.949408 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.949408 containerd[1567]: 2025-08-13 01:30:42.932 [INFO][12498] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.50/32] ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:42.949408 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.949408 containerd[1567]: 2025-08-13 01:30:42.932 [INFO][12498] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:42.949408 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.949408 containerd[1567]: 2025-08-13 01:30:42.934 [INFO][12498] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:42.949408 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.949408 containerd[1567]: 2025-08-13 01:30:42.934 [INFO][12498] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18702", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.50/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"5e:ba:12:84:12:b2", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:42.949408 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:42.949705 containerd[1567]: 2025-08-13 01:30:42.941 [INFO][12498] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:42.949705 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.020630 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:43.020641 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:43.160665 containerd[1567]: 2025-08-13 01:30:43.017 [INFO][12534] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Aug 13 01:30:43.160665 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.160665 containerd[1567]: 2025-08-13 01:30:43.017 [INFO][12534] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" iface="eth0" netns="/var/run/netns/cni-d258f6b8-dacf-69b9-2c51-d25317fbba18" Aug 13 01:30:43.160665 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.160665 containerd[1567]: 2025-08-13 01:30:43.018 [INFO][12534] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" iface="eth0" netns="/var/run/netns/cni-d258f6b8-dacf-69b9-2c51-d25317fbba18" Aug 13 01:30:43.160665 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.160665 containerd[1567]: 2025-08-13 01:30:43.024 [INFO][12534] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" after=7.407796ms iface="eth0" netns="/var/run/netns/cni-d258f6b8-dacf-69b9-2c51-d25317fbba18" Aug 13 01:30:43.160665 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.160665 containerd[1567]: 2025-08-13 01:30:43.025 [INFO][12534] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Aug 13 01:30:43.160665 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.160665 containerd[1567]: 2025-08-13 01:30:43.025 [INFO][12534] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Aug 13 01:30:43.160665 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.160665 containerd[1567]: 2025-08-13 01:30:43.078 [INFO][12545] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" HandleID="k8s-pod-network.c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:43.160665 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.160665 containerd[1567]: 2025-08-13 01:30:43.081 [INFO][12545] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:43.160665 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.160665 containerd[1567]: 2025-08-13 01:30:43.081 [INFO][12545] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:43.160665 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.160665 containerd[1567]: 2025-08-13 01:30:43.151 [INFO][12545] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" HandleID="k8s-pod-network.c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:43.160665 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.160665 containerd[1567]: 2025-08-13 01:30:43.151 [INFO][12545] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" HandleID="k8s-pod-network.c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:43.160665 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.160665 containerd[1567]: 2025-08-13 01:30:43.152 [INFO][12545] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:43.160665 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.160665 containerd[1567]: 2025-08-13 01:30:43.156 [INFO][12534] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4" Aug 13 01:30:43.160665 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.164595 systemd[1]: run-netns-cni\x2dd258f6b8\x2ddacf\x2d69b9\x2d2c51\x2dd25317fbba18.mount: Deactivated successfully. Aug 13 01:30:43.168083 containerd[1567]: time="2025-08-13T01:30:43.165854331Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4-lo: no space left on device" Aug 13 01:30:43.168566 kubelet[2720]: E0813 01:30:43.168206 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4-lo: no space left on device" Aug 13 01:30:43.168566 kubelet[2720]: E0813 01:30:43.168320 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:43.168566 kubelet[2720]: E0813 01:30:43.168387 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:43.168566 kubelet[2720]: E0813 01:30:43.168456 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:43.822935 containerd[1567]: time="2025-08-13T01:30:43.822880339Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:43.883795 containerd[1567]: time="2025-08-13T01:30:43.883735757Z" level=warning msg="container event discarded" container=d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252 type=CONTAINER_CREATED_EVENT Aug 13 01:30:43.937354 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:43.938263 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:43.954656 containerd[1567]: time="2025-08-13T01:30:43.953685858Z" level=warning msg="container event discarded" container=d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252 type=CONTAINER_STARTED_EVENT Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.867 [INFO][12554] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18774 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.867 [INFO][12554] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.893 [INFO][12569] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" HandleID="k8s-pod-network.d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.893 [INFO][12569] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" HandleID="k8s-pod-network.d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:43.893238259 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.893 [INFO][12569] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.893 [INFO][12569] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.893 [INFO][12569] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.903 [INFO][12569] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" host="172-234-25-215" Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.911 [INFO][12569] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.915 [INFO][12569] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.917 [INFO][12569] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.921 [INFO][12569] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.921 [INFO][12569] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" host="172-234-25-215" Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.922 [INFO][12569] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20 Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.925 [INFO][12569] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" host="172-234-25-215" Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.932 [INFO][12569] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.51/26] block=192.168.62.0/26 handle="k8s-pod-network.d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" host="172-234-25-215" Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.958910 containerd[1567]: 2025-08-13 01:30:43.932 [INFO][12569] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.51/26] handle="k8s-pod-network.d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" host="172-234-25-215" Aug 13 01:30:43.958910 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.962266 containerd[1567]: 2025-08-13 01:30:43.932 [INFO][12569] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:43.962266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.962266 containerd[1567]: 2025-08-13 01:30:43.932 [INFO][12569] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.51/26] IPv6=[] ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" HandleID="k8s-pod-network.d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:43.962266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.962266 containerd[1567]: 2025-08-13 01:30:43.934 [INFO][12554] cni-plugin/k8s.go 418: Populated endpoint ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18774", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"c86195f8724f579d156817f99b158148ccca852b94f383d56997830744715af4", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.51/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:43.962266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.962266 containerd[1567]: 2025-08-13 01:30:43.934 [INFO][12554] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.51/32] ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:43.962266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.962266 containerd[1567]: 2025-08-13 01:30:43.934 [INFO][12554] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:43.962266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.962266 containerd[1567]: 2025-08-13 01:30:43.937 [INFO][12554] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:43.962266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.962266 containerd[1567]: 2025-08-13 01:30:43.938 [INFO][12554] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18774", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.51/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"92:6e:9b:25:49:98", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:43.962266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:43.962588 containerd[1567]: 2025-08-13 01:30:43.949 [INFO][12554] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:43.962588 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.033487 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:44.033497 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:44.075420 kubelet[2720]: E0813 01:30:44.075332 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:30:44.124491 containerd[1567]: 2025-08-13 01:30:44.030 [INFO][12591] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Aug 13 01:30:44.124491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.124491 containerd[1567]: 2025-08-13 01:30:44.031 [INFO][12591] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" iface="eth0" netns="/var/run/netns/cni-55a81b9f-e5c3-c24f-400f-f8de3f7c36e5" Aug 13 01:30:44.124491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.124491 containerd[1567]: 2025-08-13 01:30:44.032 [INFO][12591] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" iface="eth0" netns="/var/run/netns/cni-55a81b9f-e5c3-c24f-400f-f8de3f7c36e5" Aug 13 01:30:44.124491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.124491 containerd[1567]: 2025-08-13 01:30:44.039 [INFO][12591] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" after=7.745106ms iface="eth0" netns="/var/run/netns/cni-55a81b9f-e5c3-c24f-400f-f8de3f7c36e5" Aug 13 01:30:44.124491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.124491 containerd[1567]: 2025-08-13 01:30:44.039 [INFO][12591] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Aug 13 01:30:44.124491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.124491 containerd[1567]: 2025-08-13 01:30:44.039 [INFO][12591] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Aug 13 01:30:44.124491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.124491 containerd[1567]: 2025-08-13 01:30:44.074 [INFO][12600] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" HandleID="k8s-pod-network.d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:44.124491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.124491 containerd[1567]: 2025-08-13 01:30:44.080 [INFO][12600] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:44.124491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.124491 containerd[1567]: 2025-08-13 01:30:44.081 [INFO][12600] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:44.124491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.124491 containerd[1567]: 2025-08-13 01:30:44.117 [INFO][12600] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" HandleID="k8s-pod-network.d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:44.124491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.124491 containerd[1567]: 2025-08-13 01:30:44.117 [INFO][12600] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" HandleID="k8s-pod-network.d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:44.124491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.124491 containerd[1567]: 2025-08-13 01:30:44.119 [INFO][12600] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:44.124491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.124491 containerd[1567]: 2025-08-13 01:30:44.121 [INFO][12591] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20" Aug 13 01:30:44.124491 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.128311 systemd[1]: run-netns-cni\x2d55a81b9f\x2de5c3\x2dc24f\x2d400f\x2df8de3f7c36e5.mount: Deactivated successfully. Aug 13 01:30:44.129413 containerd[1567]: time="2025-08-13T01:30:44.129237653Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20-lo: no space left on device" Aug 13 01:30:44.130003 kubelet[2720]: E0813 01:30:44.129913 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20-lo: no space left on device" Aug 13 01:30:44.130192 kubelet[2720]: E0813 01:30:44.130130 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:44.130671 kubelet[2720]: E0813 01:30:44.130649 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:44.130952 kubelet[2720]: E0813 01:30:44.130831 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:44.468890 systemd[1]: Started sshd@44-172.234.25.215:22-147.75.109.163:51252.service - OpenSSH per-connection server daemon (147.75.109.163:51252). Aug 13 01:30:44.615539 containerd[1567]: time="2025-08-13T01:30:44.615468137Z" level=warning msg="container event discarded" container=d8c34ef701a6a396ff4dafbaeb810c13feae60ed5c0c5ed8e5707817776cd252 type=CONTAINER_STOPPED_EVENT Aug 13 01:30:44.816238 sshd[12610]: Accepted publickey for core from 147.75.109.163 port 51252 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:30:44.818412 sshd-session[12610]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:30:44.827921 containerd[1567]: time="2025-08-13T01:30:44.827536396Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:44.829077 systemd-logind[1543]: New session 45 of user core. Aug 13 01:30:44.836859 systemd[1]: Started session-45.scope - Session 45 of User core. Aug 13 01:30:44.952096 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:44.952678 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.875 [INFO][12614] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18846 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.876 [INFO][12614] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.904 [INFO][12627] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" HandleID="k8s-pod-network.b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.904 [INFO][12627] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" HandleID="k8s-pod-network.b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5600), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:44.904412915 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.905 [INFO][12627] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.905 [INFO][12627] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.905 [INFO][12627] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.913 [INFO][12627] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" host="172-234-25-215" Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.917 [INFO][12627] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.921 [INFO][12627] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.923 [INFO][12627] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.928 [INFO][12627] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.928 [INFO][12627] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" host="172-234-25-215" Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.930 [INFO][12627] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629 Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.933 [INFO][12627] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" host="172-234-25-215" Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.940 [INFO][12627] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.52/26] block=192.168.62.0/26 handle="k8s-pod-network.b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" host="172-234-25-215" Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.978335 containerd[1567]: 2025-08-13 01:30:44.941 [INFO][12627] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.52/26] handle="k8s-pod-network.b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" host="172-234-25-215" Aug 13 01:30:44.978335 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.980931 containerd[1567]: 2025-08-13 01:30:44.941 [INFO][12627] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:44.980931 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.980931 containerd[1567]: 2025-08-13 01:30:44.941 [INFO][12627] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.52/26] IPv6=[] ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" HandleID="k8s-pod-network.b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:44.980931 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.980931 containerd[1567]: 2025-08-13 01:30:44.944 [INFO][12614] cni-plugin/k8s.go 418: Populated endpoint ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18846", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"d83ff5c8225ed3ef36d3b4bb9e1bcb9a3973c551ccd6f6e6f10e3ba568777d20", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.52/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:44.980931 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.980931 containerd[1567]: 2025-08-13 01:30:44.944 [INFO][12614] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.52/32] ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:44.980931 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.980931 containerd[1567]: 2025-08-13 01:30:44.944 [INFO][12614] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:44.980931 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.980931 containerd[1567]: 2025-08-13 01:30:44.955 [INFO][12614] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:44.980931 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.980931 containerd[1567]: 2025-08-13 01:30:44.957 [INFO][12614] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18846", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.52/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"66:d6:f0:07:42:4e", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:44.980931 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:44.982408 containerd[1567]: 2025-08-13 01:30:44.971 [INFO][12614] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:44.982408 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.050182 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:45.050195 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:45.165254 sshd[12623]: Connection closed by 147.75.109.163 port 51252 Aug 13 01:30:45.168331 sshd-session[12610]: pam_unix(sshd:session): session closed for user core Aug 13 01:30:45.176028 systemd[1]: sshd@44-172.234.25.215:22-147.75.109.163:51252.service: Deactivated successfully. Aug 13 01:30:45.177161 systemd-logind[1543]: Session 45 logged out. Waiting for processes to exit. Aug 13 01:30:45.179656 systemd[1]: session-45.scope: Deactivated successfully. Aug 13 01:30:45.185660 systemd-logind[1543]: Removed session 45. Aug 13 01:30:45.187824 containerd[1567]: 2025-08-13 01:30:45.046 [INFO][12651] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Aug 13 01:30:45.187824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.187824 containerd[1567]: 2025-08-13 01:30:45.046 [INFO][12651] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" iface="eth0" netns="/var/run/netns/cni-f9292238-a66a-06cc-eef9-32be8d286a51" Aug 13 01:30:45.187824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.187824 containerd[1567]: 2025-08-13 01:30:45.047 [INFO][12651] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" iface="eth0" netns="/var/run/netns/cni-f9292238-a66a-06cc-eef9-32be8d286a51" Aug 13 01:30:45.187824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.187824 containerd[1567]: 2025-08-13 01:30:45.060 [INFO][12651] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" after=14.064464ms iface="eth0" netns="/var/run/netns/cni-f9292238-a66a-06cc-eef9-32be8d286a51" Aug 13 01:30:45.187824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.187824 containerd[1567]: 2025-08-13 01:30:45.061 [INFO][12651] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Aug 13 01:30:45.187824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.187824 containerd[1567]: 2025-08-13 01:30:45.061 [INFO][12651] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Aug 13 01:30:45.187824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.187824 containerd[1567]: 2025-08-13 01:30:45.091 [INFO][12666] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" HandleID="k8s-pod-network.b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:45.187824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.187824 containerd[1567]: 2025-08-13 01:30:45.092 [INFO][12666] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:45.187824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.187824 containerd[1567]: 2025-08-13 01:30:45.092 [INFO][12666] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:45.187824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.187824 containerd[1567]: 2025-08-13 01:30:45.178 [INFO][12666] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" HandleID="k8s-pod-network.b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:45.187824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.187824 containerd[1567]: 2025-08-13 01:30:45.178 [INFO][12666] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" HandleID="k8s-pod-network.b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:45.187824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.187824 containerd[1567]: 2025-08-13 01:30:45.180 [INFO][12666] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:45.187824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.187824 containerd[1567]: 2025-08-13 01:30:45.184 [INFO][12651] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629" Aug 13 01:30:45.187824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.190237 containerd[1567]: time="2025-08-13T01:30:45.190195118Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629-lo: no space left on device" Aug 13 01:30:45.190862 kubelet[2720]: E0813 01:30:45.190447 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629-lo: no space left on device" Aug 13 01:30:45.191786 kubelet[2720]: E0813 01:30:45.191168 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:45.191786 kubelet[2720]: E0813 01:30:45.191195 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:45.191786 kubelet[2720]: E0813 01:30:45.191242 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:45.192605 systemd[1]: run-netns-cni\x2df9292238\x2da66a\x2d06cc\x2deef9\x2d32be8d286a51.mount: Deactivated successfully. Aug 13 01:30:45.362363 containerd[1567]: time="2025-08-13T01:30:45.362289240Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:45.829303 containerd[1567]: time="2025-08-13T01:30:45.829194899Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:45.939795 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:45.941537 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.871 [INFO][12681] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18921 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.871 [INFO][12681] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.893 [INFO][12692] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" HandleID="k8s-pod-network.96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.894 [INFO][12692] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" HandleID="k8s-pod-network.96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002c56d0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:45.893951659 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.894 [INFO][12692] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.894 [INFO][12692] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.894 [INFO][12692] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.903 [INFO][12692] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" host="172-234-25-215" Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.907 [INFO][12692] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.912 [INFO][12692] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.917 [INFO][12692] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.919 [INFO][12692] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.920 [INFO][12692] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" host="172-234-25-215" Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.922 [INFO][12692] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51 Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.925 [INFO][12692] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" host="172-234-25-215" Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.930 [INFO][12692] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.53/26] block=192.168.62.0/26 handle="k8s-pod-network.96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" host="172-234-25-215" Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.963317 containerd[1567]: 2025-08-13 01:30:45.930 [INFO][12692] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.53/26] handle="k8s-pod-network.96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" host="172-234-25-215" Aug 13 01:30:45.963317 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.966076 containerd[1567]: 2025-08-13 01:30:45.931 [INFO][12692] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:45.966076 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.966076 containerd[1567]: 2025-08-13 01:30:45.931 [INFO][12692] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.53/26] IPv6=[] ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" HandleID="k8s-pod-network.96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:45.966076 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.966076 containerd[1567]: 2025-08-13 01:30:45.933 [INFO][12681] cni-plugin/k8s.go 418: Populated endpoint ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18921", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"b0661ff8a0393ecf6cc4f9deb9e361debe82c45a3b6d92d15e64fb1d6afe5629", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.53/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:45.966076 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.966076 containerd[1567]: 2025-08-13 01:30:45.936 [INFO][12681] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.53/32] ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:45.966076 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.966076 containerd[1567]: 2025-08-13 01:30:45.936 [INFO][12681] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:45.966076 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.966076 containerd[1567]: 2025-08-13 01:30:45.942 [INFO][12681] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:45.966076 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.966076 containerd[1567]: 2025-08-13 01:30:45.944 [INFO][12681] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18921", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.53/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"32:90:d2:81:22:09", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:45.966076 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:45.966382 containerd[1567]: 2025-08-13 01:30:45.956 [INFO][12681] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:45.966382 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.032086 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:46.032100 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:46.077317 kubelet[2720]: E0813 01:30:46.076668 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:30:46.082398 kubelet[2720]: E0813 01:30:46.082337 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-349524588-hiNR sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2074644924: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:30:46.136043 containerd[1567]: 2025-08-13 01:30:46.029 [INFO][12717] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Aug 13 01:30:46.136043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.136043 containerd[1567]: 2025-08-13 01:30:46.030 [INFO][12717] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" iface="eth0" netns="/var/run/netns/cni-2aaaebdd-a8ca-f7b9-8ed0-41d4a2ec9ff1" Aug 13 01:30:46.136043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.136043 containerd[1567]: 2025-08-13 01:30:46.030 [INFO][12717] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" iface="eth0" netns="/var/run/netns/cni-2aaaebdd-a8ca-f7b9-8ed0-41d4a2ec9ff1" Aug 13 01:30:46.136043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.136043 containerd[1567]: 2025-08-13 01:30:46.038 [INFO][12717] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" after=8.050175ms iface="eth0" netns="/var/run/netns/cni-2aaaebdd-a8ca-f7b9-8ed0-41d4a2ec9ff1" Aug 13 01:30:46.136043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.136043 containerd[1567]: 2025-08-13 01:30:46.039 [INFO][12717] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Aug 13 01:30:46.136043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.136043 containerd[1567]: 2025-08-13 01:30:46.040 [INFO][12717] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Aug 13 01:30:46.136043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.136043 containerd[1567]: 2025-08-13 01:30:46.075 [INFO][12725] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" HandleID="k8s-pod-network.96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:46.136043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.136043 containerd[1567]: 2025-08-13 01:30:46.075 [INFO][12725] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:46.136043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.136043 containerd[1567]: 2025-08-13 01:30:46.075 [INFO][12725] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:46.136043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.136043 containerd[1567]: 2025-08-13 01:30:46.126 [INFO][12725] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" HandleID="k8s-pod-network.96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:46.136043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.136043 containerd[1567]: 2025-08-13 01:30:46.126 [INFO][12725] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" HandleID="k8s-pod-network.96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:46.136043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.136043 containerd[1567]: 2025-08-13 01:30:46.130 [INFO][12725] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:46.136043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.136043 containerd[1567]: 2025-08-13 01:30:46.133 [INFO][12717] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51" Aug 13 01:30:46.136043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.140264 systemd[1]: run-netns-cni\x2d2aaaebdd\x2da8ca\x2df7b9\x2d8ed0\x2d41d4a2ec9ff1.mount: Deactivated successfully. Aug 13 01:30:46.141340 containerd[1567]: time="2025-08-13T01:30:46.140662474Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51-lo: no space left on device" Aug 13 01:30:46.141438 kubelet[2720]: E0813 01:30:46.141049 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51-lo: no space left on device" Aug 13 01:30:46.141438 kubelet[2720]: E0813 01:30:46.141085 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:46.141438 kubelet[2720]: E0813 01:30:46.141104 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:46.141438 kubelet[2720]: E0813 01:30:46.141143 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:46.830477 containerd[1567]: time="2025-08-13T01:30:46.830428332Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:46.939878 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:46.941799 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.870 [INFO][12735] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 18991 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.870 [INFO][12735] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.891 [INFO][12746] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" HandleID="k8s-pod-network.e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.892 [INFO][12746] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" HandleID="k8s-pod-network.e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024eff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:46.891965039 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.892 [INFO][12746] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.892 [INFO][12746] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.892 [INFO][12746] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.902 [INFO][12746] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" host="172-234-25-215" Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.907 [INFO][12746] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.915 [INFO][12746] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.916 [INFO][12746] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.919 [INFO][12746] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.919 [INFO][12746] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" host="172-234-25-215" Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.920 [INFO][12746] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.924 [INFO][12746] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" host="172-234-25-215" Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.931 [INFO][12746] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.54/26] block=192.168.62.0/26 handle="k8s-pod-network.e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" host="172-234-25-215" Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.959072 containerd[1567]: 2025-08-13 01:30:46.931 [INFO][12746] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.54/26] handle="k8s-pod-network.e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" host="172-234-25-215" Aug 13 01:30:46.959072 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.960602 containerd[1567]: 2025-08-13 01:30:46.931 [INFO][12746] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:46.960602 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.960602 containerd[1567]: 2025-08-13 01:30:46.931 [INFO][12746] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.54/26] IPv6=[] ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" HandleID="k8s-pod-network.e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:46.960602 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.960602 containerd[1567]: 2025-08-13 01:30:46.935 [INFO][12735] cni-plugin/k8s.go 418: Populated endpoint ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18991", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"96388c669c31f627cf317b769dc2e91d19abf38b5d13e6968b27e6d1af2c8c51", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.54/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:46.960602 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.960602 containerd[1567]: 2025-08-13 01:30:46.935 [INFO][12735] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.54/32] ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:46.960602 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.960602 containerd[1567]: 2025-08-13 01:30:46.935 [INFO][12735] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:46.960602 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.960602 containerd[1567]: 2025-08-13 01:30:46.940 [INFO][12735] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:46.960602 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.960602 containerd[1567]: 2025-08-13 01:30:46.940 [INFO][12735] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"18991", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.54/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"a2:4c:49:8c:c4:6f", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:46.960602 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:46.961613 containerd[1567]: 2025-08-13 01:30:46.951 [INFO][12735] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:46.961613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.061239 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:47.061248 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:47.145579 containerd[1567]: 2025-08-13 01:30:47.058 [INFO][12769] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Aug 13 01:30:47.145579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.145579 containerd[1567]: 2025-08-13 01:30:47.058 [INFO][12769] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" iface="eth0" netns="/var/run/netns/cni-e05d5e61-d180-fb63-5b6b-7fa3a74aa028" Aug 13 01:30:47.145579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.145579 containerd[1567]: 2025-08-13 01:30:47.059 [INFO][12769] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" iface="eth0" netns="/var/run/netns/cni-e05d5e61-d180-fb63-5b6b-7fa3a74aa028" Aug 13 01:30:47.145579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.145579 containerd[1567]: 2025-08-13 01:30:47.066 [INFO][12769] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" after=6.886298ms iface="eth0" netns="/var/run/netns/cni-e05d5e61-d180-fb63-5b6b-7fa3a74aa028" Aug 13 01:30:47.145579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.145579 containerd[1567]: 2025-08-13 01:30:47.066 [INFO][12769] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Aug 13 01:30:47.145579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.145579 containerd[1567]: 2025-08-13 01:30:47.066 [INFO][12769] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Aug 13 01:30:47.145579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.145579 containerd[1567]: 2025-08-13 01:30:47.100 [INFO][12779] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" HandleID="k8s-pod-network.e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:47.145579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.145579 containerd[1567]: 2025-08-13 01:30:47.103 [INFO][12779] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:47.145579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.145579 containerd[1567]: 2025-08-13 01:30:47.103 [INFO][12779] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:47.145579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.145579 containerd[1567]: 2025-08-13 01:30:47.139 [INFO][12779] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" HandleID="k8s-pod-network.e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:47.145579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.145579 containerd[1567]: 2025-08-13 01:30:47.139 [INFO][12779] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" HandleID="k8s-pod-network.e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:47.145579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.145579 containerd[1567]: 2025-08-13 01:30:47.140 [INFO][12779] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:47.145579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.145579 containerd[1567]: 2025-08-13 01:30:47.143 [INFO][12769] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b" Aug 13 01:30:47.145579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.151414 containerd[1567]: time="2025-08-13T01:30:47.151312750Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b-lo: no space left on device" Aug 13 01:30:47.151964 kubelet[2720]: E0813 01:30:47.151893 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b-lo: no space left on device" Aug 13 01:30:47.153091 kubelet[2720]: E0813 01:30:47.151945 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:47.153091 kubelet[2720]: E0813 01:30:47.152309 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:47.153091 kubelet[2720]: E0813 01:30:47.152379 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:47.153561 systemd[1]: run-netns-cni\x2de05d5e61\x2dd180\x2dfb63\x2d5b6b\x2d7fa3a74aa028.mount: Deactivated successfully. Aug 13 01:30:47.642753 containerd[1567]: time="2025-08-13T01:30:47.642691374Z" level=info msg="TaskExit event in podsandbox handler container_id:\"05a7e6943f56eff710d015cc4d373aa0238d7506c1cf61f23b36bc5ebcb7ce84\" id:\"6631c522206da8fec9b89635eeac88088f4337357451a95dbb54789fc14914b3\" pid:12800 exited_at:{seconds:1755048647 nanos:642221005}" Aug 13 01:30:47.831980 containerd[1567]: time="2025-08-13T01:30:47.831943815Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:47.943441 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:47.945313 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.872 [INFO][12812] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 19069 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.872 [INFO][12812] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.894 [INFO][12825] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" HandleID="k8s-pod-network.846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.894 [INFO][12825] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" HandleID="k8s-pod-network.846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5600), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:47.89458813 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.896 [INFO][12825] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.896 [INFO][12825] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.896 [INFO][12825] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.903 [INFO][12825] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" host="172-234-25-215" Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.909 [INFO][12825] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.915 [INFO][12825] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.919 [INFO][12825] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.925 [INFO][12825] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.925 [INFO][12825] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" host="172-234-25-215" Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.927 [INFO][12825] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2 Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.930 [INFO][12825] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" host="172-234-25-215" Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.937 [INFO][12825] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.55/26] block=192.168.62.0/26 handle="k8s-pod-network.846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" host="172-234-25-215" Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.967419 containerd[1567]: 2025-08-13 01:30:47.937 [INFO][12825] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.55/26] handle="k8s-pod-network.846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" host="172-234-25-215" Aug 13 01:30:47.967419 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.971150 containerd[1567]: 2025-08-13 01:30:47.937 [INFO][12825] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:47.971150 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.971150 containerd[1567]: 2025-08-13 01:30:47.937 [INFO][12825] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.55/26] IPv6=[] ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" HandleID="k8s-pod-network.846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:47.971150 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.971150 containerd[1567]: 2025-08-13 01:30:47.940 [INFO][12812] cni-plugin/k8s.go 418: Populated endpoint ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19069", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e2a0e6b2f251c34ff0b8fa3a30bbfa9befa4cba3afee8260029c8e1bbc9c6f2b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.55/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:47.971150 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.971150 containerd[1567]: 2025-08-13 01:30:47.940 [INFO][12812] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.55/32] ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:47.971150 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.971150 containerd[1567]: 2025-08-13 01:30:47.940 [INFO][12812] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:47.971150 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.971150 containerd[1567]: 2025-08-13 01:30:47.943 [INFO][12812] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:47.971150 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.971150 containerd[1567]: 2025-08-13 01:30:47.944 [INFO][12812] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19069", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.55/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"82:e5:6d:91:a9:ae", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:47.971150 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:47.971464 containerd[1567]: 2025-08-13 01:30:47.952 [INFO][12812] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:47.971464 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.028774 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:48.028784 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:48.122384 containerd[1567]: 2025-08-13 01:30:48.026 [INFO][12850] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Aug 13 01:30:48.122384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.122384 containerd[1567]: 2025-08-13 01:30:48.027 [INFO][12850] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" iface="eth0" netns="/var/run/netns/cni-c40b1ad3-5de8-49ed-3a3e-7a2ed95c630c" Aug 13 01:30:48.122384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.122384 containerd[1567]: 2025-08-13 01:30:48.027 [INFO][12850] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" iface="eth0" netns="/var/run/netns/cni-c40b1ad3-5de8-49ed-3a3e-7a2ed95c630c" Aug 13 01:30:48.122384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.122384 containerd[1567]: 2025-08-13 01:30:48.039 [INFO][12850] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" after=12.318917ms iface="eth0" netns="/var/run/netns/cni-c40b1ad3-5de8-49ed-3a3e-7a2ed95c630c" Aug 13 01:30:48.122384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.122384 containerd[1567]: 2025-08-13 01:30:48.040 [INFO][12850] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Aug 13 01:30:48.122384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.122384 containerd[1567]: 2025-08-13 01:30:48.040 [INFO][12850] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Aug 13 01:30:48.122384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.122384 containerd[1567]: 2025-08-13 01:30:48.067 [INFO][12859] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" HandleID="k8s-pod-network.846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:48.122384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.122384 containerd[1567]: 2025-08-13 01:30:48.068 [INFO][12859] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:48.122384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.122384 containerd[1567]: 2025-08-13 01:30:48.068 [INFO][12859] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:48.122384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.122384 containerd[1567]: 2025-08-13 01:30:48.116 [INFO][12859] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" HandleID="k8s-pod-network.846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:48.122384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.122384 containerd[1567]: 2025-08-13 01:30:48.116 [INFO][12859] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" HandleID="k8s-pod-network.846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:48.122384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.122384 containerd[1567]: 2025-08-13 01:30:48.118 [INFO][12859] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:48.122384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.122384 containerd[1567]: 2025-08-13 01:30:48.120 [INFO][12850] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2" Aug 13 01:30:48.122384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.124123 containerd[1567]: time="2025-08-13T01:30:48.124028986Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2-lo: no space left on device" Aug 13 01:30:48.124284 kubelet[2720]: E0813 01:30:48.124232 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2-lo: no space left on device" Aug 13 01:30:48.124349 kubelet[2720]: E0813 01:30:48.124302 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:48.124349 kubelet[2720]: E0813 01:30:48.124325 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:48.124514 kubelet[2720]: E0813 01:30:48.124376 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:48.126568 systemd[1]: run-netns-cni\x2dc40b1ad3\x2d5de8\x2d49ed\x2d3a3e\x2d7a2ed95c630c.mount: Deactivated successfully. Aug 13 01:30:48.362447 containerd[1567]: time="2025-08-13T01:30:48.362397118Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:48.462451 kubelet[2720]: I0813 01:30:48.462406 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:30:48.462451 kubelet[2720]: I0813 01:30:48.462444 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:30:48.463650 kubelet[2720]: I0813 01:30:48.463635 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:30:48.474376 kubelet[2720]: I0813 01:30:48.474360 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:30:48.474494 kubelet[2720]: I0813 01:30:48.474478 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-kube-controllers-b57d47f84-lznkx","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:30:48.474566 kubelet[2720]: E0813 01:30:48.474505 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:30:48.474566 kubelet[2720]: E0813 01:30:48.474513 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:30:48.474566 kubelet[2720]: E0813 01:30:48.474520 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:48.474566 kubelet[2720]: E0813 01:30:48.474528 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:30:48.474566 kubelet[2720]: E0813 01:30:48.474536 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:30:48.474566 kubelet[2720]: E0813 01:30:48.474546 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:30:48.474566 kubelet[2720]: E0813 01:30:48.474554 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:30:48.474566 kubelet[2720]: E0813 01:30:48.474563 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:30:48.474566 kubelet[2720]: E0813 01:30:48.474569 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:30:48.474759 kubelet[2720]: E0813 01:30:48.474577 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:30:48.474759 kubelet[2720]: I0813 01:30:48.474586 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:30:48.833363 containerd[1567]: time="2025-08-13T01:30:48.833324540Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:48.941387 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:48.942093 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.873 [INFO][12871] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 19137 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.874 [INFO][12871] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.899 [INFO][12881] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" HandleID="k8s-pod-network.1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.900 [INFO][12881] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" HandleID="k8s-pod-network.1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5920), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:48.899414888 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.900 [INFO][12881] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.900 [INFO][12881] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.901 [INFO][12881] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.909 [INFO][12881] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" host="172-234-25-215" Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.915 [INFO][12881] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.920 [INFO][12881] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.922 [INFO][12881] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.924 [INFO][12881] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.924 [INFO][12881] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" host="172-234-25-215" Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.926 [INFO][12881] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011 Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.929 [INFO][12881] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" host="172-234-25-215" Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.934 [INFO][12881] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.56/26] block=192.168.62.0/26 handle="k8s-pod-network.1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" host="172-234-25-215" Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.963801 containerd[1567]: 2025-08-13 01:30:48.934 [INFO][12881] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.56/26] handle="k8s-pod-network.1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" host="172-234-25-215" Aug 13 01:30:48.963801 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.965347 containerd[1567]: 2025-08-13 01:30:48.934 [INFO][12881] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:48.965347 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.965347 containerd[1567]: 2025-08-13 01:30:48.935 [INFO][12881] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.56/26] IPv6=[] ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" HandleID="k8s-pod-network.1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:48.965347 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.965347 containerd[1567]: 2025-08-13 01:30:48.938 [INFO][12871] cni-plugin/k8s.go 418: Populated endpoint ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19137", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"846015162daa1cbbe19b177958dc36faa452eb4ff34d88349101a1a86a7fbec2", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.56/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:48.965347 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.965347 containerd[1567]: 2025-08-13 01:30:48.938 [INFO][12871] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.56/32] ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:48.965347 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.965347 containerd[1567]: 2025-08-13 01:30:48.938 [INFO][12871] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:48.965347 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.965347 containerd[1567]: 2025-08-13 01:30:48.941 [INFO][12871] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:48.965347 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.965347 containerd[1567]: 2025-08-13 01:30:48.941 [INFO][12871] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19137", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.56/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"82:37:af:14:fe:aa", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:48.965347 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:48.965648 containerd[1567]: 2025-08-13 01:30:48.955 [INFO][12871] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:48.965648 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.040518 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:49.040526 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:49.122902 containerd[1567]: 2025-08-13 01:30:49.035 [INFO][12903] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Aug 13 01:30:49.122902 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.122902 containerd[1567]: 2025-08-13 01:30:49.036 [INFO][12903] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" iface="eth0" netns="/var/run/netns/cni-0ccdca72-2d31-d0ee-d443-dcfa68b600e9" Aug 13 01:30:49.122902 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.122902 containerd[1567]: 2025-08-13 01:30:49.037 [INFO][12903] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" iface="eth0" netns="/var/run/netns/cni-0ccdca72-2d31-d0ee-d443-dcfa68b600e9" Aug 13 01:30:49.122902 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.122902 containerd[1567]: 2025-08-13 01:30:49.045 [INFO][12903] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" after=9.383063ms iface="eth0" netns="/var/run/netns/cni-0ccdca72-2d31-d0ee-d443-dcfa68b600e9" Aug 13 01:30:49.122902 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.122902 containerd[1567]: 2025-08-13 01:30:49.045 [INFO][12903] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Aug 13 01:30:49.122902 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.122902 containerd[1567]: 2025-08-13 01:30:49.046 [INFO][12903] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Aug 13 01:30:49.122902 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.122902 containerd[1567]: 2025-08-13 01:30:49.080 [INFO][12911] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" HandleID="k8s-pod-network.1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:49.122902 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.122902 containerd[1567]: 2025-08-13 01:30:49.081 [INFO][12911] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:49.122902 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.122902 containerd[1567]: 2025-08-13 01:30:49.081 [INFO][12911] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:49.122902 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.122902 containerd[1567]: 2025-08-13 01:30:49.116 [INFO][12911] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" HandleID="k8s-pod-network.1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:49.122902 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.122902 containerd[1567]: 2025-08-13 01:30:49.116 [INFO][12911] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" HandleID="k8s-pod-network.1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:49.122902 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.122902 containerd[1567]: 2025-08-13 01:30:49.118 [INFO][12911] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:49.122902 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.122902 containerd[1567]: 2025-08-13 01:30:49.120 [INFO][12903] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011" Aug 13 01:30:49.122902 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.126807 systemd[1]: run-netns-cni\x2d0ccdca72\x2d2d31\x2dd0ee\x2dd443\x2ddcfa68b600e9.mount: Deactivated successfully. Aug 13 01:30:49.127217 containerd[1567]: time="2025-08-13T01:30:49.127168738Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011-lo: no space left on device" Aug 13 01:30:49.127426 kubelet[2720]: E0813 01:30:49.127354 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011-lo: no space left on device" Aug 13 01:30:49.127516 kubelet[2720]: E0813 01:30:49.127498 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:49.127575 kubelet[2720]: E0813 01:30:49.127562 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:49.127679 kubelet[2720]: E0813 01:30:49.127657 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:49.835427 containerd[1567]: time="2025-08-13T01:30:49.835381632Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:49.945064 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:49.945332 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.878 [INFO][12922] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 19211 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.878 [INFO][12922] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.905 [INFO][12935] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" HandleID="k8s-pod-network.21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.905 [INFO][12935] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" HandleID="k8s-pod-network.21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f7f0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:49.905040803 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.905 [INFO][12935] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.905 [INFO][12935] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.906 [INFO][12935] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.913 [INFO][12935] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" host="172-234-25-215" Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.917 [INFO][12935] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.921 [INFO][12935] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.923 [INFO][12935] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.925 [INFO][12935] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.925 [INFO][12935] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" host="172-234-25-215" Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.926 [INFO][12935] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7 Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.929 [INFO][12935] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" host="172-234-25-215" Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.936 [INFO][12935] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.57/26] block=192.168.62.0/26 handle="k8s-pod-network.21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" host="172-234-25-215" Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.963860 containerd[1567]: 2025-08-13 01:30:49.936 [INFO][12935] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.57/26] handle="k8s-pod-network.21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" host="172-234-25-215" Aug 13 01:30:49.963860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.965125 containerd[1567]: 2025-08-13 01:30:49.936 [INFO][12935] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:49.965125 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.965125 containerd[1567]: 2025-08-13 01:30:49.936 [INFO][12935] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.57/26] IPv6=[] ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" HandleID="k8s-pod-network.21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:49.965125 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.965125 containerd[1567]: 2025-08-13 01:30:49.940 [INFO][12922] cni-plugin/k8s.go 418: Populated endpoint ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19211", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"1628801ac9c9fd59a3bac0fc4cca117a8af4e575f0a935850eadb458becba011", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.57/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:49.965125 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.965125 containerd[1567]: 2025-08-13 01:30:49.940 [INFO][12922] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.57/32] ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:49.965125 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.965125 containerd[1567]: 2025-08-13 01:30:49.940 [INFO][12922] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:49.965125 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.965125 containerd[1567]: 2025-08-13 01:30:49.944 [INFO][12922] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:49.965125 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.965125 containerd[1567]: 2025-08-13 01:30:49.948 [INFO][12922] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19211", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.57/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"2a:33:f8:7a:88:77", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:49.965125 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:49.965686 containerd[1567]: 2025-08-13 01:30:49.959 [INFO][12922] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:49.965686 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.037424 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:50.037431 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:50.120046 containerd[1567]: 2025-08-13 01:30:50.033 [INFO][12957] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Aug 13 01:30:50.120046 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.120046 containerd[1567]: 2025-08-13 01:30:50.033 [INFO][12957] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" iface="eth0" netns="/var/run/netns/cni-348f9793-2881-6c03-848b-e0d8530e141f" Aug 13 01:30:50.120046 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.120046 containerd[1567]: 2025-08-13 01:30:50.034 [INFO][12957] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" iface="eth0" netns="/var/run/netns/cni-348f9793-2881-6c03-848b-e0d8530e141f" Aug 13 01:30:50.120046 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.120046 containerd[1567]: 2025-08-13 01:30:50.042 [INFO][12957] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" after=8.016515ms iface="eth0" netns="/var/run/netns/cni-348f9793-2881-6c03-848b-e0d8530e141f" Aug 13 01:30:50.120046 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.120046 containerd[1567]: 2025-08-13 01:30:50.042 [INFO][12957] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Aug 13 01:30:50.120046 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.120046 containerd[1567]: 2025-08-13 01:30:50.042 [INFO][12957] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Aug 13 01:30:50.120046 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.120046 containerd[1567]: 2025-08-13 01:30:50.072 [INFO][12968] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" HandleID="k8s-pod-network.21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:50.120046 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.120046 containerd[1567]: 2025-08-13 01:30:50.073 [INFO][12968] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:50.120046 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.120046 containerd[1567]: 2025-08-13 01:30:50.074 [INFO][12968] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:50.120046 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.120046 containerd[1567]: 2025-08-13 01:30:50.114 [INFO][12968] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" HandleID="k8s-pod-network.21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:50.120046 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.120046 containerd[1567]: 2025-08-13 01:30:50.114 [INFO][12968] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" HandleID="k8s-pod-network.21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:50.120046 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.120046 containerd[1567]: 2025-08-13 01:30:50.115 [INFO][12968] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:50.120046 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.120046 containerd[1567]: 2025-08-13 01:30:50.117 [INFO][12957] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7" Aug 13 01:30:50.120046 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.123320 systemd[1]: run-netns-cni\x2d348f9793\x2d2881\x2d6c03\x2d848b\x2de0d8530e141f.mount: Deactivated successfully. Aug 13 01:30:50.123769 containerd[1567]: time="2025-08-13T01:30:50.123636740Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7-lo: no space left on device" Aug 13 01:30:50.124128 kubelet[2720]: E0813 01:30:50.124020 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7-lo: no space left on device" Aug 13 01:30:50.124128 kubelet[2720]: E0813 01:30:50.124091 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:50.124707 kubelet[2720]: E0813 01:30:50.124109 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:50.124707 kubelet[2720]: E0813 01:30:50.124415 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:50.227312 systemd[1]: Started sshd@45-172.234.25.215:22-147.75.109.163:57690.service - OpenSSH per-connection server daemon (147.75.109.163:57690). Aug 13 01:30:50.564399 sshd[12977]: Accepted publickey for core from 147.75.109.163 port 57690 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:30:50.566114 sshd-session[12977]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:30:50.572127 systemd-logind[1543]: New session 46 of user core. Aug 13 01:30:50.577906 systemd[1]: Started session-46.scope - Session 46 of User core. Aug 13 01:30:50.837439 containerd[1567]: time="2025-08-13T01:30:50.837357134Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:50.873783 sshd[12979]: Connection closed by 147.75.109.163 port 57690 Aug 13 01:30:50.875399 sshd-session[12977]: pam_unix(sshd:session): session closed for user core Aug 13 01:30:50.881231 systemd[1]: sshd@45-172.234.25.215:22-147.75.109.163:57690.service: Deactivated successfully. Aug 13 01:30:50.884002 systemd[1]: session-46.scope: Deactivated successfully. Aug 13 01:30:50.887460 systemd-logind[1543]: Session 46 logged out. Waiting for processes to exit. Aug 13 01:30:50.890532 systemd-logind[1543]: Removed session 46. Aug 13 01:30:50.966160 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:50.968049 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.884 [INFO][12988] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 19278 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.885 [INFO][12988] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.923 [INFO][13002] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" HandleID="k8s-pod-network.6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.923 [INFO][13002] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" HandleID="k8s-pod-network.6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f100), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:50.923104746 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.923 [INFO][13002] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.923 [INFO][13002] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.923 [INFO][13002] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.929 [INFO][13002] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" host="172-234-25-215" Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.933 [INFO][13002] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.939 [INFO][13002] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.940 [INFO][13002] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.943 [INFO][13002] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.943 [INFO][13002] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" host="172-234-25-215" Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.945 [INFO][13002] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.948 [INFO][13002] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" host="172-234-25-215" Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.956 [INFO][13002] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.58/26] block=192.168.62.0/26 handle="k8s-pod-network.6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" host="172-234-25-215" Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.987737 containerd[1567]: 2025-08-13 01:30:50.956 [INFO][13002] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.58/26] handle="k8s-pod-network.6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" host="172-234-25-215" Aug 13 01:30:50.987737 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.990949 containerd[1567]: 2025-08-13 01:30:50.956 [INFO][13002] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:50.990949 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.990949 containerd[1567]: 2025-08-13 01:30:50.956 [INFO][13002] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.58/26] IPv6=[] ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" HandleID="k8s-pod-network.6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:50.990949 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.990949 containerd[1567]: 2025-08-13 01:30:50.960 [INFO][12988] cni-plugin/k8s.go 418: Populated endpoint ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19278", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"21b3bc3a5c9cf1b56671bc8d5d5dffbd9e4be2ab9705f60dbc578345aaaf85e7", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.58/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:50.990949 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.990949 containerd[1567]: 2025-08-13 01:30:50.960 [INFO][12988] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.58/32] ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:50.990949 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.990949 containerd[1567]: 2025-08-13 01:30:50.960 [INFO][12988] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:50.990949 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.990949 containerd[1567]: 2025-08-13 01:30:50.965 [INFO][12988] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:50.990949 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.990949 containerd[1567]: 2025-08-13 01:30:50.965 [INFO][12988] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19278", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.58/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"9a:de:5a:ca:f5:73", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:50.990949 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:50.991373 containerd[1567]: 2025-08-13 01:30:50.980 [INFO][12988] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:50.991373 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.066950 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:51.067077 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:51.164844 containerd[1567]: 2025-08-13 01:30:51.064 [INFO][13027] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Aug 13 01:30:51.164844 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.164844 containerd[1567]: 2025-08-13 01:30:51.064 [INFO][13027] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" iface="eth0" netns="/var/run/netns/cni-55b8104d-94ee-178f-30f0-0a16198f7068" Aug 13 01:30:51.164844 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.164844 containerd[1567]: 2025-08-13 01:30:51.064 [INFO][13027] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" iface="eth0" netns="/var/run/netns/cni-55b8104d-94ee-178f-30f0-0a16198f7068" Aug 13 01:30:51.164844 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.164844 containerd[1567]: 2025-08-13 01:30:51.074 [INFO][13027] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" after=10.164781ms iface="eth0" netns="/var/run/netns/cni-55b8104d-94ee-178f-30f0-0a16198f7068" Aug 13 01:30:51.164844 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.164844 containerd[1567]: 2025-08-13 01:30:51.075 [INFO][13027] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Aug 13 01:30:51.164844 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.164844 containerd[1567]: 2025-08-13 01:30:51.075 [INFO][13027] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Aug 13 01:30:51.164844 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.164844 containerd[1567]: 2025-08-13 01:30:51.122 [INFO][13036] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" HandleID="k8s-pod-network.6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:51.164844 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.164844 containerd[1567]: 2025-08-13 01:30:51.122 [INFO][13036] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:51.164844 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.164844 containerd[1567]: 2025-08-13 01:30:51.122 [INFO][13036] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:51.164844 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.164844 containerd[1567]: 2025-08-13 01:30:51.155 [INFO][13036] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" HandleID="k8s-pod-network.6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:51.164844 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.164844 containerd[1567]: 2025-08-13 01:30:51.155 [INFO][13036] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" HandleID="k8s-pod-network.6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:51.164844 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.164844 containerd[1567]: 2025-08-13 01:30:51.158 [INFO][13036] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:51.164844 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.164844 containerd[1567]: 2025-08-13 01:30:51.161 [INFO][13027] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd" Aug 13 01:30:51.164844 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.170317 containerd[1567]: time="2025-08-13T01:30:51.170221560Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd-lo: no space left on device" Aug 13 01:30:51.171376 kubelet[2720]: E0813 01:30:51.170965 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd-lo: no space left on device" Aug 13 01:30:51.174223 kubelet[2720]: E0813 01:30:51.171583 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:51.174223 kubelet[2720]: E0813 01:30:51.172167 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:51.174223 kubelet[2720]: E0813 01:30:51.172786 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:51.172512 systemd[1]: run-netns-cni\x2d55b8104d\x2d94ee\x2d178f\x2d30f0\x2d0a16198f7068.mount: Deactivated successfully. Aug 13 01:30:51.363168 containerd[1567]: time="2025-08-13T01:30:51.362890946Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:51.838585 containerd[1567]: time="2025-08-13T01:30:51.838544758Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:51.942931 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:51.945487 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.878 [INFO][13043] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 19349 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.879 [INFO][13043] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.902 [INFO][13056] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" HandleID="k8s-pod-network.356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.902 [INFO][13056] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" HandleID="k8s-pod-network.356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f7d0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:51.902450321 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.902 [INFO][13056] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.902 [INFO][13056] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.902 [INFO][13056] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.909 [INFO][13056] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" host="172-234-25-215" Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.914 [INFO][13056] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.919 [INFO][13056] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.921 [INFO][13056] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.923 [INFO][13056] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.924 [INFO][13056] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" host="172-234-25-215" Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.925 [INFO][13056] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.929 [INFO][13056] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" host="172-234-25-215" Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.935 [INFO][13056] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.59/26] block=192.168.62.0/26 handle="k8s-pod-network.356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" host="172-234-25-215" Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.967385 containerd[1567]: 2025-08-13 01:30:51.935 [INFO][13056] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.59/26] handle="k8s-pod-network.356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" host="172-234-25-215" Aug 13 01:30:51.967385 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.969092 containerd[1567]: 2025-08-13 01:30:51.935 [INFO][13056] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:51.969092 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.969092 containerd[1567]: 2025-08-13 01:30:51.935 [INFO][13056] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.59/26] IPv6=[] ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" HandleID="k8s-pod-network.356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:51.969092 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.969092 containerd[1567]: 2025-08-13 01:30:51.938 [INFO][13043] cni-plugin/k8s.go 418: Populated endpoint ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19349", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"6906a697d2d98eefdbc43219abe7d4174cb201c2d8e7e06192c1518831e4a7cd", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.59/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:51.969092 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.969092 containerd[1567]: 2025-08-13 01:30:51.938 [INFO][13043] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.59/32] ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:51.969092 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.969092 containerd[1567]: 2025-08-13 01:30:51.938 [INFO][13043] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:51.969092 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.969092 containerd[1567]: 2025-08-13 01:30:51.946 [INFO][13043] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:51.969092 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.969092 containerd[1567]: 2025-08-13 01:30:51.947 [INFO][13043] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19349", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.59/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"56:e8:e7:1b:3e:c8", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:51.969092 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:51.969437 containerd[1567]: 2025-08-13 01:30:51.962 [INFO][13043] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:51.969437 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.031598 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:52.032118 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:52.125881 containerd[1567]: 2025-08-13 01:30:52.028 [INFO][13081] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Aug 13 01:30:52.125881 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.125881 containerd[1567]: 2025-08-13 01:30:52.029 [INFO][13081] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" iface="eth0" netns="/var/run/netns/cni-60473623-ea74-7fb8-c066-fc1a2b7f5cb4" Aug 13 01:30:52.125881 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.125881 containerd[1567]: 2025-08-13 01:30:52.030 [INFO][13081] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" iface="eth0" netns="/var/run/netns/cni-60473623-ea74-7fb8-c066-fc1a2b7f5cb4" Aug 13 01:30:52.125881 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.125881 containerd[1567]: 2025-08-13 01:30:52.038 [INFO][13081] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" after=8.369694ms iface="eth0" netns="/var/run/netns/cni-60473623-ea74-7fb8-c066-fc1a2b7f5cb4" Aug 13 01:30:52.125881 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.125881 containerd[1567]: 2025-08-13 01:30:52.038 [INFO][13081] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Aug 13 01:30:52.125881 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.125881 containerd[1567]: 2025-08-13 01:30:52.038 [INFO][13081] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Aug 13 01:30:52.125881 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.125881 containerd[1567]: 2025-08-13 01:30:52.076 [INFO][13092] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" HandleID="k8s-pod-network.356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:52.125881 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.125881 containerd[1567]: 2025-08-13 01:30:52.079 [INFO][13092] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:52.125881 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.125881 containerd[1567]: 2025-08-13 01:30:52.079 [INFO][13092] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:52.125881 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.125881 containerd[1567]: 2025-08-13 01:30:52.119 [INFO][13092] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" HandleID="k8s-pod-network.356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:52.125881 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.125881 containerd[1567]: 2025-08-13 01:30:52.119 [INFO][13092] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" HandleID="k8s-pod-network.356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:52.125881 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.125881 containerd[1567]: 2025-08-13 01:30:52.121 [INFO][13092] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:52.125881 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.125881 containerd[1567]: 2025-08-13 01:30:52.123 [INFO][13081] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c" Aug 13 01:30:52.125881 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.129940 systemd[1]: run-netns-cni\x2d60473623\x2dea74\x2d7fb8\x2dc066\x2dfc1a2b7f5cb4.mount: Deactivated successfully. Aug 13 01:30:52.130862 containerd[1567]: time="2025-08-13T01:30:52.130233791Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c-lo: no space left on device" Aug 13 01:30:52.130936 kubelet[2720]: E0813 01:30:52.130542 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c-lo: no space left on device" Aug 13 01:30:52.130936 kubelet[2720]: E0813 01:30:52.130586 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:52.130936 kubelet[2720]: E0813 01:30:52.130606 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:52.130936 kubelet[2720]: E0813 01:30:52.130654 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:52.840698 containerd[1567]: time="2025-08-13T01:30:52.840624791Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:52.953043 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:52.954006 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.879 [INFO][13102] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 19411 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.879 [INFO][13102] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.904 [INFO][13113] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" HandleID="k8s-pod-network.2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.905 [INFO][13113] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" HandleID="k8s-pod-network.2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:52.904739203 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.905 [INFO][13113] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.905 [INFO][13113] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.905 [INFO][13113] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.912 [INFO][13113] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" host="172-234-25-215" Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.917 [INFO][13113] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.923 [INFO][13113] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.925 [INFO][13113] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.928 [INFO][13113] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.928 [INFO][13113] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" host="172-234-25-215" Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.931 [INFO][13113] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56 Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.934 [INFO][13113] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" host="172-234-25-215" Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.941 [INFO][13113] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.60/26] block=192.168.62.0/26 handle="k8s-pod-network.2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" host="172-234-25-215" Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.973063 containerd[1567]: 2025-08-13 01:30:52.941 [INFO][13113] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.60/26] handle="k8s-pod-network.2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" host="172-234-25-215" Aug 13 01:30:52.973063 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.975292 containerd[1567]: 2025-08-13 01:30:52.941 [INFO][13113] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:52.975292 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.975292 containerd[1567]: 2025-08-13 01:30:52.941 [INFO][13113] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.60/26] IPv6=[] ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" HandleID="k8s-pod-network.2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:52.975292 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.975292 containerd[1567]: 2025-08-13 01:30:52.944 [INFO][13102] cni-plugin/k8s.go 418: Populated endpoint ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19411", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"356f19461d1816f29e175f210c8f6fafb7e683e9f0f7c1d295b9e2346b3f289c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.60/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:52.975292 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.975292 containerd[1567]: 2025-08-13 01:30:52.944 [INFO][13102] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.60/32] ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:52.975292 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.975292 containerd[1567]: 2025-08-13 01:30:52.945 [INFO][13102] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:52.975292 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.975292 containerd[1567]: 2025-08-13 01:30:52.949 [INFO][13102] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:52.975292 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.975292 containerd[1567]: 2025-08-13 01:30:52.950 [INFO][13102] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19411", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.60/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"76:ed:d5:ef:6f:5b", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:52.975292 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:52.979431 containerd[1567]: 2025-08-13 01:30:52.963 [INFO][13102] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:52.979431 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.054819 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:53.054831 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:53.090854 containerd[1567]: time="2025-08-13T01:30:53.090772110Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:30:53.150824 containerd[1567]: 2025-08-13 01:30:53.050 [INFO][13136] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Aug 13 01:30:53.150824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.150824 containerd[1567]: 2025-08-13 01:30:53.051 [INFO][13136] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" iface="eth0" netns="/var/run/netns/cni-6bc36838-d109-eb59-390b-24075c16c637" Aug 13 01:30:53.150824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.150824 containerd[1567]: 2025-08-13 01:30:53.051 [INFO][13136] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" iface="eth0" netns="/var/run/netns/cni-6bc36838-d109-eb59-390b-24075c16c637" Aug 13 01:30:53.150824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.150824 containerd[1567]: 2025-08-13 01:30:53.060 [INFO][13136] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" after=8.345095ms iface="eth0" netns="/var/run/netns/cni-6bc36838-d109-eb59-390b-24075c16c637" Aug 13 01:30:53.150824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.150824 containerd[1567]: 2025-08-13 01:30:53.060 [INFO][13136] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Aug 13 01:30:53.150824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.150824 containerd[1567]: 2025-08-13 01:30:53.060 [INFO][13136] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Aug 13 01:30:53.150824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.150824 containerd[1567]: 2025-08-13 01:30:53.102 [INFO][13146] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" HandleID="k8s-pod-network.2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:53.150824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.150824 containerd[1567]: 2025-08-13 01:30:53.103 [INFO][13146] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:53.150824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.150824 containerd[1567]: 2025-08-13 01:30:53.104 [INFO][13146] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:53.150824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.150824 containerd[1567]: 2025-08-13 01:30:53.143 [INFO][13146] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" HandleID="k8s-pod-network.2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:53.150824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.150824 containerd[1567]: 2025-08-13 01:30:53.143 [INFO][13146] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" HandleID="k8s-pod-network.2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:53.150824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.150824 containerd[1567]: 2025-08-13 01:30:53.145 [INFO][13146] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:53.150824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.150824 containerd[1567]: 2025-08-13 01:30:53.148 [INFO][13136] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56" Aug 13 01:30:53.150824 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.155512 containerd[1567]: time="2025-08-13T01:30:53.155419730Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56-lo: no space left on device" Aug 13 01:30:53.156308 kubelet[2720]: E0813 01:30:53.156075 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56-lo: no space left on device" Aug 13 01:30:53.156592 kubelet[2720]: E0813 01:30:53.156470 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:53.156592 kubelet[2720]: E0813 01:30:53.156498 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:53.156427 systemd[1]: run-netns-cni\x2d6bc36838\x2dd109\x2deb59\x2d390b\x2d24075c16c637.mount: Deactivated successfully. Aug 13 01:30:53.157945 kubelet[2720]: E0813 01:30:53.156661 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:53.842246 containerd[1567]: time="2025-08-13T01:30:53.842186536Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:53.948622 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:53.950492 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.880 [INFO][13155] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 19480 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.881 [INFO][13155] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.911 [INFO][13168] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" HandleID="k8s-pod-network.0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.911 [INFO][13168] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" HandleID="k8s-pod-network.0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002ceff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:53.911179968 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.911 [INFO][13168] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.911 [INFO][13168] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.911 [INFO][13168] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.919 [INFO][13168] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" host="172-234-25-215" Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.922 [INFO][13168] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.926 [INFO][13168] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.927 [INFO][13168] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.929 [INFO][13168] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.929 [INFO][13168] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" host="172-234-25-215" Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.931 [INFO][13168] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61 Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.934 [INFO][13168] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" host="172-234-25-215" Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.939 [INFO][13168] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.61/26] block=192.168.62.0/26 handle="k8s-pod-network.0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" host="172-234-25-215" Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.972878 containerd[1567]: 2025-08-13 01:30:53.939 [INFO][13168] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.61/26] handle="k8s-pod-network.0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" host="172-234-25-215" Aug 13 01:30:53.972878 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.973579 containerd[1567]: 2025-08-13 01:30:53.939 [INFO][13168] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:53.973579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.973579 containerd[1567]: 2025-08-13 01:30:53.939 [INFO][13168] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.61/26] IPv6=[] ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" HandleID="k8s-pod-network.0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:53.973579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.973579 containerd[1567]: 2025-08-13 01:30:53.942 [INFO][13155] cni-plugin/k8s.go 418: Populated endpoint ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19480", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"2c2bb241e66ce0ad822fc0376a50628711178680f5970a58a130724acab38a56", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.61/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:53.973579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.973579 containerd[1567]: 2025-08-13 01:30:53.942 [INFO][13155] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.61/32] ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:53.973579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.973579 containerd[1567]: 2025-08-13 01:30:53.943 [INFO][13155] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:53.973579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.973579 containerd[1567]: 2025-08-13 01:30:53.953 [INFO][13155] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:53.973579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.973579 containerd[1567]: 2025-08-13 01:30:53.953 [INFO][13155] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19480", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.61/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"12:0b:e4:cb:21:da", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:53.973579 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:53.975930 containerd[1567]: 2025-08-13 01:30:53.966 [INFO][13155] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:53.975930 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.041270 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:54.041279 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:54.144522 containerd[1567]: 2025-08-13 01:30:54.038 [INFO][13193] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Aug 13 01:30:54.144522 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.144522 containerd[1567]: 2025-08-13 01:30:54.039 [INFO][13193] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" iface="eth0" netns="/var/run/netns/cni-c2710e3c-f5cc-bbbc-6100-f37c943ece56" Aug 13 01:30:54.144522 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.144522 containerd[1567]: 2025-08-13 01:30:54.039 [INFO][13193] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" iface="eth0" netns="/var/run/netns/cni-c2710e3c-f5cc-bbbc-6100-f37c943ece56" Aug 13 01:30:54.144522 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.144522 containerd[1567]: 2025-08-13 01:30:54.046 [INFO][13193] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" after=7.506886ms iface="eth0" netns="/var/run/netns/cni-c2710e3c-f5cc-bbbc-6100-f37c943ece56" Aug 13 01:30:54.144522 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.144522 containerd[1567]: 2025-08-13 01:30:54.047 [INFO][13193] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Aug 13 01:30:54.144522 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.144522 containerd[1567]: 2025-08-13 01:30:54.047 [INFO][13193] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Aug 13 01:30:54.144522 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.144522 containerd[1567]: 2025-08-13 01:30:54.099 [INFO][13200] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" HandleID="k8s-pod-network.0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:54.144522 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.144522 containerd[1567]: 2025-08-13 01:30:54.100 [INFO][13200] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:54.144522 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.144522 containerd[1567]: 2025-08-13 01:30:54.100 [INFO][13200] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:54.144522 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.144522 containerd[1567]: 2025-08-13 01:30:54.137 [INFO][13200] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" HandleID="k8s-pod-network.0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:54.144522 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.144522 containerd[1567]: 2025-08-13 01:30:54.137 [INFO][13200] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" HandleID="k8s-pod-network.0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:54.144522 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.144522 containerd[1567]: 2025-08-13 01:30:54.139 [INFO][13200] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:54.144522 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.144522 containerd[1567]: 2025-08-13 01:30:54.141 [INFO][13193] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61" Aug 13 01:30:54.144522 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.148672 containerd[1567]: time="2025-08-13T01:30:54.148621651Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61-lo: no space left on device" Aug 13 01:30:54.149046 kubelet[2720]: E0813 01:30:54.148980 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61-lo: no space left on device" Aug 13 01:30:54.149200 kubelet[2720]: E0813 01:30:54.149137 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:54.149190 systemd[1]: run-netns-cni\x2dc2710e3c\x2df5cc\x2dbbbc\x2d6100\x2df37c943ece56.mount: Deactivated successfully. Aug 13 01:30:54.150251 kubelet[2720]: E0813 01:30:54.149338 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:54.150251 kubelet[2720]: E0813 01:30:54.149985 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:54.362841 containerd[1567]: time="2025-08-13T01:30:54.362774496Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:54.844477 containerd[1567]: time="2025-08-13T01:30:54.844424768Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:54.943640 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:54.946502 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.881 [INFO][13210] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 19547 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.881 [INFO][13210] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.905 [INFO][13225] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" HandleID="k8s-pod-network.a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.906 [INFO][13225] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" HandleID="k8s-pod-network.a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f0f0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:54.905950645 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.906 [INFO][13225] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.906 [INFO][13225] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.906 [INFO][13225] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.911 [INFO][13225] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" host="172-234-25-215" Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.915 [INFO][13225] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.920 [INFO][13225] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.922 [INFO][13225] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.925 [INFO][13225] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.925 [INFO][13225] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" host="172-234-25-215" Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.927 [INFO][13225] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.931 [INFO][13225] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" host="172-234-25-215" Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.937 [INFO][13225] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.62/26] block=192.168.62.0/26 handle="k8s-pod-network.a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" host="172-234-25-215" Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.964037 containerd[1567]: 2025-08-13 01:30:54.937 [INFO][13225] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.62/26] handle="k8s-pod-network.a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" host="172-234-25-215" Aug 13 01:30:54.964037 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.965455 containerd[1567]: 2025-08-13 01:30:54.937 [INFO][13225] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:54.965455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.965455 containerd[1567]: 2025-08-13 01:30:54.937 [INFO][13225] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.62/26] IPv6=[] ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" HandleID="k8s-pod-network.a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:54.965455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.965455 containerd[1567]: 2025-08-13 01:30:54.940 [INFO][13210] cni-plugin/k8s.go 418: Populated endpoint ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19547", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0d0c099eff3f8d40d5f951bef138f55902aa0f04d0c605fb65c3a21979a7bc61", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.62/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:54.965455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.965455 containerd[1567]: 2025-08-13 01:30:54.940 [INFO][13210] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.62/32] ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:54.965455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.965455 containerd[1567]: 2025-08-13 01:30:54.940 [INFO][13210] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:54.965455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.965455 containerd[1567]: 2025-08-13 01:30:54.949 [INFO][13210] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:54.965455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.965455 containerd[1567]: 2025-08-13 01:30:54.949 [INFO][13210] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19547", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.62/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"8e:9b:e2:75:82:bc", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:54.965455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:54.967577 containerd[1567]: 2025-08-13 01:30:54.958 [INFO][13210] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:54.967577 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.077613 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:55.077625 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:55.249535 containerd[1567]: 2025-08-13 01:30:55.074 [INFO][13248] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Aug 13 01:30:55.249535 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.249535 containerd[1567]: 2025-08-13 01:30:55.075 [INFO][13248] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" iface="eth0" netns="/var/run/netns/cni-8e8aafe6-c44f-a38f-41ca-3816cd436d77" Aug 13 01:30:55.249535 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.249535 containerd[1567]: 2025-08-13 01:30:55.075 [INFO][13248] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" iface="eth0" netns="/var/run/netns/cni-8e8aafe6-c44f-a38f-41ca-3816cd436d77" Aug 13 01:30:55.249535 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.249535 containerd[1567]: 2025-08-13 01:30:55.082 [INFO][13248] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" after=7.230846ms iface="eth0" netns="/var/run/netns/cni-8e8aafe6-c44f-a38f-41ca-3816cd436d77" Aug 13 01:30:55.249535 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.249535 containerd[1567]: 2025-08-13 01:30:55.085 [INFO][13248] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Aug 13 01:30:55.249535 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.249535 containerd[1567]: 2025-08-13 01:30:55.085 [INFO][13248] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Aug 13 01:30:55.249535 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.249535 containerd[1567]: 2025-08-13 01:30:55.159 [INFO][13256] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" HandleID="k8s-pod-network.a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:55.249535 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.249535 containerd[1567]: 2025-08-13 01:30:55.162 [INFO][13256] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:55.249535 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.249535 containerd[1567]: 2025-08-13 01:30:55.162 [INFO][13256] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:55.249535 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.249535 containerd[1567]: 2025-08-13 01:30:55.241 [INFO][13256] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" HandleID="k8s-pod-network.a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:55.249535 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.249535 containerd[1567]: 2025-08-13 01:30:55.241 [INFO][13256] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" HandleID="k8s-pod-network.a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:55.249535 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.249535 containerd[1567]: 2025-08-13 01:30:55.243 [INFO][13256] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:55.249535 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.249535 containerd[1567]: 2025-08-13 01:30:55.245 [INFO][13248] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad" Aug 13 01:30:55.249535 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.254698 systemd[1]: run-netns-cni\x2d8e8aafe6\x2dc44f\x2da38f\x2d41ca\x2d3816cd436d77.mount: Deactivated successfully. Aug 13 01:30:55.256509 containerd[1567]: time="2025-08-13T01:30:55.256117330Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad-lo: no space left on device" Aug 13 01:30:55.258338 kubelet[2720]: E0813 01:30:55.258035 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad-lo: no space left on device" Aug 13 01:30:55.258338 kubelet[2720]: E0813 01:30:55.258099 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:55.258338 kubelet[2720]: E0813 01:30:55.258119 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:55.259619 kubelet[2720]: E0813 01:30:55.259414 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:55.848846 containerd[1567]: time="2025-08-13T01:30:55.848798258Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:55.939046 systemd[1]: Started sshd@46-172.234.25.215:22-147.75.109.163:57704.service - OpenSSH per-connection server daemon (147.75.109.163:57704). Aug 13 01:30:55.979399 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:55.979607 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.886 [INFO][13269] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 19607 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.887 [INFO][13269] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.919 [INFO][13280] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" HandleID="k8s-pod-network.db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.919 [INFO][13280] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" HandleID="k8s-pod-network.db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc000259060), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:55.919248088 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.919 [INFO][13280] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.919 [INFO][13280] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.919 [INFO][13280] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.926 [INFO][13280] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" host="172-234-25-215" Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.931 [INFO][13280] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.941 [INFO][13280] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.943 [INFO][13280] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.949 [INFO][13280] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.950 [INFO][13280] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" host="172-234-25-215" Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.953 [INFO][13280] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.964 [INFO][13280] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" host="172-234-25-215" Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.972 [INFO][13280] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.63/26] block=192.168.62.0/26 handle="k8s-pod-network.db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" host="172-234-25-215" Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.994764 containerd[1567]: 2025-08-13 01:30:55.972 [INFO][13280] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.63/26] handle="k8s-pod-network.db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" host="172-234-25-215" Aug 13 01:30:55.994764 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.996303 containerd[1567]: 2025-08-13 01:30:55.972 [INFO][13280] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:55.996303 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.996303 containerd[1567]: 2025-08-13 01:30:55.972 [INFO][13280] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.63/26] IPv6=[] ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" HandleID="k8s-pod-network.db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:55.996303 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.996303 containerd[1567]: 2025-08-13 01:30:55.976 [INFO][13269] cni-plugin/k8s.go 418: Populated endpoint ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19607", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"a61f880e83cd2b6f9dc2b23f4b9e10d9a4fd001e729afa61d7c415b5a2e939ad", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.63/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:55.996303 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.996303 containerd[1567]: 2025-08-13 01:30:55.976 [INFO][13269] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.63/32] ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:55.996303 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.996303 containerd[1567]: 2025-08-13 01:30:55.976 [INFO][13269] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:55.996303 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.996303 containerd[1567]: 2025-08-13 01:30:55.978 [INFO][13269] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:55.996303 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.996303 containerd[1567]: 2025-08-13 01:30:55.979 [INFO][13269] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19607", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.63/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"66:62:9f:c9:72:09", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:55.996303 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:55.997226 containerd[1567]: 2025-08-13 01:30:55.987 [INFO][13269] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:55.997226 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.077752 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:56.077766 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:56.081796 kubelet[2720]: E0813 01:30:56.081163 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:30:56.095233 kubelet[2720]: E0813 01:30:56.094695 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-299933554-aTh- sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2568257827: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:30:56.216847 containerd[1567]: 2025-08-13 01:30:56.073 [INFO][13306] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Aug 13 01:30:56.216847 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.216847 containerd[1567]: 2025-08-13 01:30:56.074 [INFO][13306] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" iface="eth0" netns="/var/run/netns/cni-f84de670-ea97-81d7-ea61-5c87c53ffa1c" Aug 13 01:30:56.216847 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.216847 containerd[1567]: 2025-08-13 01:30:56.075 [INFO][13306] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" iface="eth0" netns="/var/run/netns/cni-f84de670-ea97-81d7-ea61-5c87c53ffa1c" Aug 13 01:30:56.216847 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.216847 containerd[1567]: 2025-08-13 01:30:56.087 [INFO][13306] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" after=12.644396ms iface="eth0" netns="/var/run/netns/cni-f84de670-ea97-81d7-ea61-5c87c53ffa1c" Aug 13 01:30:56.216847 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.216847 containerd[1567]: 2025-08-13 01:30:56.087 [INFO][13306] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Aug 13 01:30:56.216847 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.216847 containerd[1567]: 2025-08-13 01:30:56.087 [INFO][13306] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Aug 13 01:30:56.216847 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.216847 containerd[1567]: 2025-08-13 01:30:56.144 [INFO][13315] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" HandleID="k8s-pod-network.db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:56.216847 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.216847 containerd[1567]: 2025-08-13 01:30:56.147 [INFO][13315] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:56.216847 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.216847 containerd[1567]: 2025-08-13 01:30:56.148 [INFO][13315] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:56.216847 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.216847 containerd[1567]: 2025-08-13 01:30:56.203 [INFO][13315] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" HandleID="k8s-pod-network.db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:56.216847 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.216847 containerd[1567]: 2025-08-13 01:30:56.203 [INFO][13315] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" HandleID="k8s-pod-network.db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:56.216847 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.216847 containerd[1567]: 2025-08-13 01:30:56.205 [INFO][13315] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:56.216847 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.216847 containerd[1567]: 2025-08-13 01:30:56.212 [INFO][13306] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c" Aug 13 01:30:56.216847 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.220769 containerd[1567]: time="2025-08-13T01:30:56.220682703Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c-lo: no space left on device" Aug 13 01:30:56.221183 kubelet[2720]: E0813 01:30:56.221151 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c-lo: no space left on device" Aug 13 01:30:56.221270 kubelet[2720]: E0813 01:30:56.221254 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:56.221328 kubelet[2720]: E0813 01:30:56.221314 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:56.221422 kubelet[2720]: E0813 01:30:56.221399 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:56.222332 systemd[1]: run-netns-cni\x2df84de670\x2dea97\x2d81d7\x2dea61\x2d5c87c53ffa1c.mount: Deactivated successfully. Aug 13 01:30:56.291374 sshd[13287]: Accepted publickey for core from 147.75.109.163 port 57704 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:30:56.291960 sshd-session[13287]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:30:56.297910 systemd-logind[1543]: New session 47 of user core. Aug 13 01:30:56.305046 systemd[1]: Started session-47.scope - Session 47 of User core. Aug 13 01:30:56.430705 containerd[1567]: time="2025-08-13T01:30:56.430592786Z" level=warning msg="container event discarded" container=86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382 type=CONTAINER_STOPPED_EVENT Aug 13 01:30:56.477129 containerd[1567]: time="2025-08-13T01:30:56.477029160Z" level=warning msg="container event discarded" container=19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec type=CONTAINER_STOPPED_EVENT Aug 13 01:30:56.600997 sshd[13323]: Connection closed by 147.75.109.163 port 57704 Aug 13 01:30:56.602377 sshd-session[13287]: pam_unix(sshd:session): session closed for user core Aug 13 01:30:56.607902 systemd[1]: sshd@46-172.234.25.215:22-147.75.109.163:57704.service: Deactivated successfully. Aug 13 01:30:56.612078 systemd[1]: session-47.scope: Deactivated successfully. Aug 13 01:30:56.615553 systemd-logind[1543]: Session 47 logged out. Waiting for processes to exit. Aug 13 01:30:56.616964 systemd-logind[1543]: Removed session 47. Aug 13 01:30:56.851432 containerd[1567]: time="2025-08-13T01:30:56.851349970Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:56.961122 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:56.962665 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.891 [INFO][13335] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 19675 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.892 [INFO][13335] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.918 [INFO][13348] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" HandleID="k8s-pod-network.f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.918 [INFO][13348] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" HandleID="k8s-pod-network.f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5710), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:56.918043577 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.918 [INFO][13348] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.918 [INFO][13348] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.918 [INFO][13348] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.923 [INFO][13348] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" host="172-234-25-215" Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.927 [INFO][13348] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.930 [INFO][13348] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.932 [INFO][13348] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.934 [INFO][13348] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.934 [INFO][13348] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" host="172-234-25-215" Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.936 [INFO][13348] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227 Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.939 [INFO][13348] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" host="172-234-25-215" Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.948 [INFO][13348] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.4/26] block=192.168.62.0/26 handle="k8s-pod-network.f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" host="172-234-25-215" Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.980906 containerd[1567]: 2025-08-13 01:30:56.948 [INFO][13348] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.4/26] handle="k8s-pod-network.f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" host="172-234-25-215" Aug 13 01:30:56.980906 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.982653 containerd[1567]: 2025-08-13 01:30:56.948 [INFO][13348] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:56.982653 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.982653 containerd[1567]: 2025-08-13 01:30:56.949 [INFO][13348] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.4/26] IPv6=[] ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" HandleID="k8s-pod-network.f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:56.982653 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.982653 containerd[1567]: 2025-08-13 01:30:56.952 [INFO][13335] cni-plugin/k8s.go 418: Populated endpoint ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19675", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"db4996d5416ff16c84eee1ead45a8c82d3e573f95eaa54cd9f1b5d45ad2ead6c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.4/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:56.982653 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.982653 containerd[1567]: 2025-08-13 01:30:56.952 [INFO][13335] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.4/32] ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:56.982653 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.982653 containerd[1567]: 2025-08-13 01:30:56.952 [INFO][13335] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:56.982653 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.982653 containerd[1567]: 2025-08-13 01:30:56.965 [INFO][13335] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:56.982653 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.982653 containerd[1567]: 2025-08-13 01:30:56.965 [INFO][13335] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19675", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.4/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"56:44:a3:e5:9e:08", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:56.982653 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:56.985012 containerd[1567]: 2025-08-13 01:30:56.974 [INFO][13335] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:56.985012 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:57.044287 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:57.045145 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:57.139186 containerd[1567]: 2025-08-13 01:30:57.040 [INFO][13371] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Aug 13 01:30:57.139186 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:57.139186 containerd[1567]: 2025-08-13 01:30:57.041 [INFO][13371] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" iface="eth0" netns="/var/run/netns/cni-d314a63a-4629-b439-9f82-822cd60acc54" Aug 13 01:30:57.139186 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:57.139186 containerd[1567]: 2025-08-13 01:30:57.042 [INFO][13371] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" iface="eth0" netns="/var/run/netns/cni-d314a63a-4629-b439-9f82-822cd60acc54" Aug 13 01:30:57.139186 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:57.139186 containerd[1567]: 2025-08-13 01:30:57.049 [INFO][13371] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" after=7.853535ms iface="eth0" netns="/var/run/netns/cni-d314a63a-4629-b439-9f82-822cd60acc54" Aug 13 01:30:57.139186 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:57.139186 containerd[1567]: 2025-08-13 01:30:57.049 [INFO][13371] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Aug 13 01:30:57.139186 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:57.139186 containerd[1567]: 2025-08-13 01:30:57.049 [INFO][13371] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Aug 13 01:30:57.139186 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:57.139186 containerd[1567]: 2025-08-13 01:30:57.095 [INFO][13380] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" HandleID="k8s-pod-network.f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:57.139186 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:57.139186 containerd[1567]: 2025-08-13 01:30:57.095 [INFO][13380] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:57.139186 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:57.139186 containerd[1567]: 2025-08-13 01:30:57.096 [INFO][13380] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:57.139186 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:57.139186 containerd[1567]: 2025-08-13 01:30:57.132 [INFO][13380] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" HandleID="k8s-pod-network.f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:57.139186 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:57.139186 containerd[1567]: 2025-08-13 01:30:57.132 [INFO][13380] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" HandleID="k8s-pod-network.f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:57.139186 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:57.139186 containerd[1567]: 2025-08-13 01:30:57.134 [INFO][13380] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:57.139186 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:57.139186 containerd[1567]: 2025-08-13 01:30:57.136 [INFO][13371] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227" Aug 13 01:30:57.139186 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:57.143614 containerd[1567]: time="2025-08-13T01:30:57.143488622Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227-lo: no space left on device" Aug 13 01:30:57.144091 kubelet[2720]: E0813 01:30:57.144055 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227-lo: no space left on device" Aug 13 01:30:57.144625 kubelet[2720]: E0813 01:30:57.144106 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:57.144625 kubelet[2720]: E0813 01:30:57.144127 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:57.144625 kubelet[2720]: E0813 01:30:57.144176 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:57.145031 systemd[1]: run-netns-cni\x2dd314a63a\x2d4629\x2db439\x2d9f82\x2d822cd60acc54.mount: Deactivated successfully. Aug 13 01:30:57.218508 containerd[1567]: time="2025-08-13T01:30:57.218420964Z" level=warning msg="container event discarded" container=86dda32658ae0c799dd2b9356d3bd78b7b11df15454ed1fdaf85ff0a07c50382 type=CONTAINER_DELETED_EVENT Aug 13 01:30:57.363101 containerd[1567]: time="2025-08-13T01:30:57.363052498Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:30:57.490858 containerd[1567]: time="2025-08-13T01:30:57.490806863Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-controller-manager-172-234-25-215_96c354cf60eb29e83ce001f635cc0212/kube-controller-manager/0.log\"" error="write /var/log/pods/kube-system_kube-controller-manager-172-234-25-215_96c354cf60eb29e83ce001f635cc0212/kube-controller-manager/0.log: no space left on device" Aug 13 01:30:57.507049 containerd[1567]: time="2025-08-13T01:30:57.506992953Z" level=warning msg="container event discarded" container=19994eb90cedb84f25a096d6f991af0f8c3852814b623efa7eaa3eb89da7dbec type=CONTAINER_DELETED_EVENT Aug 13 01:30:57.853518 containerd[1567]: time="2025-08-13T01:30:57.853430794Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:57.980113 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:57.984286 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:57.998025 containerd[1567]: time="2025-08-13T01:30:57.997630018Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-controller-manager-172-234-25-215_96c354cf60eb29e83ce001f635cc0212/kube-controller-manager/0.log\"" error="write /var/log/pods/kube-system_kube-controller-manager-172-234-25-215_96c354cf60eb29e83ce001f635cc0212/kube-controller-manager/0.log: no space left on device" Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.897 [INFO][13390] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 19732 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.897 [INFO][13390] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.931 [INFO][13403] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" HandleID="k8s-pod-network.2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.932 [INFO][13403] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" HandleID="k8s-pod-network.2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cf270), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:57.93181069 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.932 [INFO][13403] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.932 [INFO][13403] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.932 [INFO][13403] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.939 [INFO][13403] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" host="172-234-25-215" Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.948 [INFO][13403] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.954 [INFO][13403] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.956 [INFO][13403] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.960 [INFO][13403] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.961 [INFO][13403] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" host="172-234-25-215" Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.963 [INFO][13403] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807 Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.966 [INFO][13403] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" host="172-234-25-215" Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.973 [INFO][13403] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.5/26] block=192.168.62.0/26 handle="k8s-pod-network.2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" host="172-234-25-215" Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.012279 containerd[1567]: 2025-08-13 01:30:57.973 [INFO][13403] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.5/26] handle="k8s-pod-network.2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" host="172-234-25-215" Aug 13 01:30:58.012279 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.014581 containerd[1567]: 2025-08-13 01:30:57.973 [INFO][13403] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:58.014581 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.014581 containerd[1567]: 2025-08-13 01:30:57.973 [INFO][13403] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.5/26] IPv6=[] ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" HandleID="k8s-pod-network.2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:58.014581 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.014581 containerd[1567]: 2025-08-13 01:30:57.976 [INFO][13390] cni-plugin/k8s.go 418: Populated endpoint ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19732", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"f5563e6ecc1c849e6dbfc1061cc5bdef84e86fba783cddd5928e1d7ce5931227", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.5/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:58.014581 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.014581 containerd[1567]: 2025-08-13 01:30:57.976 [INFO][13390] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.5/32] ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:58.014581 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.014581 containerd[1567]: 2025-08-13 01:30:57.976 [INFO][13390] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:58.014581 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.014581 containerd[1567]: 2025-08-13 01:30:57.990 [INFO][13390] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:58.014581 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.014581 containerd[1567]: 2025-08-13 01:30:57.992 [INFO][13390] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19732", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.5/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"9a:e4:c0:df:3b:75", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:58.014581 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.016198 containerd[1567]: 2025-08-13 01:30:58.001 [INFO][13390] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:58.016198 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.074703 kubelet[2720]: E0813 01:30:58.073602 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:30:58.076143 kubelet[2720]: E0813 01:30:58.075568 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:30:58.077911 kubelet[2720]: E0813 01:30:58.077880 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-349524588-hiNR sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2074644924: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:30:58.089593 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:58.089602 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:58.210554 containerd[1567]: 2025-08-13 01:30:58.087 [INFO][13426] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Aug 13 01:30:58.210554 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.210554 containerd[1567]: 2025-08-13 01:30:58.087 [INFO][13426] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" iface="eth0" netns="/var/run/netns/cni-4a84e689-e4c8-f6f7-9222-80794b6d7512" Aug 13 01:30:58.210554 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.210554 containerd[1567]: 2025-08-13 01:30:58.089 [INFO][13426] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" iface="eth0" netns="/var/run/netns/cni-4a84e689-e4c8-f6f7-9222-80794b6d7512" Aug 13 01:30:58.210554 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.210554 containerd[1567]: 2025-08-13 01:30:58.100 [INFO][13426] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" after=11.964018ms iface="eth0" netns="/var/run/netns/cni-4a84e689-e4c8-f6f7-9222-80794b6d7512" Aug 13 01:30:58.210554 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.210554 containerd[1567]: 2025-08-13 01:30:58.100 [INFO][13426] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Aug 13 01:30:58.210554 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.210554 containerd[1567]: 2025-08-13 01:30:58.100 [INFO][13426] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Aug 13 01:30:58.210554 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.210554 containerd[1567]: 2025-08-13 01:30:58.145 [INFO][13435] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" HandleID="k8s-pod-network.2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:58.210554 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.210554 containerd[1567]: 2025-08-13 01:30:58.145 [INFO][13435] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:58.210554 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.210554 containerd[1567]: 2025-08-13 01:30:58.146 [INFO][13435] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:58.210554 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.210554 containerd[1567]: 2025-08-13 01:30:58.200 [INFO][13435] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" HandleID="k8s-pod-network.2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:58.210554 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.210554 containerd[1567]: 2025-08-13 01:30:58.200 [INFO][13435] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" HandleID="k8s-pod-network.2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:58.210554 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.210554 containerd[1567]: 2025-08-13 01:30:58.205 [INFO][13435] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:58.210554 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.210554 containerd[1567]: 2025-08-13 01:30:58.207 [INFO][13426] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807" Aug 13 01:30:58.210554 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:58.214795 containerd[1567]: time="2025-08-13T01:30:58.214683128Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807-lo: no space left on device" Aug 13 01:30:58.215349 systemd[1]: run-netns-cni\x2d4a84e689\x2de4c8\x2df6f7\x2d9222\x2d80794b6d7512.mount: Deactivated successfully. Aug 13 01:30:58.217500 kubelet[2720]: E0813 01:30:58.215920 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807-lo: no space left on device" Aug 13 01:30:58.217500 kubelet[2720]: E0813 01:30:58.215968 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:58.217500 kubelet[2720]: E0813 01:30:58.215985 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:58.217500 kubelet[2720]: E0813 01:30:58.216031 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:58.496251 kubelet[2720]: I0813 01:30:58.496181 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:30:58.496540 kubelet[2720]: I0813 01:30:58.496388 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:30:58.498265 kubelet[2720]: I0813 01:30:58.498242 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:30:58.516047 kubelet[2720]: I0813 01:30:58.516025 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:30:58.516168 kubelet[2720]: I0813 01:30:58.516149 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:30:58.516240 kubelet[2720]: E0813 01:30:58.516181 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:58.516240 kubelet[2720]: E0813 01:30:58.516190 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:30:58.516240 kubelet[2720]: E0813 01:30:58.516207 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:30:58.516240 kubelet[2720]: E0813 01:30:58.516216 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:30:58.516240 kubelet[2720]: E0813 01:30:58.516226 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:30:58.516240 kubelet[2720]: E0813 01:30:58.516234 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:30:58.516240 kubelet[2720]: E0813 01:30:58.516241 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:30:58.516423 kubelet[2720]: E0813 01:30:58.516250 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:30:58.516423 kubelet[2720]: E0813 01:30:58.516256 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:30:58.516423 kubelet[2720]: E0813 01:30:58.516264 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:30:58.516423 kubelet[2720]: I0813 01:30:58.516275 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:30:58.774943 containerd[1567]: time="2025-08-13T01:30:58.774811376Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-typha-5bf7f956b7-szw5v_d0c18708-6c9b-42ce-bbc4-7fe0748e1776/calico-typha/0.log\"" error="write /var/log/pods/calico-system_calico-typha-5bf7f956b7-szw5v_d0c18708-6c9b-42ce-bbc4-7fe0748e1776/calico-typha/0.log: no space left on device" Aug 13 01:30:58.856318 containerd[1567]: time="2025-08-13T01:30:58.856199906Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:58.979446 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:58.980282 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.901 [INFO][13448] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 19815 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.902 [INFO][13448] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.934 [INFO][13459] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" HandleID="k8s-pod-network.78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.935 [INFO][13459] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" HandleID="k8s-pod-network.78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:58.934872951 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.935 [INFO][13459] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.935 [INFO][13459] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.935 [INFO][13459] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.942 [INFO][13459] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" host="172-234-25-215" Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.947 [INFO][13459] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.952 [INFO][13459] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.956 [INFO][13459] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.959 [INFO][13459] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.959 [INFO][13459] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" host="172-234-25-215" Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.960 [INFO][13459] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98 Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.964 [INFO][13459] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" host="172-234-25-215" Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.972 [INFO][13459] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.6/26] block=192.168.62.0/26 handle="k8s-pod-network.78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" host="172-234-25-215" Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.000266 containerd[1567]: 2025-08-13 01:30:58.972 [INFO][13459] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.6/26] handle="k8s-pod-network.78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" host="172-234-25-215" Aug 13 01:30:59.000266 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.003773 containerd[1567]: 2025-08-13 01:30:58.972 [INFO][13459] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:59.003773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.003773 containerd[1567]: 2025-08-13 01:30:58.972 [INFO][13459] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.6/26] IPv6=[] ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" HandleID="k8s-pod-network.78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.003773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.003773 containerd[1567]: 2025-08-13 01:30:58.975 [INFO][13448] cni-plugin/k8s.go 418: Populated endpoint ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19815", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"2d3f7e9a67f5408f8b61eaf8029f80b5797f52853d5156ffaa71125760bdd807", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.6/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:59.003773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.003773 containerd[1567]: 2025-08-13 01:30:58.975 [INFO][13448] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.6/32] ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.003773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.003773 containerd[1567]: 2025-08-13 01:30:58.975 [INFO][13448] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.003773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.003773 containerd[1567]: 2025-08-13 01:30:58.980 [INFO][13448] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.003773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.003773 containerd[1567]: 2025-08-13 01:30:58.981 [INFO][13448] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19815", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.6/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"f2:10:1e:73:96:8c", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:59.003773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.004086 containerd[1567]: 2025-08-13 01:30:58.991 [INFO][13448] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.004086 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.071316 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:30:59.071540 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:30:59.175104 containerd[1567]: 2025-08-13 01:30:59.069 [INFO][13483] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Aug 13 01:30:59.175104 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.175104 containerd[1567]: 2025-08-13 01:30:59.069 [INFO][13483] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" iface="eth0" netns="/var/run/netns/cni-dc3fd0d0-1517-3ad4-0120-9736f589f7be" Aug 13 01:30:59.175104 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.175104 containerd[1567]: 2025-08-13 01:30:59.070 [INFO][13483] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" iface="eth0" netns="/var/run/netns/cni-dc3fd0d0-1517-3ad4-0120-9736f589f7be" Aug 13 01:30:59.175104 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.175104 containerd[1567]: 2025-08-13 01:30:59.079 [INFO][13483] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" after=9.353582ms iface="eth0" netns="/var/run/netns/cni-dc3fd0d0-1517-3ad4-0120-9736f589f7be" Aug 13 01:30:59.175104 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.175104 containerd[1567]: 2025-08-13 01:30:59.079 [INFO][13483] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Aug 13 01:30:59.175104 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.175104 containerd[1567]: 2025-08-13 01:30:59.079 [INFO][13483] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Aug 13 01:30:59.175104 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.175104 containerd[1567]: 2025-08-13 01:30:59.126 [INFO][13492] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" HandleID="k8s-pod-network.78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.175104 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.175104 containerd[1567]: 2025-08-13 01:30:59.126 [INFO][13492] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:59.175104 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.175104 containerd[1567]: 2025-08-13 01:30:59.126 [INFO][13492] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:59.175104 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.175104 containerd[1567]: 2025-08-13 01:30:59.168 [INFO][13492] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" HandleID="k8s-pod-network.78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.175104 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.175104 containerd[1567]: 2025-08-13 01:30:59.168 [INFO][13492] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" HandleID="k8s-pod-network.78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.175104 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.175104 containerd[1567]: 2025-08-13 01:30:59.170 [INFO][13492] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:59.175104 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.175104 containerd[1567]: 2025-08-13 01:30:59.172 [INFO][13483] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98" Aug 13 01:30:59.175104 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.177606 containerd[1567]: time="2025-08-13T01:30:59.177553283Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98-lo: no space left on device" Aug 13 01:30:59.177907 kubelet[2720]: E0813 01:30:59.177803 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98-lo: no space left on device" Aug 13 01:30:59.177952 kubelet[2720]: E0813 01:30:59.177919 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:59.178151 systemd[1]: run-netns-cni\x2ddc3fd0d0\x2d1517\x2d3ad4\x2d0120\x2d9736f589f7be.mount: Deactivated successfully. Aug 13 01:30:59.179502 kubelet[2720]: E0813 01:30:59.178651 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:30:59.179502 kubelet[2720]: E0813 01:30:59.178779 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:30:59.859731 containerd[1567]: time="2025-08-13T01:30:59.859631737Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:30:59.964184 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:30:59.965587 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.896 [INFO][13503] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 19883 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.897 [INFO][13503] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.925 [INFO][13515] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" HandleID="k8s-pod-network.1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.925 [INFO][13515] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" HandleID="k8s-pod-network.1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cd8a0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:30:59.925106907 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.925 [INFO][13515] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.925 [INFO][13515] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.925 [INFO][13515] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.930 [INFO][13515] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" host="172-234-25-215" Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.934 [INFO][13515] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.938 [INFO][13515] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.939 [INFO][13515] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.941 [INFO][13515] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.942 [INFO][13515] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" host="172-234-25-215" Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.943 [INFO][13515] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3 Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.946 [INFO][13515] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" host="172-234-25-215" Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.954 [INFO][13515] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.7/26] block=192.168.62.0/26 handle="k8s-pod-network.1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" host="172-234-25-215" Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.986415 containerd[1567]: 2025-08-13 01:30:59.954 [INFO][13515] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.7/26] handle="k8s-pod-network.1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" host="172-234-25-215" Aug 13 01:30:59.986415 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.989061 containerd[1567]: 2025-08-13 01:30:59.954 [INFO][13515] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:30:59.989061 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.989061 containerd[1567]: 2025-08-13 01:30:59.954 [INFO][13515] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.7/26] IPv6=[] ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" HandleID="k8s-pod-network.1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.989061 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.989061 containerd[1567]: 2025-08-13 01:30:59.959 [INFO][13503] cni-plugin/k8s.go 418: Populated endpoint ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19883", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"78c935aea25168f9ca3a6ae1b38a465664c82b83fe39bbfb07bc22d52a52ae98", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.7/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:59.989061 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.989061 containerd[1567]: 2025-08-13 01:30:59.959 [INFO][13503] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.7/32] ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.989061 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.989061 containerd[1567]: 2025-08-13 01:30:59.959 [INFO][13503] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.989061 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.989061 containerd[1567]: 2025-08-13 01:30:59.964 [INFO][13503] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.989061 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.989061 containerd[1567]: 2025-08-13 01:30:59.966 [INFO][13503] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19883", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.7/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"02:ff:3b:4d:c7:8a", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:30:59.989061 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:30:59.989395 containerd[1567]: 2025-08-13 01:30:59.981 [INFO][13503] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:30:59.989395 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:00.063887 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:00.063898 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:00.082262 kubelet[2720]: E0813 01:31:00.082223 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:31:00.156964 containerd[1567]: 2025-08-13 01:31:00.061 [INFO][13537] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Aug 13 01:31:00.156964 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:00.156964 containerd[1567]: 2025-08-13 01:31:00.062 [INFO][13537] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" iface="eth0" netns="/var/run/netns/cni-1ec2e339-dd63-f6fe-4b9b-f6cbed158b76" Aug 13 01:31:00.156964 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:00.156964 containerd[1567]: 2025-08-13 01:31:00.062 [INFO][13537] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" iface="eth0" netns="/var/run/netns/cni-1ec2e339-dd63-f6fe-4b9b-f6cbed158b76" Aug 13 01:31:00.156964 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:00.156964 containerd[1567]: 2025-08-13 01:31:00.071 [INFO][13537] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" after=8.608514ms iface="eth0" netns="/var/run/netns/cni-1ec2e339-dd63-f6fe-4b9b-f6cbed158b76" Aug 13 01:31:00.156964 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:00.156964 containerd[1567]: 2025-08-13 01:31:00.071 [INFO][13537] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Aug 13 01:31:00.156964 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:00.156964 containerd[1567]: 2025-08-13 01:31:00.071 [INFO][13537] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Aug 13 01:31:00.156964 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:00.156964 containerd[1567]: 2025-08-13 01:31:00.116 [INFO][13547] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" HandleID="k8s-pod-network.1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:00.156964 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:00.156964 containerd[1567]: 2025-08-13 01:31:00.116 [INFO][13547] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:00.156964 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:00.156964 containerd[1567]: 2025-08-13 01:31:00.116 [INFO][13547] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:00.156964 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:00.156964 containerd[1567]: 2025-08-13 01:31:00.148 [INFO][13547] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" HandleID="k8s-pod-network.1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:00.156964 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:00.156964 containerd[1567]: 2025-08-13 01:31:00.149 [INFO][13547] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" HandleID="k8s-pod-network.1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:00.156964 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:00.156964 containerd[1567]: 2025-08-13 01:31:00.151 [INFO][13547] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:00.156964 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:00.156964 containerd[1567]: 2025-08-13 01:31:00.153 [INFO][13537] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3" Aug 13 01:31:00.156964 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:00.161542 containerd[1567]: time="2025-08-13T01:31:00.161497311Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3-lo: no space left on device" Aug 13 01:31:00.161774 kubelet[2720]: E0813 01:31:00.161703 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3-lo: no space left on device" Aug 13 01:31:00.161817 kubelet[2720]: E0813 01:31:00.161777 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:00.161817 kubelet[2720]: E0813 01:31:00.161798 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:00.161867 kubelet[2720]: E0813 01:31:00.161842 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:00.162188 systemd[1]: run-netns-cni\x2d1ec2e339\x2ddd63\x2df6fe\x2d4b9b\x2df6cbed158b76.mount: Deactivated successfully. Aug 13 01:31:00.363254 containerd[1567]: time="2025-08-13T01:31:00.363181280Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:31:00.860030 containerd[1567]: time="2025-08-13T01:31:00.859783535Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:00.980190 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:00.981517 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.894 [INFO][13555] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 19939 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.895 [INFO][13555] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.933 [INFO][13569] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" HandleID="k8s-pod-network.373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.933 [INFO][13569] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" HandleID="k8s-pod-network.373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:00.933143031 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.933 [INFO][13569] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.933 [INFO][13569] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.933 [INFO][13569] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.940 [INFO][13569] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" host="172-234-25-215" Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.945 [INFO][13569] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.955 [INFO][13569] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.956 [INFO][13569] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.959 [INFO][13569] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.959 [INFO][13569] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" host="172-234-25-215" Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.960 [INFO][13569] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6 Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.964 [INFO][13569] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" host="172-234-25-215" Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.971 [INFO][13569] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.8/26] block=192.168.62.0/26 handle="k8s-pod-network.373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" host="172-234-25-215" Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.001062 containerd[1567]: 2025-08-13 01:31:00.971 [INFO][13569] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.8/26] handle="k8s-pod-network.373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" host="172-234-25-215" Aug 13 01:31:01.001062 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.003673 containerd[1567]: 2025-08-13 01:31:00.971 [INFO][13569] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:01.003673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.003673 containerd[1567]: 2025-08-13 01:31:00.971 [INFO][13569] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.8/26] IPv6=[] ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" HandleID="k8s-pod-network.373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:01.003673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.003673 containerd[1567]: 2025-08-13 01:31:00.974 [INFO][13555] cni-plugin/k8s.go 418: Populated endpoint ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19939", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"1f729ca061af40eb947f57c4170773d07967cb6538fc3e775e1c140ae443afb3", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.8/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:01.003673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.003673 containerd[1567]: 2025-08-13 01:31:00.974 [INFO][13555] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.8/32] ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:01.003673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.003673 containerd[1567]: 2025-08-13 01:31:00.975 [INFO][13555] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:01.003673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.003673 containerd[1567]: 2025-08-13 01:31:00.982 [INFO][13555] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:01.003673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.003673 containerd[1567]: 2025-08-13 01:31:00.982 [INFO][13555] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"19939", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.8/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"5e:75:83:0e:4f:fe", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:01.003673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.005111 containerd[1567]: 2025-08-13 01:31:00.992 [INFO][13555] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:01.005111 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.069031 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:01.069039 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:01.147536 containerd[1567]: 2025-08-13 01:31:01.062 [INFO][13593] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Aug 13 01:31:01.147536 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.147536 containerd[1567]: 2025-08-13 01:31:01.065 [INFO][13593] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" iface="eth0" netns="/var/run/netns/cni-97971ffd-7dff-4ffc-5ff9-3af6b1e39e0c" Aug 13 01:31:01.147536 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.147536 containerd[1567]: 2025-08-13 01:31:01.066 [INFO][13593] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" iface="eth0" netns="/var/run/netns/cni-97971ffd-7dff-4ffc-5ff9-3af6b1e39e0c" Aug 13 01:31:01.147536 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.147536 containerd[1567]: 2025-08-13 01:31:01.076 [INFO][13593] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" after=10.244001ms iface="eth0" netns="/var/run/netns/cni-97971ffd-7dff-4ffc-5ff9-3af6b1e39e0c" Aug 13 01:31:01.147536 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.147536 containerd[1567]: 2025-08-13 01:31:01.076 [INFO][13593] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Aug 13 01:31:01.147536 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.147536 containerd[1567]: 2025-08-13 01:31:01.076 [INFO][13593] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Aug 13 01:31:01.147536 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.147536 containerd[1567]: 2025-08-13 01:31:01.109 [INFO][13604] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" HandleID="k8s-pod-network.373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:01.147536 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.147536 containerd[1567]: 2025-08-13 01:31:01.109 [INFO][13604] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:01.147536 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.147536 containerd[1567]: 2025-08-13 01:31:01.109 [INFO][13604] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:01.147536 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.147536 containerd[1567]: 2025-08-13 01:31:01.141 [INFO][13604] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" HandleID="k8s-pod-network.373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:01.147536 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.147536 containerd[1567]: 2025-08-13 01:31:01.141 [INFO][13604] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" HandleID="k8s-pod-network.373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:01.147536 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.147536 containerd[1567]: 2025-08-13 01:31:01.142 [INFO][13604] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:01.147536 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.147536 containerd[1567]: 2025-08-13 01:31:01.144 [INFO][13593] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6" Aug 13 01:31:01.147536 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.152157 containerd[1567]: time="2025-08-13T01:31:01.151983517Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6-lo: no space left on device" Aug 13 01:31:01.153120 kubelet[2720]: E0813 01:31:01.152948 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6-lo: no space left on device" Aug 13 01:31:01.153120 kubelet[2720]: E0813 01:31:01.153028 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:01.153120 kubelet[2720]: E0813 01:31:01.153051 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:01.153508 kubelet[2720]: E0813 01:31:01.153122 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:01.158013 systemd[1]: run-netns-cni\x2d97971ffd\x2d7dff\x2d4ffc\x2d5ff9\x2d3af6b1e39e0c.mount: Deactivated successfully. Aug 13 01:31:01.662634 systemd[1]: Started sshd@47-172.234.25.215:22-147.75.109.163:55190.service - OpenSSH per-connection server daemon (147.75.109.163:55190). Aug 13 01:31:01.861497 containerd[1567]: time="2025-08-13T01:31:01.861438090Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:01.977231 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:01.978840 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.904 [INFO][13616] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20007 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.906 [INFO][13616] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.934 [INFO][13629] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" HandleID="k8s-pod-network.beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.934 [INFO][13629] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" HandleID="k8s-pod-network.beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:01.934232196 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.934 [INFO][13629] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.934 [INFO][13629] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.934 [INFO][13629] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.942 [INFO][13629] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" host="172-234-25-215" Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.948 [INFO][13629] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.955 [INFO][13629] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.957 [INFO][13629] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.959 [INFO][13629] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.959 [INFO][13629] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" host="172-234-25-215" Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.961 [INFO][13629] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449 Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.964 [INFO][13629] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" host="172-234-25-215" Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.970 [INFO][13629] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.9/26] block=192.168.62.0/26 handle="k8s-pod-network.beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" host="172-234-25-215" Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:01.999891 containerd[1567]: 2025-08-13 01:31:01.971 [INFO][13629] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.9/26] handle="k8s-pod-network.beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" host="172-234-25-215" Aug 13 01:31:01.999891 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.000640 containerd[1567]: 2025-08-13 01:31:01.971 [INFO][13629] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:02.000640 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.000640 containerd[1567]: 2025-08-13 01:31:01.971 [INFO][13629] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.9/26] IPv6=[] ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" HandleID="k8s-pod-network.beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:02.000640 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.000640 containerd[1567]: 2025-08-13 01:31:01.973 [INFO][13616] cni-plugin/k8s.go 418: Populated endpoint ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20007", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"373ae129e6184a14eb55307ab2e360abdb38f3f63241ff9619182f105d355ec6", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.9/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:02.000640 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.000640 containerd[1567]: 2025-08-13 01:31:01.974 [INFO][13616] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.9/32] ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:02.000640 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.000640 containerd[1567]: 2025-08-13 01:31:01.974 [INFO][13616] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:02.000640 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.000640 containerd[1567]: 2025-08-13 01:31:01.977 [INFO][13616] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:02.000640 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.000640 containerd[1567]: 2025-08-13 01:31:01.979 [INFO][13616] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20007", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.9/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ae:96:41:f7:6f:c6", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:02.000640 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.003355 containerd[1567]: 2025-08-13 01:31:01.989 [INFO][13616] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:02.003355 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.007394 sshd[13613]: Accepted publickey for core from 147.75.109.163 port 55190 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:31:02.013464 sshd-session[13613]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:31:02.023915 systemd-logind[1543]: New session 48 of user core. Aug 13 01:31:02.030810 systemd[1]: Started session-48.scope - Session 48 of User core. Aug 13 01:31:02.124161 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:02.124200 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:02.232015 containerd[1567]: 2025-08-13 01:31:02.121 [INFO][13651] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Aug 13 01:31:02.232015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.232015 containerd[1567]: 2025-08-13 01:31:02.122 [INFO][13651] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" iface="eth0" netns="/var/run/netns/cni-35cf850b-43e1-6c76-32f9-6be41d2a5690" Aug 13 01:31:02.232015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.232015 containerd[1567]: 2025-08-13 01:31:02.123 [INFO][13651] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" iface="eth0" netns="/var/run/netns/cni-35cf850b-43e1-6c76-32f9-6be41d2a5690" Aug 13 01:31:02.232015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.232015 containerd[1567]: 2025-08-13 01:31:02.131 [INFO][13651] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" after=8.234744ms iface="eth0" netns="/var/run/netns/cni-35cf850b-43e1-6c76-32f9-6be41d2a5690" Aug 13 01:31:02.232015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.232015 containerd[1567]: 2025-08-13 01:31:02.131 [INFO][13651] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Aug 13 01:31:02.232015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.232015 containerd[1567]: 2025-08-13 01:31:02.131 [INFO][13651] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Aug 13 01:31:02.232015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.232015 containerd[1567]: 2025-08-13 01:31:02.170 [INFO][13663] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" HandleID="k8s-pod-network.beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:02.232015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.232015 containerd[1567]: 2025-08-13 01:31:02.171 [INFO][13663] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:02.232015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.232015 containerd[1567]: 2025-08-13 01:31:02.171 [INFO][13663] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:02.232015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.232015 containerd[1567]: 2025-08-13 01:31:02.221 [INFO][13663] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" HandleID="k8s-pod-network.beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:02.232015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.232015 containerd[1567]: 2025-08-13 01:31:02.221 [INFO][13663] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" HandleID="k8s-pod-network.beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:02.232015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.232015 containerd[1567]: 2025-08-13 01:31:02.223 [INFO][13663] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:02.232015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.232015 containerd[1567]: 2025-08-13 01:31:02.227 [INFO][13651] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449" Aug 13 01:31:02.232015 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:02.233637 containerd[1567]: time="2025-08-13T01:31:02.233599984Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449-lo: no space left on device" Aug 13 01:31:02.236020 kubelet[2720]: E0813 01:31:02.235979 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449-lo: no space left on device" Aug 13 01:31:02.236323 kubelet[2720]: E0813 01:31:02.236032 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:02.236323 kubelet[2720]: E0813 01:31:02.236052 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:02.236323 kubelet[2720]: E0813 01:31:02.236091 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:02.237628 systemd[1]: run-netns-cni\x2d35cf850b\x2d43e1\x2d6c76\x2d32f9\x2d6be41d2a5690.mount: Deactivated successfully. Aug 13 01:31:02.338069 sshd[13655]: Connection closed by 147.75.109.163 port 55190 Aug 13 01:31:02.338974 sshd-session[13613]: pam_unix(sshd:session): session closed for user core Aug 13 01:31:02.344481 systemd-logind[1543]: Session 48 logged out. Waiting for processes to exit. Aug 13 01:31:02.345007 systemd[1]: sshd@47-172.234.25.215:22-147.75.109.163:55190.service: Deactivated successfully. Aug 13 01:31:02.348451 systemd[1]: session-48.scope: Deactivated successfully. Aug 13 01:31:02.351693 systemd-logind[1543]: Removed session 48. Aug 13 01:31:02.862605 containerd[1567]: time="2025-08-13T01:31:02.862563317Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:02.987377 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:02.988293 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.904 [INFO][13684] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20076 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.905 [INFO][13684] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.941 [INFO][13697] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" HandleID="k8s-pod-network.7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.941 [INFO][13697] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" HandleID="k8s-pod-network.7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024ef50), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:02.941586971 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.941 [INFO][13697] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.942 [INFO][13697] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.942 [INFO][13697] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.949 [INFO][13697] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" host="172-234-25-215" Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.955 [INFO][13697] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.961 [INFO][13697] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.963 [INFO][13697] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.967 [INFO][13697] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.968 [INFO][13697] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" host="172-234-25-215" Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.970 [INFO][13697] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0 Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.974 [INFO][13697] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" host="172-234-25-215" Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.980 [INFO][13697] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.10/26] block=192.168.62.0/26 handle="k8s-pod-network.7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" host="172-234-25-215" Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.003168 containerd[1567]: 2025-08-13 01:31:02.980 [INFO][13697] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.10/26] handle="k8s-pod-network.7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" host="172-234-25-215" Aug 13 01:31:03.003168 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.004510 containerd[1567]: 2025-08-13 01:31:02.980 [INFO][13697] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:03.004510 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.004510 containerd[1567]: 2025-08-13 01:31:02.980 [INFO][13697] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.10/26] IPv6=[] ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" HandleID="k8s-pod-network.7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:03.004510 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.004510 containerd[1567]: 2025-08-13 01:31:02.983 [INFO][13684] cni-plugin/k8s.go 418: Populated endpoint ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20076", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"beef3a944479d0c18ef3f7a4bea79f0d1543a85db749e8c884fa94d92c685449", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.10/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:03.004510 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.004510 containerd[1567]: 2025-08-13 01:31:02.983 [INFO][13684] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.10/32] ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:03.004510 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.004510 containerd[1567]: 2025-08-13 01:31:02.983 [INFO][13684] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:03.004510 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.004510 containerd[1567]: 2025-08-13 01:31:02.989 [INFO][13684] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:03.004510 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.004510 containerd[1567]: 2025-08-13 01:31:02.989 [INFO][13684] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20076", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.10/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"5e:07:cc:ce:7f:96", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:03.004510 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.006615 containerd[1567]: 2025-08-13 01:31:03.000 [INFO][13684] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:03.006615 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.072759 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:03.072767 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:03.175692 containerd[1567]: 2025-08-13 01:31:03.070 [INFO][13718] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Aug 13 01:31:03.175692 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.175692 containerd[1567]: 2025-08-13 01:31:03.071 [INFO][13718] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" iface="eth0" netns="/var/run/netns/cni-d0fb1325-4859-af51-0161-e651c6d31c60" Aug 13 01:31:03.175692 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.175692 containerd[1567]: 2025-08-13 01:31:03.071 [INFO][13718] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" iface="eth0" netns="/var/run/netns/cni-d0fb1325-4859-af51-0161-e651c6d31c60" Aug 13 01:31:03.175692 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.175692 containerd[1567]: 2025-08-13 01:31:03.078 [INFO][13718] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" after=6.957907ms iface="eth0" netns="/var/run/netns/cni-d0fb1325-4859-af51-0161-e651c6d31c60" Aug 13 01:31:03.175692 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.175692 containerd[1567]: 2025-08-13 01:31:03.078 [INFO][13718] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Aug 13 01:31:03.175692 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.175692 containerd[1567]: 2025-08-13 01:31:03.078 [INFO][13718] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Aug 13 01:31:03.175692 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.175692 containerd[1567]: 2025-08-13 01:31:03.131 [INFO][13729] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" HandleID="k8s-pod-network.7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:03.175692 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.175692 containerd[1567]: 2025-08-13 01:31:03.132 [INFO][13729] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:03.175692 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.175692 containerd[1567]: 2025-08-13 01:31:03.132 [INFO][13729] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:03.175692 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.175692 containerd[1567]: 2025-08-13 01:31:03.168 [INFO][13729] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" HandleID="k8s-pod-network.7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:03.175692 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.175692 containerd[1567]: 2025-08-13 01:31:03.169 [INFO][13729] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" HandleID="k8s-pod-network.7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:03.175692 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.175692 containerd[1567]: 2025-08-13 01:31:03.170 [INFO][13729] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:03.175692 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.175692 containerd[1567]: 2025-08-13 01:31:03.173 [INFO][13718] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0" Aug 13 01:31:03.175692 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:03.179814 containerd[1567]: time="2025-08-13T01:31:03.179287813Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0-lo: no space left on device" Aug 13 01:31:03.181203 kubelet[2720]: E0813 01:31:03.181154 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0-lo: no space left on device" Aug 13 01:31:03.181271 kubelet[2720]: E0813 01:31:03.181209 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:03.181271 kubelet[2720]: E0813 01:31:03.181231 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:03.181502 systemd[1]: run-netns-cni\x2dd0fb1325\x2d4859\x2daf51\x2d0161\x2de651c6d31c60.mount: Deactivated successfully. Aug 13 01:31:03.182554 kubelet[2720]: E0813 01:31:03.181762 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:03.363672 containerd[1567]: time="2025-08-13T01:31:03.363316624Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:31:03.865014 containerd[1567]: time="2025-08-13T01:31:03.864757230Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:03.979155 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:03.979414 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.912 [INFO][13740] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20143 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.913 [INFO][13740] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.937 [INFO][13752] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" HandleID="k8s-pod-network.0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.937 [INFO][13752] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" HandleID="k8s-pod-network.0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f020), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:03.937049787 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.937 [INFO][13752] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.937 [INFO][13752] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.937 [INFO][13752] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.944 [INFO][13752] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" host="172-234-25-215" Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.950 [INFO][13752] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.955 [INFO][13752] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.957 [INFO][13752] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.960 [INFO][13752] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.960 [INFO][13752] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" host="172-234-25-215" Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.962 [INFO][13752] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3 Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.966 [INFO][13752] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" host="172-234-25-215" Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.971 [INFO][13752] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.11/26] block=192.168.62.0/26 handle="k8s-pod-network.0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" host="172-234-25-215" Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.000455 containerd[1567]: 2025-08-13 01:31:03.971 [INFO][13752] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.11/26] handle="k8s-pod-network.0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" host="172-234-25-215" Aug 13 01:31:04.000455 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.001709 containerd[1567]: 2025-08-13 01:31:03.971 [INFO][13752] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:04.001709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.001709 containerd[1567]: 2025-08-13 01:31:03.971 [INFO][13752] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.11/26] IPv6=[] ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" HandleID="k8s-pod-network.0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:04.001709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.001709 containerd[1567]: 2025-08-13 01:31:03.975 [INFO][13740] cni-plugin/k8s.go 418: Populated endpoint ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20143", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"7eff589a31b05a0ae65c9e8815c8ec112f9412ab734d729cbd335bc7a53009f0", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.11/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:04.001709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.001709 containerd[1567]: 2025-08-13 01:31:03.975 [INFO][13740] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.11/32] ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:04.001709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.001709 containerd[1567]: 2025-08-13 01:31:03.975 [INFO][13740] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:04.001709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.001709 containerd[1567]: 2025-08-13 01:31:03.979 [INFO][13740] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:04.001709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.001709 containerd[1567]: 2025-08-13 01:31:03.981 [INFO][13740] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20143", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.11/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"f6:14:5c:3e:60:a6", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:04.001709 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.005256 containerd[1567]: 2025-08-13 01:31:03.994 [INFO][13740] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:04.005256 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.081018 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:04.081134 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:04.181809 containerd[1567]: 2025-08-13 01:31:04.075 [INFO][13775] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Aug 13 01:31:04.181809 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.181809 containerd[1567]: 2025-08-13 01:31:04.078 [INFO][13775] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" iface="eth0" netns="/var/run/netns/cni-42baf35e-75cd-71a4-91de-f76a64507f4d" Aug 13 01:31:04.181809 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.181809 containerd[1567]: 2025-08-13 01:31:04.079 [INFO][13775] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" iface="eth0" netns="/var/run/netns/cni-42baf35e-75cd-71a4-91de-f76a64507f4d" Aug 13 01:31:04.181809 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.181809 containerd[1567]: 2025-08-13 01:31:04.085 [INFO][13775] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" after=6.071948ms iface="eth0" netns="/var/run/netns/cni-42baf35e-75cd-71a4-91de-f76a64507f4d" Aug 13 01:31:04.181809 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.181809 containerd[1567]: 2025-08-13 01:31:04.085 [INFO][13775] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Aug 13 01:31:04.181809 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.181809 containerd[1567]: 2025-08-13 01:31:04.085 [INFO][13775] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Aug 13 01:31:04.181809 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.181809 containerd[1567]: 2025-08-13 01:31:04.129 [INFO][13786] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" HandleID="k8s-pod-network.0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:04.181809 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.181809 containerd[1567]: 2025-08-13 01:31:04.129 [INFO][13786] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:04.181809 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.181809 containerd[1567]: 2025-08-13 01:31:04.129 [INFO][13786] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:04.181809 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.181809 containerd[1567]: 2025-08-13 01:31:04.175 [INFO][13786] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" HandleID="k8s-pod-network.0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:04.181809 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.181809 containerd[1567]: 2025-08-13 01:31:04.175 [INFO][13786] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" HandleID="k8s-pod-network.0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:04.181809 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.181809 containerd[1567]: 2025-08-13 01:31:04.176 [INFO][13786] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:04.181809 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.181809 containerd[1567]: 2025-08-13 01:31:04.178 [INFO][13775] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3" Aug 13 01:31:04.181809 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:04.184040 containerd[1567]: time="2025-08-13T01:31:04.183948073Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3-lo: no space left on device" Aug 13 01:31:04.184507 kubelet[2720]: E0813 01:31:04.184332 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3-lo: no space left on device" Aug 13 01:31:04.184507 kubelet[2720]: E0813 01:31:04.184403 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:04.184507 kubelet[2720]: E0813 01:31:04.184425 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:04.185304 kubelet[2720]: E0813 01:31:04.184581 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:04.186965 systemd[1]: run-netns-cni\x2d42baf35e\x2d75cd\x2d71a4\x2d91de\x2df76a64507f4d.mount: Deactivated successfully. Aug 13 01:31:04.866630 containerd[1567]: time="2025-08-13T01:31:04.866564756Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:04.979787 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:04.980869 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.908 [INFO][13796] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20212 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.909 [INFO][13796] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.931 [INFO][13809] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" HandleID="k8s-pod-network.70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.931 [INFO][13809] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" HandleID="k8s-pod-network.70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:04.931618226 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.931 [INFO][13809] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.931 [INFO][13809] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.932 [INFO][13809] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.938 [INFO][13809] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" host="172-234-25-215" Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.947 [INFO][13809] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.956 [INFO][13809] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.958 [INFO][13809] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.961 [INFO][13809] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.961 [INFO][13809] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" host="172-234-25-215" Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.964 [INFO][13809] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.968 [INFO][13809] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" host="172-234-25-215" Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.973 [INFO][13809] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.12/26] block=192.168.62.0/26 handle="k8s-pod-network.70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" host="172-234-25-215" Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.008479 containerd[1567]: 2025-08-13 01:31:04.974 [INFO][13809] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.12/26] handle="k8s-pod-network.70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" host="172-234-25-215" Aug 13 01:31:05.008479 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.009644 containerd[1567]: 2025-08-13 01:31:04.974 [INFO][13809] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:05.009644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.009644 containerd[1567]: 2025-08-13 01:31:04.974 [INFO][13809] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.12/26] IPv6=[] ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" HandleID="k8s-pod-network.70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:05.009644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.009644 containerd[1567]: 2025-08-13 01:31:04.976 [INFO][13796] cni-plugin/k8s.go 418: Populated endpoint ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20212", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0114382ca6e7f225b0fb693e8a90463aff3bf26be80261403136af8dedb6c7e3", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.12/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:05.009644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.009644 containerd[1567]: 2025-08-13 01:31:04.976 [INFO][13796] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.12/32] ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:05.009644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.009644 containerd[1567]: 2025-08-13 01:31:04.977 [INFO][13796] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:05.009644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.009644 containerd[1567]: 2025-08-13 01:31:04.981 [INFO][13796] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:05.009644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.009644 containerd[1567]: 2025-08-13 01:31:04.983 [INFO][13796] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20212", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.12/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"96:75:75:d4:c6:7b", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:05.009644 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.009984 containerd[1567]: 2025-08-13 01:31:04.996 [INFO][13796] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:05.009984 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.081639 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:05.082963 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:05.172453 containerd[1567]: 2025-08-13 01:31:05.079 [INFO][13833] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Aug 13 01:31:05.172453 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.172453 containerd[1567]: 2025-08-13 01:31:05.080 [INFO][13833] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" iface="eth0" netns="/var/run/netns/cni-1d15bb2b-bb2b-e65e-5ab7-674cddee4986" Aug 13 01:31:05.172453 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.172453 containerd[1567]: 2025-08-13 01:31:05.080 [INFO][13833] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" iface="eth0" netns="/var/run/netns/cni-1d15bb2b-bb2b-e65e-5ab7-674cddee4986" Aug 13 01:31:05.172453 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.172453 containerd[1567]: 2025-08-13 01:31:05.089 [INFO][13833] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" after=8.762644ms iface="eth0" netns="/var/run/netns/cni-1d15bb2b-bb2b-e65e-5ab7-674cddee4986" Aug 13 01:31:05.172453 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.172453 containerd[1567]: 2025-08-13 01:31:05.089 [INFO][13833] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Aug 13 01:31:05.172453 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.172453 containerd[1567]: 2025-08-13 01:31:05.089 [INFO][13833] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Aug 13 01:31:05.172453 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.172453 containerd[1567]: 2025-08-13 01:31:05.132 [INFO][13842] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" HandleID="k8s-pod-network.70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:05.172453 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.172453 containerd[1567]: 2025-08-13 01:31:05.132 [INFO][13842] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:05.172453 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.172453 containerd[1567]: 2025-08-13 01:31:05.132 [INFO][13842] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:05.172453 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.172453 containerd[1567]: 2025-08-13 01:31:05.165 [INFO][13842] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" HandleID="k8s-pod-network.70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:05.172453 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.172453 containerd[1567]: 2025-08-13 01:31:05.166 [INFO][13842] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" HandleID="k8s-pod-network.70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:05.172453 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.172453 containerd[1567]: 2025-08-13 01:31:05.167 [INFO][13842] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:05.172453 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.172453 containerd[1567]: 2025-08-13 01:31:05.169 [INFO][13833] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de" Aug 13 01:31:05.172453 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:05.175617 containerd[1567]: time="2025-08-13T01:31:05.174535109Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de-lo: no space left on device" Aug 13 01:31:05.176006 kubelet[2720]: E0813 01:31:05.175961 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de-lo: no space left on device" Aug 13 01:31:05.176121 kubelet[2720]: E0813 01:31:05.176102 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:05.176184 kubelet[2720]: E0813 01:31:05.176169 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:05.176471 kubelet[2720]: E0813 01:31:05.176264 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:05.176895 systemd[1]: run-netns-cni\x2d1d15bb2b\x2dbb2b\x2de65e\x2d5ab7\x2d674cddee4986.mount: Deactivated successfully. Aug 13 01:31:05.868415 containerd[1567]: time="2025-08-13T01:31:05.868114141Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:05.983200 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:05.987460 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.914 [INFO][13851] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20289 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.914 [INFO][13851] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.938 [INFO][13863] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" HandleID="k8s-pod-network.e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.938 [INFO][13863] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" HandleID="k8s-pod-network.e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f2b0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:05.938463752 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.939 [INFO][13863] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.939 [INFO][13863] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.940 [INFO][13863] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.948 [INFO][13863] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" host="172-234-25-215" Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.953 [INFO][13863] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.960 [INFO][13863] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.962 [INFO][13863] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.965 [INFO][13863] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.965 [INFO][13863] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" host="172-234-25-215" Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.967 [INFO][13863] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.970 [INFO][13863] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" host="172-234-25-215" Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.976 [INFO][13863] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.13/26] block=192.168.62.0/26 handle="k8s-pod-network.e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" host="172-234-25-215" Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.012144 containerd[1567]: 2025-08-13 01:31:05.976 [INFO][13863] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.13/26] handle="k8s-pod-network.e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" host="172-234-25-215" Aug 13 01:31:06.012144 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.016155 containerd[1567]: 2025-08-13 01:31:05.976 [INFO][13863] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:06.016155 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.016155 containerd[1567]: 2025-08-13 01:31:05.976 [INFO][13863] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.13/26] IPv6=[] ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" HandleID="k8s-pod-network.e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:06.016155 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.016155 containerd[1567]: 2025-08-13 01:31:05.979 [INFO][13851] cni-plugin/k8s.go 418: Populated endpoint ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20289", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"70952a193726170ab2e386b6794b18eb39b563ef236822c5ba0879bf5a0248de", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.13/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:06.016155 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.016155 containerd[1567]: 2025-08-13 01:31:05.980 [INFO][13851] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.13/32] ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:06.016155 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.016155 containerd[1567]: 2025-08-13 01:31:05.980 [INFO][13851] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:06.016155 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.016155 containerd[1567]: 2025-08-13 01:31:05.983 [INFO][13851] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:06.016155 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.016155 containerd[1567]: 2025-08-13 01:31:05.984 [INFO][13851] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20289", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.13/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"8a:ff:08:f3:a2:b1", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:06.016155 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.016464 containerd[1567]: 2025-08-13 01:31:06.005 [INFO][13851] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:06.016464 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.096192 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:06.096360 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:06.196153 containerd[1567]: 2025-08-13 01:31:06.091 [INFO][13887] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Aug 13 01:31:06.196153 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.196153 containerd[1567]: 2025-08-13 01:31:06.092 [INFO][13887] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" iface="eth0" netns="/var/run/netns/cni-688f4a16-0810-b583-90d5-cef90af9237a" Aug 13 01:31:06.196153 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.196153 containerd[1567]: 2025-08-13 01:31:06.092 [INFO][13887] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" iface="eth0" netns="/var/run/netns/cni-688f4a16-0810-b583-90d5-cef90af9237a" Aug 13 01:31:06.196153 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.196153 containerd[1567]: 2025-08-13 01:31:06.102 [INFO][13887] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" after=10.014711ms iface="eth0" netns="/var/run/netns/cni-688f4a16-0810-b583-90d5-cef90af9237a" Aug 13 01:31:06.196153 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.196153 containerd[1567]: 2025-08-13 01:31:06.103 [INFO][13887] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Aug 13 01:31:06.196153 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.196153 containerd[1567]: 2025-08-13 01:31:06.105 [INFO][13887] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Aug 13 01:31:06.196153 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.196153 containerd[1567]: 2025-08-13 01:31:06.155 [INFO][13898] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" HandleID="k8s-pod-network.e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:06.196153 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.196153 containerd[1567]: 2025-08-13 01:31:06.155 [INFO][13898] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:06.196153 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.196153 containerd[1567]: 2025-08-13 01:31:06.155 [INFO][13898] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:06.196153 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.196153 containerd[1567]: 2025-08-13 01:31:06.188 [INFO][13898] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" HandleID="k8s-pod-network.e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:06.196153 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.196153 containerd[1567]: 2025-08-13 01:31:06.188 [INFO][13898] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" HandleID="k8s-pod-network.e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:06.196153 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.196153 containerd[1567]: 2025-08-13 01:31:06.189 [INFO][13898] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:06.196153 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.196153 containerd[1567]: 2025-08-13 01:31:06.193 [INFO][13887] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e" Aug 13 01:31:06.196153 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:06.201286 containerd[1567]: time="2025-08-13T01:31:06.201161518Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e-lo: no space left on device" Aug 13 01:31:06.202124 kubelet[2720]: E0813 01:31:06.201889 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e-lo: no space left on device" Aug 13 01:31:06.202408 kubelet[2720]: E0813 01:31:06.202141 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:06.202408 kubelet[2720]: E0813 01:31:06.202161 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:06.202408 kubelet[2720]: E0813 01:31:06.202205 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:06.203617 systemd[1]: run-netns-cni\x2d688f4a16\x2d0810\x2db583\x2d90d5\x2dcef90af9237a.mount: Deactivated successfully. Aug 13 01:31:06.363448 containerd[1567]: time="2025-08-13T01:31:06.363394839Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:31:06.870515 containerd[1567]: time="2025-08-13T01:31:06.870459415Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:06.997395 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:07.002575 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.912 [INFO][13907] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20352 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.912 [INFO][13907] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.937 [INFO][13920] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" HandleID="k8s-pod-network.5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.938 [INFO][13920] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" HandleID="k8s-pod-network.5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cd740), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:06.937972081 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.938 [INFO][13920] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.938 [INFO][13920] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.938 [INFO][13920] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.944 [INFO][13920] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" host="172-234-25-215" Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.954 [INFO][13920] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.965 [INFO][13920] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.968 [INFO][13920] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.972 [INFO][13920] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.972 [INFO][13920] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" host="172-234-25-215" Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.974 [INFO][13920] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.977 [INFO][13920] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" host="172-234-25-215" Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.984 [INFO][13920] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.14/26] block=192.168.62.0/26 handle="k8s-pod-network.5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" host="172-234-25-215" Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.025448 containerd[1567]: 2025-08-13 01:31:06.987 [INFO][13920] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.14/26] handle="k8s-pod-network.5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" host="172-234-25-215" Aug 13 01:31:07.025448 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.029121 containerd[1567]: 2025-08-13 01:31:06.987 [INFO][13920] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:07.029121 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.029121 containerd[1567]: 2025-08-13 01:31:06.987 [INFO][13920] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.14/26] IPv6=[] ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" HandleID="k8s-pod-network.5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:07.029121 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.029121 containerd[1567]: 2025-08-13 01:31:06.990 [INFO][13907] cni-plugin/k8s.go 418: Populated endpoint ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20352", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e2f98d85e5f50696d4eebc28e1f85941d269d523a465c469cc40e0e73fac916e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.14/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:07.029121 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.029121 containerd[1567]: 2025-08-13 01:31:06.990 [INFO][13907] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.14/32] ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:07.029121 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.029121 containerd[1567]: 2025-08-13 01:31:06.990 [INFO][13907] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:07.029121 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.029121 containerd[1567]: 2025-08-13 01:31:07.002 [INFO][13907] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:07.029121 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.029121 containerd[1567]: 2025-08-13 01:31:07.006 [INFO][13907] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20352", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.14/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ae:36:e2:b8:60:1a", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:07.029121 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.029476 containerd[1567]: 2025-08-13 01:31:07.018 [INFO][13907] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:07.029476 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.092136 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:07.092146 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:07.183258 containerd[1567]: 2025-08-13 01:31:07.090 [INFO][13945] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Aug 13 01:31:07.183258 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.183258 containerd[1567]: 2025-08-13 01:31:07.090 [INFO][13945] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" iface="eth0" netns="/var/run/netns/cni-433ae519-a9c3-7f9c-0129-b63fef3693e3" Aug 13 01:31:07.183258 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.183258 containerd[1567]: 2025-08-13 01:31:07.091 [INFO][13945] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" iface="eth0" netns="/var/run/netns/cni-433ae519-a9c3-7f9c-0129-b63fef3693e3" Aug 13 01:31:07.183258 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.183258 containerd[1567]: 2025-08-13 01:31:07.099 [INFO][13945] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" after=9.216173ms iface="eth0" netns="/var/run/netns/cni-433ae519-a9c3-7f9c-0129-b63fef3693e3" Aug 13 01:31:07.183258 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.183258 containerd[1567]: 2025-08-13 01:31:07.100 [INFO][13945] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Aug 13 01:31:07.183258 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.183258 containerd[1567]: 2025-08-13 01:31:07.100 [INFO][13945] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Aug 13 01:31:07.183258 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.183258 containerd[1567]: 2025-08-13 01:31:07.130 [INFO][13953] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" HandleID="k8s-pod-network.5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:07.183258 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.183258 containerd[1567]: 2025-08-13 01:31:07.131 [INFO][13953] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:07.183258 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.183258 containerd[1567]: 2025-08-13 01:31:07.131 [INFO][13953] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:07.183258 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.183258 containerd[1567]: 2025-08-13 01:31:07.174 [INFO][13953] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" HandleID="k8s-pod-network.5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:07.183258 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.183258 containerd[1567]: 2025-08-13 01:31:07.174 [INFO][13953] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" HandleID="k8s-pod-network.5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:07.183258 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.183258 containerd[1567]: 2025-08-13 01:31:07.175 [INFO][13953] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:07.183258 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.183258 containerd[1567]: 2025-08-13 01:31:07.179 [INFO][13945] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb" Aug 13 01:31:07.183258 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:07.186354 containerd[1567]: time="2025-08-13T01:31:07.186259184Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb-lo: no space left on device" Aug 13 01:31:07.187145 kubelet[2720]: E0813 01:31:07.186787 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb-lo: no space left on device" Aug 13 01:31:07.187145 kubelet[2720]: E0813 01:31:07.186883 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:07.187323 kubelet[2720]: E0813 01:31:07.187305 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:07.187673 kubelet[2720]: E0813 01:31:07.187458 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:07.189049 systemd[1]: run-netns-cni\x2d433ae519\x2da9c3\x2d7f9c\x2d0129\x2db63fef3693e3.mount: Deactivated successfully. Aug 13 01:31:07.399017 systemd[1]: Started sshd@48-172.234.25.215:22-147.75.109.163:55206.service - OpenSSH per-connection server daemon (147.75.109.163:55206). Aug 13 01:31:07.726957 sshd[13963]: Accepted publickey for core from 147.75.109.163 port 55206 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:31:07.728677 sshd-session[13963]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:31:07.734933 systemd-logind[1543]: New session 49 of user core. Aug 13 01:31:07.738936 systemd[1]: Started session-49.scope - Session 49 of User core. Aug 13 01:31:07.872658 containerd[1567]: time="2025-08-13T01:31:07.872405721Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:08.040026 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:08.042616 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:07.924 [INFO][13967] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20418 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:07.924 [INFO][13967] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:07.977 [INFO][13985] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" HandleID="k8s-pod-network.60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:07.978 [INFO][13985] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" HandleID="k8s-pod-network.60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002ad350), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:07.977933467 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:07.978 [INFO][13985] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:07.978 [INFO][13985] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:07.978 [INFO][13985] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:07.986 [INFO][13985] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" host="172-234-25-215" Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:07.994 [INFO][13985] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:08.010 [INFO][13985] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:08.014 [INFO][13985] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:08.017 [INFO][13985] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:08.017 [INFO][13985] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" host="172-234-25-215" Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:08.019 [INFO][13985] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:08.024 [INFO][13985] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" host="172-234-25-215" Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:08.032 [INFO][13985] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.15/26] block=192.168.62.0/26 handle="k8s-pod-network.60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" host="172-234-25-215" Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.062271 containerd[1567]: 2025-08-13 01:31:08.032 [INFO][13985] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.15/26] handle="k8s-pod-network.60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" host="172-234-25-215" Aug 13 01:31:08.062271 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.065484 containerd[1567]: 2025-08-13 01:31:08.032 [INFO][13985] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:08.065484 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.065484 containerd[1567]: 2025-08-13 01:31:08.032 [INFO][13985] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.15/26] IPv6=[] ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" HandleID="k8s-pod-network.60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:08.065484 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.065484 containerd[1567]: 2025-08-13 01:31:08.035 [INFO][13967] cni-plugin/k8s.go 418: Populated endpoint ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20418", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"5fe15bd219c2d5e3e198258aa5153cc3e9e6537f686f1550be739f3f44c8f6bb", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.15/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:08.065484 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.065484 containerd[1567]: 2025-08-13 01:31:08.035 [INFO][13967] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.15/32] ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:08.065484 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.065484 containerd[1567]: 2025-08-13 01:31:08.035 [INFO][13967] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:08.065484 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.065484 containerd[1567]: 2025-08-13 01:31:08.043 [INFO][13967] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:08.065484 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.065484 containerd[1567]: 2025-08-13 01:31:08.044 [INFO][13967] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20418", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.15/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"66:90:59:60:00:a7", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:08.065484 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.064422 sshd-session[13963]: pam_unix(sshd:session): session closed for user core Aug 13 01:31:08.070220 sshd[13965]: Connection closed by 147.75.109.163 port 55206 Aug 13 01:31:08.071398 containerd[1567]: 2025-08-13 01:31:08.056 [INFO][13967] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:08.071398 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.077124 systemd[1]: sshd@48-172.234.25.215:22-147.75.109.163:55206.service: Deactivated successfully. Aug 13 01:31:08.084137 systemd[1]: session-49.scope: Deactivated successfully. Aug 13 01:31:08.089540 systemd-logind[1543]: Session 49 logged out. Waiting for processes to exit. Aug 13 01:31:08.091576 systemd-logind[1543]: Removed session 49. Aug 13 01:31:08.175873 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:08.175886 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:08.262822 containerd[1567]: 2025-08-13 01:31:08.174 [INFO][14009] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Aug 13 01:31:08.262822 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.262822 containerd[1567]: 2025-08-13 01:31:08.175 [INFO][14009] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" iface="eth0" netns="/var/run/netns/cni-30cdb768-3ddd-4fc0-542d-92815b72fbf8" Aug 13 01:31:08.262822 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.262822 containerd[1567]: 2025-08-13 01:31:08.175 [INFO][14009] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" iface="eth0" netns="/var/run/netns/cni-30cdb768-3ddd-4fc0-542d-92815b72fbf8" Aug 13 01:31:08.262822 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.262822 containerd[1567]: 2025-08-13 01:31:08.183 [INFO][14009] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" after=8.918263ms iface="eth0" netns="/var/run/netns/cni-30cdb768-3ddd-4fc0-542d-92815b72fbf8" Aug 13 01:31:08.262822 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.262822 containerd[1567]: 2025-08-13 01:31:08.184 [INFO][14009] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Aug 13 01:31:08.262822 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.262822 containerd[1567]: 2025-08-13 01:31:08.184 [INFO][14009] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Aug 13 01:31:08.262822 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.262822 containerd[1567]: 2025-08-13 01:31:08.212 [INFO][14021] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" HandleID="k8s-pod-network.60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:08.262822 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.262822 containerd[1567]: 2025-08-13 01:31:08.212 [INFO][14021] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:08.262822 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.262822 containerd[1567]: 2025-08-13 01:31:08.212 [INFO][14021] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:08.262822 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.262822 containerd[1567]: 2025-08-13 01:31:08.254 [INFO][14021] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" HandleID="k8s-pod-network.60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:08.262822 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.262822 containerd[1567]: 2025-08-13 01:31:08.254 [INFO][14021] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" HandleID="k8s-pod-network.60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:08.262822 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.262822 containerd[1567]: 2025-08-13 01:31:08.255 [INFO][14021] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:08.262822 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.262822 containerd[1567]: 2025-08-13 01:31:08.259 [INFO][14009] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e" Aug 13 01:31:08.262822 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:08.270158 containerd[1567]: time="2025-08-13T01:31:08.269124610Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e-lo: no space left on device" Aug 13 01:31:08.268878 systemd[1]: run-netns-cni\x2d30cdb768\x2d3ddd\x2d4fc0\x2d542d\x2d92815b72fbf8.mount: Deactivated successfully. Aug 13 01:31:08.271506 kubelet[2720]: E0813 01:31:08.271023 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e-lo: no space left on device" Aug 13 01:31:08.271506 kubelet[2720]: E0813 01:31:08.271098 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:08.271506 kubelet[2720]: E0813 01:31:08.271119 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:08.271506 kubelet[2720]: E0813 01:31:08.271240 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:08.534218 kubelet[2720]: I0813 01:31:08.534185 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:31:08.534332 kubelet[2720]: I0813 01:31:08.534237 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:31:08.536078 kubelet[2720]: I0813 01:31:08.536049 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:31:08.548403 kubelet[2720]: I0813 01:31:08.548372 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:31:08.548541 kubelet[2720]: I0813 01:31:08.548493 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["calico-system/calico-kube-controllers-b57d47f84-lznkx","kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:31:08.548605 kubelet[2720]: E0813 01:31:08.548548 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:08.548605 kubelet[2720]: E0813 01:31:08.548557 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:31:08.548605 kubelet[2720]: E0813 01:31:08.548563 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:31:08.548605 kubelet[2720]: E0813 01:31:08.548572 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:31:08.548605 kubelet[2720]: E0813 01:31:08.548581 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:31:08.548605 kubelet[2720]: E0813 01:31:08.548604 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:31:08.548760 kubelet[2720]: E0813 01:31:08.548613 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:31:08.548760 kubelet[2720]: E0813 01:31:08.548623 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:31:08.548760 kubelet[2720]: E0813 01:31:08.548632 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:31:08.548760 kubelet[2720]: E0813 01:31:08.548639 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:31:08.548760 kubelet[2720]: I0813 01:31:08.548648 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:31:08.874541 containerd[1567]: time="2025-08-13T01:31:08.874101726Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:08.987463 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:08.989465 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.916 [INFO][14032] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20502 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.917 [INFO][14032] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.940 [INFO][14044] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" HandleID="k8s-pod-network.9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.940 [INFO][14044] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" HandleID="k8s-pod-network.9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cd7f0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:08.940554944 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.940 [INFO][14044] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.940 [INFO][14044] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.940 [INFO][14044] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.947 [INFO][14044] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" host="172-234-25-215" Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.954 [INFO][14044] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.961 [INFO][14044] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.963 [INFO][14044] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.966 [INFO][14044] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.966 [INFO][14044] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" host="172-234-25-215" Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.968 [INFO][14044] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12 Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.971 [INFO][14044] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" host="172-234-25-215" Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.977 [INFO][14044] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.16/26] block=192.168.62.0/26 handle="k8s-pod-network.9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" host="172-234-25-215" Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.013999 containerd[1567]: 2025-08-13 01:31:08.977 [INFO][14044] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.16/26] handle="k8s-pod-network.9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" host="172-234-25-215" Aug 13 01:31:09.013999 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.015789 containerd[1567]: 2025-08-13 01:31:08.977 [INFO][14044] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:09.015789 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.015789 containerd[1567]: 2025-08-13 01:31:08.977 [INFO][14044] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.16/26] IPv6=[] ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" HandleID="k8s-pod-network.9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.015789 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.015789 containerd[1567]: 2025-08-13 01:31:08.981 [INFO][14032] cni-plugin/k8s.go 418: Populated endpoint ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20502", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"60a141378e8e72bf3b408a807ae44f7dae8deea617e361248068ce5f4085712e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.16/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:09.015789 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.015789 containerd[1567]: 2025-08-13 01:31:08.981 [INFO][14032] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.16/32] ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.015789 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.015789 containerd[1567]: 2025-08-13 01:31:08.981 [INFO][14032] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.015789 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.015789 containerd[1567]: 2025-08-13 01:31:08.989 [INFO][14032] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.015789 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.015789 containerd[1567]: 2025-08-13 01:31:08.990 [INFO][14032] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20502", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.16/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"aa:7c:ad:63:93:4f", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:09.015789 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.017768 containerd[1567]: 2025-08-13 01:31:09.006 [INFO][14032] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.017768 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.057560 containerd[1567]: time="2025-08-13T01:31:09.057477239Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:31:09.075117 kubelet[2720]: E0813 01:31:09.075084 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:31:09.076206 kubelet[2720]: E0813 01:31:09.076170 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-299933554-aTh- sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2568257827: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:31:09.084617 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:09.084900 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:09.168238 containerd[1567]: 2025-08-13 01:31:09.081 [INFO][14067] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Aug 13 01:31:09.168238 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.168238 containerd[1567]: 2025-08-13 01:31:09.081 [INFO][14067] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" iface="eth0" netns="/var/run/netns/cni-ebaf0b6e-3045-ddfa-c44e-f4d6018a6ef7" Aug 13 01:31:09.168238 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.168238 containerd[1567]: 2025-08-13 01:31:09.082 [INFO][14067] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" iface="eth0" netns="/var/run/netns/cni-ebaf0b6e-3045-ddfa-c44e-f4d6018a6ef7" Aug 13 01:31:09.168238 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.168238 containerd[1567]: 2025-08-13 01:31:09.090 [INFO][14067] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" after=9.595222ms iface="eth0" netns="/var/run/netns/cni-ebaf0b6e-3045-ddfa-c44e-f4d6018a6ef7" Aug 13 01:31:09.168238 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.168238 containerd[1567]: 2025-08-13 01:31:09.090 [INFO][14067] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Aug 13 01:31:09.168238 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.168238 containerd[1567]: 2025-08-13 01:31:09.090 [INFO][14067] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Aug 13 01:31:09.168238 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.168238 containerd[1567]: 2025-08-13 01:31:09.124 [INFO][14077] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" HandleID="k8s-pod-network.9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.168238 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.168238 containerd[1567]: 2025-08-13 01:31:09.125 [INFO][14077] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:09.168238 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.168238 containerd[1567]: 2025-08-13 01:31:09.125 [INFO][14077] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:09.168238 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.168238 containerd[1567]: 2025-08-13 01:31:09.161 [INFO][14077] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" HandleID="k8s-pod-network.9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.168238 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.168238 containerd[1567]: 2025-08-13 01:31:09.161 [INFO][14077] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" HandleID="k8s-pod-network.9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.168238 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.168238 containerd[1567]: 2025-08-13 01:31:09.163 [INFO][14077] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:09.168238 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.168238 containerd[1567]: 2025-08-13 01:31:09.165 [INFO][14067] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12" Aug 13 01:31:09.168238 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.171332 systemd[1]: run-netns-cni\x2debaf0b6e\x2d3045\x2dddfa\x2dc44e\x2df4d6018a6ef7.mount: Deactivated successfully. Aug 13 01:31:09.173794 containerd[1567]: time="2025-08-13T01:31:09.172467576Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12-lo: no space left on device" Aug 13 01:31:09.173886 kubelet[2720]: E0813 01:31:09.172685 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12-lo: no space left on device" Aug 13 01:31:09.173886 kubelet[2720]: E0813 01:31:09.172747 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:09.173886 kubelet[2720]: E0813 01:31:09.172767 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:09.173886 kubelet[2720]: E0813 01:31:09.172812 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:09.364620 containerd[1567]: time="2025-08-13T01:31:09.364566943Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:31:09.875490 containerd[1567]: time="2025-08-13T01:31:09.875443093Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:09.978370 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:09.979788 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.913 [INFO][14089] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20564 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.913 [INFO][14089] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.937 [INFO][14098] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" HandleID="k8s-pod-network.ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.938 [INFO][14098] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" HandleID="k8s-pod-network.ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f210), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:09.937920418 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.938 [INFO][14098] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.938 [INFO][14098] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.938 [INFO][14098] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.945 [INFO][14098] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" host="172-234-25-215" Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.950 [INFO][14098] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.957 [INFO][14098] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.959 [INFO][14098] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.961 [INFO][14098] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.961 [INFO][14098] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" host="172-234-25-215" Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.962 [INFO][14098] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3 Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.965 [INFO][14098] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" host="172-234-25-215" Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.972 [INFO][14098] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.17/26] block=192.168.62.0/26 handle="k8s-pod-network.ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" host="172-234-25-215" Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.998161 containerd[1567]: 2025-08-13 01:31:09.972 [INFO][14098] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.17/26] handle="k8s-pod-network.ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" host="172-234-25-215" Aug 13 01:31:09.998161 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.999914 containerd[1567]: 2025-08-13 01:31:09.972 [INFO][14098] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:09.999914 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.999914 containerd[1567]: 2025-08-13 01:31:09.972 [INFO][14098] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.17/26] IPv6=[] ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" HandleID="k8s-pod-network.ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.999914 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.999914 containerd[1567]: 2025-08-13 01:31:09.975 [INFO][14089] cni-plugin/k8s.go 418: Populated endpoint ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20564", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"9c2a971f166f4cc3ce9209923a1b05875e4a6f9ce898957a77e4977b0ba5eb12", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.17/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:09.999914 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.999914 containerd[1567]: 2025-08-13 01:31:09.975 [INFO][14089] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.17/32] ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.999914 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.999914 containerd[1567]: 2025-08-13 01:31:09.975 [INFO][14089] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.999914 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.999914 containerd[1567]: 2025-08-13 01:31:09.980 [INFO][14089] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:09.999914 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:09.999914 containerd[1567]: 2025-08-13 01:31:09.980 [INFO][14089] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20564", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.17/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"b6:60:4c:65:30:e1", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:09.999914 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.000248 containerd[1567]: 2025-08-13 01:31:09.989 [INFO][14089] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:10.000248 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.083602 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:10.083767 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:10.184819 containerd[1567]: 2025-08-13 01:31:10.081 [INFO][14120] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Aug 13 01:31:10.184819 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.184819 containerd[1567]: 2025-08-13 01:31:10.082 [INFO][14120] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" iface="eth0" netns="/var/run/netns/cni-db8765e0-94b6-6f41-5e84-f57680597d76" Aug 13 01:31:10.184819 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.184819 containerd[1567]: 2025-08-13 01:31:10.082 [INFO][14120] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" iface="eth0" netns="/var/run/netns/cni-db8765e0-94b6-6f41-5e84-f57680597d76" Aug 13 01:31:10.184819 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.184819 containerd[1567]: 2025-08-13 01:31:10.088 [INFO][14120] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" after=6.511948ms iface="eth0" netns="/var/run/netns/cni-db8765e0-94b6-6f41-5e84-f57680597d76" Aug 13 01:31:10.184819 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.184819 containerd[1567]: 2025-08-13 01:31:10.088 [INFO][14120] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Aug 13 01:31:10.184819 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.184819 containerd[1567]: 2025-08-13 01:31:10.089 [INFO][14120] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Aug 13 01:31:10.184819 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.184819 containerd[1567]: 2025-08-13 01:31:10.128 [INFO][14130] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" HandleID="k8s-pod-network.ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:10.184819 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.184819 containerd[1567]: 2025-08-13 01:31:10.129 [INFO][14130] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:10.184819 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.184819 containerd[1567]: 2025-08-13 01:31:10.129 [INFO][14130] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:10.184819 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.184819 containerd[1567]: 2025-08-13 01:31:10.178 [INFO][14130] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" HandleID="k8s-pod-network.ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:10.184819 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.184819 containerd[1567]: 2025-08-13 01:31:10.178 [INFO][14130] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" HandleID="k8s-pod-network.ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:10.184819 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.184819 containerd[1567]: 2025-08-13 01:31:10.180 [INFO][14130] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:10.184819 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.184819 containerd[1567]: 2025-08-13 01:31:10.181 [INFO][14120] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3" Aug 13 01:31:10.184819 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:10.188412 containerd[1567]: time="2025-08-13T01:31:10.188314206Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3-lo: no space left on device" Aug 13 01:31:10.188550 systemd[1]: run-netns-cni\x2ddb8765e0\x2d94b6\x2d6f41\x2d5e84\x2df57680597d76.mount: Deactivated successfully. Aug 13 01:31:10.190162 kubelet[2720]: E0813 01:31:10.189951 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3-lo: no space left on device" Aug 13 01:31:10.190162 kubelet[2720]: E0813 01:31:10.190029 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:10.191160 kubelet[2720]: E0813 01:31:10.190050 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:10.191160 kubelet[2720]: E0813 01:31:10.190613 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:10.877420 containerd[1567]: time="2025-08-13T01:31:10.877374168Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:10.998582 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:11.007891 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.920 [INFO][14142] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20633 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.920 [INFO][14142] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.944 [INFO][14155] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" HandleID="k8s-pod-network.69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.945 [INFO][14155] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" HandleID="k8s-pod-network.69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024eff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:10.944507014 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.945 [INFO][14155] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.945 [INFO][14155] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.945 [INFO][14155] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.957 [INFO][14155] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" host="172-234-25-215" Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.961 [INFO][14155] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.966 [INFO][14155] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.968 [INFO][14155] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.971 [INFO][14155] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.972 [INFO][14155] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" host="172-234-25-215" Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.973 [INFO][14155] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.977 [INFO][14155] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" host="172-234-25-215" Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.985 [INFO][14155] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.18/26] block=192.168.62.0/26 handle="k8s-pod-network.69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" host="172-234-25-215" Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.035673 containerd[1567]: 2025-08-13 01:31:10.986 [INFO][14155] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.18/26] handle="k8s-pod-network.69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" host="172-234-25-215" Aug 13 01:31:11.035673 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.036785 containerd[1567]: 2025-08-13 01:31:10.986 [INFO][14155] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:11.036785 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.036785 containerd[1567]: 2025-08-13 01:31:10.986 [INFO][14155] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.18/26] IPv6=[] ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" HandleID="k8s-pod-network.69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:11.036785 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.036785 containerd[1567]: 2025-08-13 01:31:10.991 [INFO][14142] cni-plugin/k8s.go 418: Populated endpoint ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20633", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ab989a90855e6d43f994c72bf05cad0bf3b457663bb515a476dcb6a54fd4c4f3", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.18/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:11.036785 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.036785 containerd[1567]: 2025-08-13 01:31:10.991 [INFO][14142] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.18/32] ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:11.036785 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.036785 containerd[1567]: 2025-08-13 01:31:10.991 [INFO][14142] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:11.036785 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.036785 containerd[1567]: 2025-08-13 01:31:11.011 [INFO][14142] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:11.036785 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.036785 containerd[1567]: 2025-08-13 01:31:11.014 [INFO][14142] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20633", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.18/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"86:52:74:cc:81:31", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:11.036785 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.037105 containerd[1567]: 2025-08-13 01:31:11.029 [INFO][14142] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:11.037105 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.097594 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:11.097604 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:11.178572 containerd[1567]: 2025-08-13 01:31:11.094 [INFO][14180] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Aug 13 01:31:11.178572 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.178572 containerd[1567]: 2025-08-13 01:31:11.095 [INFO][14180] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" iface="eth0" netns="/var/run/netns/cni-3edeac90-8aef-45f4-6b86-df5fe727efee" Aug 13 01:31:11.178572 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.178572 containerd[1567]: 2025-08-13 01:31:11.096 [INFO][14180] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" iface="eth0" netns="/var/run/netns/cni-3edeac90-8aef-45f4-6b86-df5fe727efee" Aug 13 01:31:11.178572 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.178572 containerd[1567]: 2025-08-13 01:31:11.103 [INFO][14180] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" after=7.665946ms iface="eth0" netns="/var/run/netns/cni-3edeac90-8aef-45f4-6b86-df5fe727efee" Aug 13 01:31:11.178572 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.178572 containerd[1567]: 2025-08-13 01:31:11.103 [INFO][14180] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Aug 13 01:31:11.178572 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.178572 containerd[1567]: 2025-08-13 01:31:11.103 [INFO][14180] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Aug 13 01:31:11.178572 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.178572 containerd[1567]: 2025-08-13 01:31:11.140 [INFO][14187] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" HandleID="k8s-pod-network.69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:11.178572 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.178572 containerd[1567]: 2025-08-13 01:31:11.140 [INFO][14187] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:11.178572 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.178572 containerd[1567]: 2025-08-13 01:31:11.140 [INFO][14187] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:11.178572 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.178572 containerd[1567]: 2025-08-13 01:31:11.172 [INFO][14187] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" HandleID="k8s-pod-network.69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:11.178572 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.178572 containerd[1567]: 2025-08-13 01:31:11.172 [INFO][14187] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" HandleID="k8s-pod-network.69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:11.178572 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.178572 containerd[1567]: 2025-08-13 01:31:11.174 [INFO][14187] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:11.178572 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.178572 containerd[1567]: 2025-08-13 01:31:11.176 [INFO][14180] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f" Aug 13 01:31:11.178572 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:11.182363 systemd[1]: run-netns-cni\x2d3edeac90\x2d8aef\x2d45f4\x2d6b86\x2ddf5fe727efee.mount: Deactivated successfully. Aug 13 01:31:11.183962 containerd[1567]: time="2025-08-13T01:31:11.183835663Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f-lo: no space left on device" Aug 13 01:31:11.184179 kubelet[2720]: E0813 01:31:11.184142 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f-lo: no space left on device" Aug 13 01:31:11.184223 kubelet[2720]: E0813 01:31:11.184200 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:11.184250 kubelet[2720]: E0813 01:31:11.184221 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:11.184295 kubelet[2720]: E0813 01:31:11.184270 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:11.879279 containerd[1567]: time="2025-08-13T01:31:11.879225533Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:11.990287 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:11.991092 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.922 [INFO][14197] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20703 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.922 [INFO][14197] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.945 [INFO][14208] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" HandleID="k8s-pod-network.ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.946 [INFO][14208] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" HandleID="k8s-pod-network.ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002c5020), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:11.94584635 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.946 [INFO][14208] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.947 [INFO][14208] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.947 [INFO][14208] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.956 [INFO][14208] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" host="172-234-25-215" Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.961 [INFO][14208] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.964 [INFO][14208] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.967 [INFO][14208] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.970 [INFO][14208] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.970 [INFO][14208] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" host="172-234-25-215" Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.971 [INFO][14208] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992 Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.974 [INFO][14208] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" host="172-234-25-215" Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.981 [INFO][14208] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.19/26] block=192.168.62.0/26 handle="k8s-pod-network.ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" host="172-234-25-215" Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.013524 containerd[1567]: 2025-08-13 01:31:11.981 [INFO][14208] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.19/26] handle="k8s-pod-network.ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" host="172-234-25-215" Aug 13 01:31:12.013524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.015373 containerd[1567]: 2025-08-13 01:31:11.981 [INFO][14208] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:12.015373 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.015373 containerd[1567]: 2025-08-13 01:31:11.981 [INFO][14208] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.19/26] IPv6=[] ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" HandleID="k8s-pod-network.ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:12.015373 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.015373 containerd[1567]: 2025-08-13 01:31:11.983 [INFO][14197] cni-plugin/k8s.go 418: Populated endpoint ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20703", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"69d320a80e35704bb57986f1ff40edef316cbe08746892248d1ebcf89a264f5f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.19/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:12.015373 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.015373 containerd[1567]: 2025-08-13 01:31:11.985 [INFO][14197] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.19/32] ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:12.015373 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.015373 containerd[1567]: 2025-08-13 01:31:11.985 [INFO][14197] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:12.015373 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.015373 containerd[1567]: 2025-08-13 01:31:11.989 [INFO][14197] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:12.015373 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.015373 containerd[1567]: 2025-08-13 01:31:11.989 [INFO][14197] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20703", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.19/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"06:87:7a:cb:93:9b", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:12.015373 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.015688 containerd[1567]: 2025-08-13 01:31:12.002 [INFO][14197] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:12.015688 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.114986 kubelet[2720]: E0813 01:31:12.114866 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:31:12.120662 kubelet[2720]: E0813 01:31:12.119698 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-349524588-hiNR sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2074644924: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:31:12.132862 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:12.133800 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:12.225245 containerd[1567]: 2025-08-13 01:31:12.128 [INFO][14230] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Aug 13 01:31:12.225245 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.225245 containerd[1567]: 2025-08-13 01:31:12.128 [INFO][14230] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" iface="eth0" netns="/var/run/netns/cni-e27c7e4d-698f-538e-63cc-da1e5a15839c" Aug 13 01:31:12.225245 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.225245 containerd[1567]: 2025-08-13 01:31:12.131 [INFO][14230] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" iface="eth0" netns="/var/run/netns/cni-e27c7e4d-698f-538e-63cc-da1e5a15839c" Aug 13 01:31:12.225245 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.225245 containerd[1567]: 2025-08-13 01:31:12.139 [INFO][14230] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" after=10.995369ms iface="eth0" netns="/var/run/netns/cni-e27c7e4d-698f-538e-63cc-da1e5a15839c" Aug 13 01:31:12.225245 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.225245 containerd[1567]: 2025-08-13 01:31:12.140 [INFO][14230] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Aug 13 01:31:12.225245 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.225245 containerd[1567]: 2025-08-13 01:31:12.140 [INFO][14230] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Aug 13 01:31:12.225245 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.225245 containerd[1567]: 2025-08-13 01:31:12.179 [INFO][14241] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" HandleID="k8s-pod-network.ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:12.225245 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.225245 containerd[1567]: 2025-08-13 01:31:12.181 [INFO][14241] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:12.225245 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.225245 containerd[1567]: 2025-08-13 01:31:12.182 [INFO][14241] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:12.225245 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.225245 containerd[1567]: 2025-08-13 01:31:12.219 [INFO][14241] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" HandleID="k8s-pod-network.ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:12.225245 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.225245 containerd[1567]: 2025-08-13 01:31:12.219 [INFO][14241] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" HandleID="k8s-pod-network.ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:12.225245 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.225245 containerd[1567]: 2025-08-13 01:31:12.221 [INFO][14241] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:12.225245 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.225245 containerd[1567]: 2025-08-13 01:31:12.223 [INFO][14230] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992" Aug 13 01:31:12.225245 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:12.229441 systemd[1]: run-netns-cni\x2de27c7e4d\x2d698f\x2d538e\x2d63cc\x2dda1e5a15839c.mount: Deactivated successfully. Aug 13 01:31:12.231056 containerd[1567]: time="2025-08-13T01:31:12.230978676Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992-lo: no space left on device" Aug 13 01:31:12.232408 kubelet[2720]: E0813 01:31:12.232133 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992-lo: no space left on device" Aug 13 01:31:12.232408 kubelet[2720]: E0813 01:31:12.232187 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:12.232408 kubelet[2720]: E0813 01:31:12.232209 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:12.232408 kubelet[2720]: E0813 01:31:12.232258 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:12.368144 containerd[1567]: time="2025-08-13T01:31:12.368070303Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:31:12.880648 containerd[1567]: time="2025-08-13T01:31:12.880575339Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:12.981025 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:12.983832 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.918 [INFO][14253] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20769 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.918 [INFO][14253] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.943 [INFO][14266] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" HandleID="k8s-pod-network.e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.943 [INFO][14266] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" HandleID="k8s-pod-network.e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f730), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:12.943094134 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.943 [INFO][14266] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.943 [INFO][14266] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.943 [INFO][14266] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.951 [INFO][14266] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" host="172-234-25-215" Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.957 [INFO][14266] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.961 [INFO][14266] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.963 [INFO][14266] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.965 [INFO][14266] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.965 [INFO][14266] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" host="172-234-25-215" Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.966 [INFO][14266] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5 Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.970 [INFO][14266] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" host="172-234-25-215" Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.975 [INFO][14266] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.20/26] block=192.168.62.0/26 handle="k8s-pod-network.e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" host="172-234-25-215" Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.000442 containerd[1567]: 2025-08-13 01:31:12.975 [INFO][14266] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.20/26] handle="k8s-pod-network.e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" host="172-234-25-215" Aug 13 01:31:13.000442 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.001444 containerd[1567]: 2025-08-13 01:31:12.975 [INFO][14266] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:13.001444 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.001444 containerd[1567]: 2025-08-13 01:31:12.975 [INFO][14266] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.20/26] IPv6=[] ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" HandleID="k8s-pod-network.e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:13.001444 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.001444 containerd[1567]: 2025-08-13 01:31:12.977 [INFO][14253] cni-plugin/k8s.go 418: Populated endpoint ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20769", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ccc5e75e3d3b38eb2ba60bbc14cb0b5ee728c7b161ae65aebb44cfee1c72a992", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.20/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:13.001444 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.001444 containerd[1567]: 2025-08-13 01:31:12.978 [INFO][14253] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.20/32] ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:13.001444 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.001444 containerd[1567]: 2025-08-13 01:31:12.978 [INFO][14253] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:13.001444 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.001444 containerd[1567]: 2025-08-13 01:31:12.983 [INFO][14253] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:13.001444 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.001444 containerd[1567]: 2025-08-13 01:31:12.985 [INFO][14253] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20769", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.20/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"12:97:3e:6d:59:c2", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:13.001444 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.005398 containerd[1567]: 2025-08-13 01:31:12.993 [INFO][14253] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:13.005398 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.068872 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:13.069020 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:13.130915 systemd[1]: Started sshd@49-172.234.25.215:22-147.75.109.163:50860.service - OpenSSH per-connection server daemon (147.75.109.163:50860). Aug 13 01:31:13.185384 containerd[1567]: 2025-08-13 01:31:13.064 [INFO][14289] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Aug 13 01:31:13.185384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.185384 containerd[1567]: 2025-08-13 01:31:13.065 [INFO][14289] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" iface="eth0" netns="/var/run/netns/cni-bc6c75b9-df4c-7dd4-1ac4-d6e291a0c3f3" Aug 13 01:31:13.185384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.185384 containerd[1567]: 2025-08-13 01:31:13.066 [INFO][14289] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" iface="eth0" netns="/var/run/netns/cni-bc6c75b9-df4c-7dd4-1ac4-d6e291a0c3f3" Aug 13 01:31:13.185384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.185384 containerd[1567]: 2025-08-13 01:31:13.074 [INFO][14289] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" after=8.502755ms iface="eth0" netns="/var/run/netns/cni-bc6c75b9-df4c-7dd4-1ac4-d6e291a0c3f3" Aug 13 01:31:13.185384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.185384 containerd[1567]: 2025-08-13 01:31:13.074 [INFO][14289] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Aug 13 01:31:13.185384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.185384 containerd[1567]: 2025-08-13 01:31:13.074 [INFO][14289] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Aug 13 01:31:13.185384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.185384 containerd[1567]: 2025-08-13 01:31:13.130 [INFO][14297] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" HandleID="k8s-pod-network.e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:13.185384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.185384 containerd[1567]: 2025-08-13 01:31:13.131 [INFO][14297] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:13.185384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.185384 containerd[1567]: 2025-08-13 01:31:13.131 [INFO][14297] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:13.185384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.185384 containerd[1567]: 2025-08-13 01:31:13.176 [INFO][14297] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" HandleID="k8s-pod-network.e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:13.185384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.185384 containerd[1567]: 2025-08-13 01:31:13.176 [INFO][14297] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" HandleID="k8s-pod-network.e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:13.185384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.185384 containerd[1567]: 2025-08-13 01:31:13.177 [INFO][14297] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:13.185384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.185384 containerd[1567]: 2025-08-13 01:31:13.182 [INFO][14289] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5" Aug 13 01:31:13.185384 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:13.187552 systemd[1]: run-netns-cni\x2dbc6c75b9\x2ddf4c\x2d7dd4\x2d1ac4\x2dd6e291a0c3f3.mount: Deactivated successfully. Aug 13 01:31:13.188838 containerd[1567]: time="2025-08-13T01:31:13.188779232Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5-lo: no space left on device" Aug 13 01:31:13.189542 kubelet[2720]: E0813 01:31:13.189271 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5-lo: no space left on device" Aug 13 01:31:13.189542 kubelet[2720]: E0813 01:31:13.189326 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:13.189542 kubelet[2720]: E0813 01:31:13.189345 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:13.189542 kubelet[2720]: E0813 01:31:13.189388 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:13.472331 sshd[14307]: Accepted publickey for core from 147.75.109.163 port 50860 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:31:13.473990 sshd-session[14307]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:31:13.478701 systemd-logind[1543]: New session 50 of user core. Aug 13 01:31:13.483902 systemd[1]: Started session-50.scope - Session 50 of User core. Aug 13 01:31:13.780049 sshd[14310]: Connection closed by 147.75.109.163 port 50860 Aug 13 01:31:13.780657 sshd-session[14307]: pam_unix(sshd:session): session closed for user core Aug 13 01:31:13.785427 systemd[1]: sshd@49-172.234.25.215:22-147.75.109.163:50860.service: Deactivated successfully. Aug 13 01:31:13.787899 systemd[1]: session-50.scope: Deactivated successfully. Aug 13 01:31:13.789446 systemd-logind[1543]: Session 50 logged out. Waiting for processes to exit. Aug 13 01:31:13.791344 systemd-logind[1543]: Removed session 50. Aug 13 01:31:13.882628 containerd[1567]: time="2025-08-13T01:31:13.882584415Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:13.993914 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:13.996616 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.928 [INFO][14325] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20845 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.928 [INFO][14325] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.954 [INFO][14335] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" HandleID="k8s-pod-network.ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.954 [INFO][14335] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" HandleID="k8s-pod-network.ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d4ff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:13.954108433 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.954 [INFO][14335] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.954 [INFO][14335] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.954 [INFO][14335] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.960 [INFO][14335] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" host="172-234-25-215" Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.964 [INFO][14335] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.969 [INFO][14335] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.971 [INFO][14335] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.974 [INFO][14335] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.974 [INFO][14335] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" host="172-234-25-215" Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.976 [INFO][14335] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.981 [INFO][14335] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" host="172-234-25-215" Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.987 [INFO][14335] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.21/26] block=192.168.62.0/26 handle="k8s-pod-network.ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" host="172-234-25-215" Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.019813 containerd[1567]: 2025-08-13 01:31:13.987 [INFO][14335] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.21/26] handle="k8s-pod-network.ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" host="172-234-25-215" Aug 13 01:31:14.019813 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.030024 containerd[1567]: 2025-08-13 01:31:13.988 [INFO][14335] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:14.030024 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.030024 containerd[1567]: 2025-08-13 01:31:13.988 [INFO][14335] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.21/26] IPv6=[] ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" HandleID="k8s-pod-network.ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:14.030024 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.030024 containerd[1567]: 2025-08-13 01:31:13.991 [INFO][14325] cni-plugin/k8s.go 418: Populated endpoint ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20845", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e62d96c34153967f3e1c5a5a4925ba256e76f91e9200f36328778137c639a0b5", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.21/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:14.030024 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.030024 containerd[1567]: 2025-08-13 01:31:13.991 [INFO][14325] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.21/32] ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:14.030024 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.030024 containerd[1567]: 2025-08-13 01:31:13.991 [INFO][14325] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:14.030024 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.030024 containerd[1567]: 2025-08-13 01:31:13.994 [INFO][14325] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:14.030024 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.030024 containerd[1567]: 2025-08-13 01:31:13.995 [INFO][14325] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20845", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.21/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"fe:f5:51:30:54:40", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:14.030024 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.032791 containerd[1567]: 2025-08-13 01:31:14.015 [INFO][14325] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:14.032791 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.119594 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:14.119602 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:14.227043 containerd[1567]: 2025-08-13 01:31:14.115 [INFO][14360] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Aug 13 01:31:14.227043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.227043 containerd[1567]: 2025-08-13 01:31:14.116 [INFO][14360] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" iface="eth0" netns="/var/run/netns/cni-73e32d19-0b4a-fb86-7588-efa6283d8358" Aug 13 01:31:14.227043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.227043 containerd[1567]: 2025-08-13 01:31:14.116 [INFO][14360] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" iface="eth0" netns="/var/run/netns/cni-73e32d19-0b4a-fb86-7588-efa6283d8358" Aug 13 01:31:14.227043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.227043 containerd[1567]: 2025-08-13 01:31:14.158 [INFO][14360] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" after=41.935893ms iface="eth0" netns="/var/run/netns/cni-73e32d19-0b4a-fb86-7588-efa6283d8358" Aug 13 01:31:14.227043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.227043 containerd[1567]: 2025-08-13 01:31:14.158 [INFO][14360] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Aug 13 01:31:14.227043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.227043 containerd[1567]: 2025-08-13 01:31:14.158 [INFO][14360] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Aug 13 01:31:14.227043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.227043 containerd[1567]: 2025-08-13 01:31:14.187 [INFO][14373] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" HandleID="k8s-pod-network.ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:14.227043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.227043 containerd[1567]: 2025-08-13 01:31:14.187 [INFO][14373] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:14.227043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.227043 containerd[1567]: 2025-08-13 01:31:14.188 [INFO][14373] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:14.227043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.227043 containerd[1567]: 2025-08-13 01:31:14.220 [INFO][14373] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" HandleID="k8s-pod-network.ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:14.227043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.227043 containerd[1567]: 2025-08-13 01:31:14.220 [INFO][14373] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" HandleID="k8s-pod-network.ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:14.227043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.227043 containerd[1567]: 2025-08-13 01:31:14.222 [INFO][14373] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:14.227043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.227043 containerd[1567]: 2025-08-13 01:31:14.224 [INFO][14360] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d" Aug 13 01:31:14.227043 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:14.230037 containerd[1567]: time="2025-08-13T01:31:14.228838507Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d-lo: no space left on device" Aug 13 01:31:14.230364 systemd[1]: run-netns-cni\x2d73e32d19\x2d0b4a\x2dfb86\x2d7588\x2defa6283d8358.mount: Deactivated successfully. Aug 13 01:31:14.230578 kubelet[2720]: E0813 01:31:14.230545 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d-lo: no space left on device" Aug 13 01:31:14.231790 kubelet[2720]: E0813 01:31:14.231444 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:14.231865 kubelet[2720]: E0813 01:31:14.231833 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:14.231950 kubelet[2720]: E0813 01:31:14.231924 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:14.895534 containerd[1567]: time="2025-08-13T01:31:14.895331260Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:15.030558 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:15.032396 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:14.939 [INFO][14379] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20904 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:14.939 [INFO][14379] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:14.982 [INFO][14394] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" HandleID="k8s-pod-network.575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:14.983 [INFO][14394] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" HandleID="k8s-pod-network.575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cd960), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:14.982535588 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:14.983 [INFO][14394] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:14.983 [INFO][14394] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:14.983 [INFO][14394] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:14.991 [INFO][14394] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" host="172-234-25-215" Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:14.997 [INFO][14394] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:15.003 [INFO][14394] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:15.007 [INFO][14394] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:15.010 [INFO][14394] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:15.010 [INFO][14394] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" host="172-234-25-215" Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:15.012 [INFO][14394] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283 Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:15.015 [INFO][14394] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" host="172-234-25-215" Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:15.021 [INFO][14394] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.22/26] block=192.168.62.0/26 handle="k8s-pod-network.575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" host="172-234-25-215" Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.047228 containerd[1567]: 2025-08-13 01:31:15.021 [INFO][14394] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.22/26] handle="k8s-pod-network.575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" host="172-234-25-215" Aug 13 01:31:15.047228 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.050918 containerd[1567]: 2025-08-13 01:31:15.021 [INFO][14394] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:15.050918 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.050918 containerd[1567]: 2025-08-13 01:31:15.021 [INFO][14394] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.22/26] IPv6=[] ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" HandleID="k8s-pod-network.575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:15.050918 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.050918 containerd[1567]: 2025-08-13 01:31:15.024 [INFO][14379] cni-plugin/k8s.go 418: Populated endpoint ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20904", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"ca3bb6b85729595197824c261d82c0fbc7bd969372fcd1d76c061b67fd0c843d", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.22/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:15.050918 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.050918 containerd[1567]: 2025-08-13 01:31:15.025 [INFO][14379] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.22/32] ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:15.050918 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.050918 containerd[1567]: 2025-08-13 01:31:15.025 [INFO][14379] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:15.050918 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.050918 containerd[1567]: 2025-08-13 01:31:15.033 [INFO][14379] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:15.050918 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.050918 containerd[1567]: 2025-08-13 01:31:15.033 [INFO][14379] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20904", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.22/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"be:14:65:de:5e:3f", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:15.050918 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.051424 containerd[1567]: 2025-08-13 01:31:15.042 [INFO][14379] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:15.051424 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.135955 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:15.135964 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:15.231767 containerd[1567]: 2025-08-13 01:31:15.132 [INFO][14419] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Aug 13 01:31:15.231767 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.231767 containerd[1567]: 2025-08-13 01:31:15.133 [INFO][14419] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" iface="eth0" netns="/var/run/netns/cni-14ba47a4-9594-d561-ad18-42345f5e969b" Aug 13 01:31:15.231767 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.231767 containerd[1567]: 2025-08-13 01:31:15.133 [INFO][14419] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" iface="eth0" netns="/var/run/netns/cni-14ba47a4-9594-d561-ad18-42345f5e969b" Aug 13 01:31:15.231767 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.231767 containerd[1567]: 2025-08-13 01:31:15.139 [INFO][14419] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" after=6.222638ms iface="eth0" netns="/var/run/netns/cni-14ba47a4-9594-d561-ad18-42345f5e969b" Aug 13 01:31:15.231767 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.231767 containerd[1567]: 2025-08-13 01:31:15.140 [INFO][14419] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Aug 13 01:31:15.231767 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.231767 containerd[1567]: 2025-08-13 01:31:15.140 [INFO][14419] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Aug 13 01:31:15.231767 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.231767 containerd[1567]: 2025-08-13 01:31:15.179 [INFO][14428] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" HandleID="k8s-pod-network.575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:15.231767 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.231767 containerd[1567]: 2025-08-13 01:31:15.179 [INFO][14428] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:15.231767 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.231767 containerd[1567]: 2025-08-13 01:31:15.179 [INFO][14428] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:15.231767 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.231767 containerd[1567]: 2025-08-13 01:31:15.224 [INFO][14428] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" HandleID="k8s-pod-network.575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:15.231767 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.231767 containerd[1567]: 2025-08-13 01:31:15.224 [INFO][14428] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" HandleID="k8s-pod-network.575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:15.231767 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.231767 containerd[1567]: 2025-08-13 01:31:15.226 [INFO][14428] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:15.231767 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.231767 containerd[1567]: 2025-08-13 01:31:15.229 [INFO][14419] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283" Aug 13 01:31:15.231767 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:15.234953 containerd[1567]: time="2025-08-13T01:31:15.234843174Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283-lo: no space left on device" Aug 13 01:31:15.235740 kubelet[2720]: E0813 01:31:15.235610 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283-lo: no space left on device" Aug 13 01:31:15.235740 kubelet[2720]: E0813 01:31:15.235673 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:15.235740 kubelet[2720]: E0813 01:31:15.235694 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:15.235985 systemd[1]: run-netns-cni\x2d14ba47a4\x2d9594\x2dd561\x2dad18\x2d42345f5e969b.mount: Deactivated successfully. Aug 13 01:31:15.236611 kubelet[2720]: E0813 01:31:15.236350 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:15.368329 containerd[1567]: time="2025-08-13T01:31:15.368200869Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:31:15.896447 containerd[1567]: time="2025-08-13T01:31:15.896402636Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:16.009918 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:16.011124 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.939 [INFO][14439] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 20976 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.939 [INFO][14439] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.969 [INFO][14449] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" HandleID="k8s-pod-network.15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.969 [INFO][14449] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" HandleID="k8s-pod-network.15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f610), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:15.969659602 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.970 [INFO][14449] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.970 [INFO][14449] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.970 [INFO][14449] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.975 [INFO][14449] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" host="172-234-25-215" Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.979 [INFO][14449] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.984 [INFO][14449] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.987 [INFO][14449] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.989 [INFO][14449] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.989 [INFO][14449] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" host="172-234-25-215" Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.991 [INFO][14449] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:15.994 [INFO][14449] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" host="172-234-25-215" Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:16.002 [INFO][14449] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.23/26] block=192.168.62.0/26 handle="k8s-pod-network.15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" host="172-234-25-215" Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.025613 containerd[1567]: 2025-08-13 01:31:16.002 [INFO][14449] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.23/26] handle="k8s-pod-network.15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" host="172-234-25-215" Aug 13 01:31:16.025613 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.028532 containerd[1567]: 2025-08-13 01:31:16.002 [INFO][14449] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:16.028532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.028532 containerd[1567]: 2025-08-13 01:31:16.002 [INFO][14449] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.23/26] IPv6=[] ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" HandleID="k8s-pod-network.15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:16.028532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.028532 containerd[1567]: 2025-08-13 01:31:16.005 [INFO][14439] cni-plugin/k8s.go 418: Populated endpoint ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20976", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"575241fe1cf96d958d31c9faea4bd35deeaee7bb87006b2711a40ddb4f6f7283", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.23/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:16.028532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.028532 containerd[1567]: 2025-08-13 01:31:16.005 [INFO][14439] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.23/32] ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:16.028532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.028532 containerd[1567]: 2025-08-13 01:31:16.006 [INFO][14439] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:16.028532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.028532 containerd[1567]: 2025-08-13 01:31:16.011 [INFO][14439] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:16.028532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.028532 containerd[1567]: 2025-08-13 01:31:16.011 [INFO][14439] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"20976", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.23/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ce:e6:07:59:39:95", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:16.028532 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.030642 containerd[1567]: 2025-08-13 01:31:16.021 [INFO][14439] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:16.030642 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.105858 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:16.105868 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:16.203780 containerd[1567]: 2025-08-13 01:31:16.100 [INFO][14474] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Aug 13 01:31:16.203780 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.203780 containerd[1567]: 2025-08-13 01:31:16.100 [INFO][14474] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" iface="eth0" netns="/var/run/netns/cni-a4e8f91f-a147-41fa-eab7-e74cd1a70622" Aug 13 01:31:16.203780 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.203780 containerd[1567]: 2025-08-13 01:31:16.102 [INFO][14474] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" iface="eth0" netns="/var/run/netns/cni-a4e8f91f-a147-41fa-eab7-e74cd1a70622" Aug 13 01:31:16.203780 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.203780 containerd[1567]: 2025-08-13 01:31:16.112 [INFO][14474] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" after=12.301107ms iface="eth0" netns="/var/run/netns/cni-a4e8f91f-a147-41fa-eab7-e74cd1a70622" Aug 13 01:31:16.203780 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.203780 containerd[1567]: 2025-08-13 01:31:16.113 [INFO][14474] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Aug 13 01:31:16.203780 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.203780 containerd[1567]: 2025-08-13 01:31:16.113 [INFO][14474] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Aug 13 01:31:16.203780 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.203780 containerd[1567]: 2025-08-13 01:31:16.151 [INFO][14484] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" HandleID="k8s-pod-network.15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:16.203780 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.203780 containerd[1567]: 2025-08-13 01:31:16.152 [INFO][14484] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:16.203780 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.203780 containerd[1567]: 2025-08-13 01:31:16.153 [INFO][14484] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:16.203780 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.203780 containerd[1567]: 2025-08-13 01:31:16.193 [INFO][14484] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" HandleID="k8s-pod-network.15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:16.203780 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.203780 containerd[1567]: 2025-08-13 01:31:16.193 [INFO][14484] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" HandleID="k8s-pod-network.15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:16.203780 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.203780 containerd[1567]: 2025-08-13 01:31:16.194 [INFO][14484] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:16.203780 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.203780 containerd[1567]: 2025-08-13 01:31:16.197 [INFO][14474] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc" Aug 13 01:31:16.203780 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:16.203780 containerd[1567]: time="2025-08-13T01:31:16.201182445Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc-lo: no space left on device" Aug 13 01:31:16.204312 kubelet[2720]: E0813 01:31:16.201521 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc-lo: no space left on device" Aug 13 01:31:16.204312 kubelet[2720]: E0813 01:31:16.201627 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:16.204312 kubelet[2720]: E0813 01:31:16.201649 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:16.204312 kubelet[2720]: E0813 01:31:16.201945 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:16.206803 systemd[1]: run-netns-cni\x2da4e8f91f\x2da147\x2d41fa\x2deab7\x2de74cd1a70622.mount: Deactivated successfully. Aug 13 01:31:16.898211 containerd[1567]: time="2025-08-13T01:31:16.898150073Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:17.000274 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:17.005187 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.933 [INFO][14493] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21038 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.934 [INFO][14493] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.961 [INFO][14506] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" HandleID="k8s-pod-network.d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.961 [INFO][14506] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" HandleID="k8s-pod-network.d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5680), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:16.961255846 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.962 [INFO][14506] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.962 [INFO][14506] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.962 [INFO][14506] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.971 [INFO][14506] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" host="172-234-25-215" Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.976 [INFO][14506] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.980 [INFO][14506] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.981 [INFO][14506] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.983 [INFO][14506] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.984 [INFO][14506] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" host="172-234-25-215" Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.985 [INFO][14506] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82 Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.988 [INFO][14506] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" host="172-234-25-215" Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.993 [INFO][14506] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.24/26] block=192.168.62.0/26 handle="k8s-pod-network.d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" host="172-234-25-215" Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025039 containerd[1567]: 2025-08-13 01:31:16.993 [INFO][14506] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.24/26] handle="k8s-pod-network.d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" host="172-234-25-215" Aug 13 01:31:17.025039 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025671 containerd[1567]: 2025-08-13 01:31:16.993 [INFO][14506] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:17.025671 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025671 containerd[1567]: 2025-08-13 01:31:16.993 [INFO][14506] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.24/26] IPv6=[] ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" HandleID="k8s-pod-network.d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:17.025671 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025671 containerd[1567]: 2025-08-13 01:31:16.996 [INFO][14493] cni-plugin/k8s.go 418: Populated endpoint ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21038", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"15219b89f26151addaa30da7cee82a4415fb54b35c62fc1e70e4384d61b4b8bc", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.24/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:17.025671 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025671 containerd[1567]: 2025-08-13 01:31:16.996 [INFO][14493] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.24/32] ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:17.025671 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025671 containerd[1567]: 2025-08-13 01:31:16.996 [INFO][14493] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:17.025671 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025671 containerd[1567]: 2025-08-13 01:31:17.001 [INFO][14493] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:17.025671 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025671 containerd[1567]: 2025-08-13 01:31:17.003 [INFO][14493] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21038", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.24/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"9e:fe:55:ae:7a:c6", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:17.025671 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.025970 containerd[1567]: 2025-08-13 01:31:17.019 [INFO][14493] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:17.025970 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.093613 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:17.093621 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:17.201583 containerd[1567]: 2025-08-13 01:31:17.088 [INFO][14532] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Aug 13 01:31:17.201583 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.201583 containerd[1567]: 2025-08-13 01:31:17.089 [INFO][14532] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" iface="eth0" netns="/var/run/netns/cni-936656e0-2e32-5d34-ea75-d53d084642fa" Aug 13 01:31:17.201583 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.201583 containerd[1567]: 2025-08-13 01:31:17.092 [INFO][14532] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" iface="eth0" netns="/var/run/netns/cni-936656e0-2e32-5d34-ea75-d53d084642fa" Aug 13 01:31:17.201583 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.201583 containerd[1567]: 2025-08-13 01:31:17.100 [INFO][14532] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" after=10.71496ms iface="eth0" netns="/var/run/netns/cni-936656e0-2e32-5d34-ea75-d53d084642fa" Aug 13 01:31:17.201583 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.201583 containerd[1567]: 2025-08-13 01:31:17.102 [INFO][14532] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Aug 13 01:31:17.201583 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.201583 containerd[1567]: 2025-08-13 01:31:17.106 [INFO][14532] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Aug 13 01:31:17.201583 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.201583 containerd[1567]: 2025-08-13 01:31:17.159 [INFO][14542] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" HandleID="k8s-pod-network.d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:17.201583 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.201583 containerd[1567]: 2025-08-13 01:31:17.160 [INFO][14542] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:17.201583 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.201583 containerd[1567]: 2025-08-13 01:31:17.160 [INFO][14542] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:17.201583 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.201583 containerd[1567]: 2025-08-13 01:31:17.193 [INFO][14542] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" HandleID="k8s-pod-network.d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:17.201583 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.201583 containerd[1567]: 2025-08-13 01:31:17.193 [INFO][14542] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" HandleID="k8s-pod-network.d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:17.201583 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.201583 containerd[1567]: 2025-08-13 01:31:17.195 [INFO][14542] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:17.201583 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.201583 containerd[1567]: 2025-08-13 01:31:17.197 [INFO][14532] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82" Aug 13 01:31:17.201583 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:17.205961 containerd[1567]: time="2025-08-13T01:31:17.205910136Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82-lo: no space left on device" Aug 13 01:31:17.206197 kubelet[2720]: E0813 01:31:17.206163 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82-lo: no space left on device" Aug 13 01:31:17.206457 kubelet[2720]: E0813 01:31:17.206217 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:17.206457 kubelet[2720]: E0813 01:31:17.206286 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:17.206785 kubelet[2720]: E0813 01:31:17.206753 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:17.207931 systemd[1]: run-netns-cni\x2d936656e0\x2d2e32\x2d5d34\x2dea75\x2dd53d084642fa.mount: Deactivated successfully. Aug 13 01:31:17.659371 containerd[1567]: time="2025-08-13T01:31:17.659319862Z" level=info msg="TaskExit event in podsandbox handler container_id:\"05a7e6943f56eff710d015cc4d373aa0238d7506c1cf61f23b36bc5ebcb7ce84\" id:\"9753af5fe9377160a436dc99126dda8a2b7a32aad619ee239fba0387b0f18592\" pid:14562 exited_at:{seconds:1755048677 nanos:659042242}" Aug 13 01:31:17.900906 containerd[1567]: time="2025-08-13T01:31:17.900850197Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:18.011073 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:18.011823 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.942 [INFO][14575] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21105 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.942 [INFO][14575] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.972 [INFO][14589] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" HandleID="k8s-pod-network.809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.973 [INFO][14589] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" HandleID="k8s-pod-network.809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002c5130), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:17.972906045 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.973 [INFO][14589] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.973 [INFO][14589] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.973 [INFO][14589] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.978 [INFO][14589] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" host="172-234-25-215" Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.982 [INFO][14589] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.986 [INFO][14589] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.987 [INFO][14589] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.990 [INFO][14589] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.990 [INFO][14589] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" host="172-234-25-215" Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.991 [INFO][14589] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67 Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:17.994 [INFO][14589] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" host="172-234-25-215" Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:18.003 [INFO][14589] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.25/26] block=192.168.62.0/26 handle="k8s-pod-network.809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" host="172-234-25-215" Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.031296 containerd[1567]: 2025-08-13 01:31:18.003 [INFO][14589] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.25/26] handle="k8s-pod-network.809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" host="172-234-25-215" Aug 13 01:31:18.031296 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.032982 containerd[1567]: 2025-08-13 01:31:18.003 [INFO][14589] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:18.032982 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.032982 containerd[1567]: 2025-08-13 01:31:18.003 [INFO][14589] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.25/26] IPv6=[] ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" HandleID="k8s-pod-network.809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:18.032982 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.032982 containerd[1567]: 2025-08-13 01:31:18.006 [INFO][14575] cni-plugin/k8s.go 418: Populated endpoint ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21105", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"d56596ad708ffc2bcc6ef721fe496cff89d16387cb4dc1aecdebedf8becddf82", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.25/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:18.032982 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.032982 containerd[1567]: 2025-08-13 01:31:18.007 [INFO][14575] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.25/32] ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:18.032982 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.032982 containerd[1567]: 2025-08-13 01:31:18.007 [INFO][14575] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:18.032982 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.032982 containerd[1567]: 2025-08-13 01:31:18.013 [INFO][14575] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:18.032982 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.032982 containerd[1567]: 2025-08-13 01:31:18.014 [INFO][14575] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21105", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.25/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"c2:c4:d8:50:a7:46", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:18.032982 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.033488 containerd[1567]: 2025-08-13 01:31:18.024 [INFO][14575] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:18.033488 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.105316 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:18.105328 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:18.221600 containerd[1567]: 2025-08-13 01:31:18.099 [INFO][14611] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Aug 13 01:31:18.221600 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.221600 containerd[1567]: 2025-08-13 01:31:18.101 [INFO][14611] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" iface="eth0" netns="/var/run/netns/cni-989a0c24-ff9b-0366-4110-1e844841fd5e" Aug 13 01:31:18.221600 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.221600 containerd[1567]: 2025-08-13 01:31:18.102 [INFO][14611] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" iface="eth0" netns="/var/run/netns/cni-989a0c24-ff9b-0366-4110-1e844841fd5e" Aug 13 01:31:18.221600 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.221600 containerd[1567]: 2025-08-13 01:31:18.112 [INFO][14611] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" after=10.47593ms iface="eth0" netns="/var/run/netns/cni-989a0c24-ff9b-0366-4110-1e844841fd5e" Aug 13 01:31:18.221600 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.221600 containerd[1567]: 2025-08-13 01:31:18.112 [INFO][14611] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Aug 13 01:31:18.221600 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.221600 containerd[1567]: 2025-08-13 01:31:18.114 [INFO][14611] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Aug 13 01:31:18.221600 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.221600 containerd[1567]: 2025-08-13 01:31:18.146 [INFO][14621] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" HandleID="k8s-pod-network.809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:18.221600 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.221600 containerd[1567]: 2025-08-13 01:31:18.147 [INFO][14621] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:18.221600 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.221600 containerd[1567]: 2025-08-13 01:31:18.147 [INFO][14621] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:18.221600 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.221600 containerd[1567]: 2025-08-13 01:31:18.214 [INFO][14621] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" HandleID="k8s-pod-network.809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:18.221600 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.221600 containerd[1567]: 2025-08-13 01:31:18.214 [INFO][14621] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" HandleID="k8s-pod-network.809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:18.221600 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.221600 containerd[1567]: 2025-08-13 01:31:18.216 [INFO][14621] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:18.221600 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.221600 containerd[1567]: 2025-08-13 01:31:18.218 [INFO][14611] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67" Aug 13 01:31:18.221600 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:18.223537 containerd[1567]: time="2025-08-13T01:31:18.223430543Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67-lo: no space left on device" Aug 13 01:31:18.223880 kubelet[2720]: E0813 01:31:18.223847 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67-lo: no space left on device" Aug 13 01:31:18.224766 kubelet[2720]: E0813 01:31:18.224218 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:18.224766 kubelet[2720]: E0813 01:31:18.224240 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:18.224766 kubelet[2720]: E0813 01:31:18.224399 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:18.227074 systemd[1]: run-netns-cni\x2d989a0c24\x2dff9b\x2d0366\x2d4110\x2d1e844841fd5e.mount: Deactivated successfully. Aug 13 01:31:18.368592 containerd[1567]: time="2025-08-13T01:31:18.368475266Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:31:18.574025 kubelet[2720]: I0813 01:31:18.573995 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:31:18.574025 kubelet[2720]: I0813 01:31:18.574034 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:31:18.575645 kubelet[2720]: I0813 01:31:18.575611 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:31:18.586160 kubelet[2720]: I0813 01:31:18.586144 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:31:18.586271 kubelet[2720]: I0813 01:31:18.586255 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-kube-controllers-b57d47f84-lznkx","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:31:18.586342 kubelet[2720]: E0813 01:31:18.586282 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:31:18.586342 kubelet[2720]: E0813 01:31:18.586291 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:31:18.586342 kubelet[2720]: E0813 01:31:18.586299 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:18.586342 kubelet[2720]: E0813 01:31:18.586308 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:31:18.586342 kubelet[2720]: E0813 01:31:18.586316 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:31:18.586342 kubelet[2720]: E0813 01:31:18.586323 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:31:18.586342 kubelet[2720]: E0813 01:31:18.586331 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:31:18.586342 kubelet[2720]: E0813 01:31:18.586340 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:31:18.586342 kubelet[2720]: E0813 01:31:18.586347 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:31:18.586516 kubelet[2720]: E0813 01:31:18.586354 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:31:18.586516 kubelet[2720]: I0813 01:31:18.586362 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:31:18.843908 systemd[1]: Started sshd@50-172.234.25.215:22-147.75.109.163:59034.service - OpenSSH per-connection server daemon (147.75.109.163:59034). Aug 13 01:31:18.904660 containerd[1567]: time="2025-08-13T01:31:18.904607669Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:19.019604 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:19.019949 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:18.945 [INFO][14633] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21177 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:18.947 [INFO][14633] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:18.978 [INFO][14644] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" HandleID="k8s-pod-network.b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:18.978 [INFO][14644] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" HandleID="k8s-pod-network.b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024ef60), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:18.978263344 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:18.978 [INFO][14644] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:18.978 [INFO][14644] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:18.978 [INFO][14644] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:18.984 [INFO][14644] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" host="172-234-25-215" Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:18.988 [INFO][14644] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:18.993 [INFO][14644] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:18.995 [INFO][14644] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:18.999 [INFO][14644] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:18.999 [INFO][14644] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" host="172-234-25-215" Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:19.000 [INFO][14644] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3 Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:19.004 [INFO][14644] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" host="172-234-25-215" Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:19.010 [INFO][14644] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.26/26] block=192.168.62.0/26 handle="k8s-pod-network.b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" host="172-234-25-215" Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.040189 containerd[1567]: 2025-08-13 01:31:19.010 [INFO][14644] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.26/26] handle="k8s-pod-network.b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" host="172-234-25-215" Aug 13 01:31:19.040189 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.042196 containerd[1567]: 2025-08-13 01:31:19.010 [INFO][14644] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:19.042196 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.042196 containerd[1567]: 2025-08-13 01:31:19.010 [INFO][14644] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.26/26] IPv6=[] ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" HandleID="k8s-pod-network.b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:19.042196 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.042196 containerd[1567]: 2025-08-13 01:31:19.013 [INFO][14633] cni-plugin/k8s.go 418: Populated endpoint ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21177", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"809e4668598a7e65e5515d8be5e50c38edd3a0d7c4442219b4a91aa485bdec67", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.26/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:19.042196 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.042196 containerd[1567]: 2025-08-13 01:31:19.013 [INFO][14633] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.26/32] ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:19.042196 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.042196 containerd[1567]: 2025-08-13 01:31:19.013 [INFO][14633] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:19.042196 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.042196 containerd[1567]: 2025-08-13 01:31:19.019 [INFO][14633] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:19.042196 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.042196 containerd[1567]: 2025-08-13 01:31:19.020 [INFO][14633] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21177", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.26/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"36:07:2f:cf:25:69", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:19.042196 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.042671 containerd[1567]: 2025-08-13 01:31:19.031 [INFO][14633] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:19.042671 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.106886 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:19.107765 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:19.188497 sshd[14630]: Accepted publickey for core from 147.75.109.163 port 59034 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:31:19.190454 sshd-session[14630]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:31:19.193060 containerd[1567]: 2025-08-13 01:31:19.103 [INFO][14667] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Aug 13 01:31:19.193060 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.193060 containerd[1567]: 2025-08-13 01:31:19.105 [INFO][14667] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" iface="eth0" netns="/var/run/netns/cni-70b9a423-3ba3-ef62-0dbc-e862ac009b95" Aug 13 01:31:19.193060 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.193060 containerd[1567]: 2025-08-13 01:31:19.105 [INFO][14667] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" iface="eth0" netns="/var/run/netns/cni-70b9a423-3ba3-ef62-0dbc-e862ac009b95" Aug 13 01:31:19.193060 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.193060 containerd[1567]: 2025-08-13 01:31:19.115 [INFO][14667] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" after=10.408151ms iface="eth0" netns="/var/run/netns/cni-70b9a423-3ba3-ef62-0dbc-e862ac009b95" Aug 13 01:31:19.193060 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.193060 containerd[1567]: 2025-08-13 01:31:19.115 [INFO][14667] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Aug 13 01:31:19.193060 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.193060 containerd[1567]: 2025-08-13 01:31:19.115 [INFO][14667] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Aug 13 01:31:19.193060 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.193060 containerd[1567]: 2025-08-13 01:31:19.147 [INFO][14676] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" HandleID="k8s-pod-network.b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:19.193060 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.193060 containerd[1567]: 2025-08-13 01:31:19.149 [INFO][14676] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:19.193060 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.193060 containerd[1567]: 2025-08-13 01:31:19.149 [INFO][14676] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:19.193060 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.193060 containerd[1567]: 2025-08-13 01:31:19.185 [INFO][14676] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" HandleID="k8s-pod-network.b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:19.193060 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.193060 containerd[1567]: 2025-08-13 01:31:19.185 [INFO][14676] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" HandleID="k8s-pod-network.b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:19.193060 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.193060 containerd[1567]: 2025-08-13 01:31:19.186 [INFO][14676] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:19.193060 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.193060 containerd[1567]: 2025-08-13 01:31:19.189 [INFO][14667] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3" Aug 13 01:31:19.193060 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:19.196648 systemd[1]: run-netns-cni\x2d70b9a423\x2d3ba3\x2def62\x2d0dbc\x2de862ac009b95.mount: Deactivated successfully. Aug 13 01:31:19.200199 containerd[1567]: time="2025-08-13T01:31:19.199976176Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3-lo: no space left on device" Aug 13 01:31:19.201775 kubelet[2720]: E0813 01:31:19.201539 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3-lo: no space left on device" Aug 13 01:31:19.201775 kubelet[2720]: E0813 01:31:19.201593 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:19.201775 kubelet[2720]: E0813 01:31:19.201616 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:19.201775 kubelet[2720]: E0813 01:31:19.201667 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:19.204664 systemd-logind[1543]: New session 51 of user core. Aug 13 01:31:19.210899 systemd[1]: Started session-51.scope - Session 51 of User core. Aug 13 01:31:19.511627 sshd[14685]: Connection closed by 147.75.109.163 port 59034 Aug 13 01:31:19.510942 sshd-session[14630]: pam_unix(sshd:session): session closed for user core Aug 13 01:31:19.516501 systemd-logind[1543]: Session 51 logged out. Waiting for processes to exit. Aug 13 01:31:19.518866 systemd[1]: sshd@50-172.234.25.215:22-147.75.109.163:59034.service: Deactivated successfully. Aug 13 01:31:19.522533 systemd[1]: session-51.scope: Deactivated successfully. Aug 13 01:31:19.525213 systemd-logind[1543]: Removed session 51. Aug 13 01:31:19.908090 containerd[1567]: time="2025-08-13T01:31:19.907436693Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:20.006056 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:20.007081 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.940 [INFO][14697] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21243 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.941 [INFO][14697] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.966 [INFO][14709] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" HandleID="k8s-pod-network.9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.966 [INFO][14709] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" HandleID="k8s-pod-network.9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cfcb0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:19.966511104 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.966 [INFO][14709] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.966 [INFO][14709] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.966 [INFO][14709] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.973 [INFO][14709] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" host="172-234-25-215" Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.977 [INFO][14709] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.982 [INFO][14709] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.983 [INFO][14709] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.986 [INFO][14709] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.986 [INFO][14709] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" host="172-234-25-215" Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.987 [INFO][14709] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:19.992 [INFO][14709] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" host="172-234-25-215" Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:20.000 [INFO][14709] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.27/26] block=192.168.62.0/26 handle="k8s-pod-network.9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" host="172-234-25-215" Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.026763 containerd[1567]: 2025-08-13 01:31:20.000 [INFO][14709] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.27/26] handle="k8s-pod-network.9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" host="172-234-25-215" Aug 13 01:31:20.026763 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.028704 containerd[1567]: 2025-08-13 01:31:20.000 [INFO][14709] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:20.028704 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.028704 containerd[1567]: 2025-08-13 01:31:20.000 [INFO][14709] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.27/26] IPv6=[] ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" HandleID="k8s-pod-network.9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:20.028704 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.028704 containerd[1567]: 2025-08-13 01:31:20.002 [INFO][14697] cni-plugin/k8s.go 418: Populated endpoint ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21243", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"b050c621fb8ebe344978402107717c9773617aa2bff0cf3db2ec046a1d2d0dd3", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.27/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:20.028704 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.028704 containerd[1567]: 2025-08-13 01:31:20.003 [INFO][14697] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.27/32] ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:20.028704 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.028704 containerd[1567]: 2025-08-13 01:31:20.003 [INFO][14697] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:20.028704 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.028704 containerd[1567]: 2025-08-13 01:31:20.007 [INFO][14697] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:20.028704 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.028704 containerd[1567]: 2025-08-13 01:31:20.007 [INFO][14697] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21243", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.27/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"72:a3:3e:13:6c:3d", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:20.028704 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.029388 containerd[1567]: 2025-08-13 01:31:20.019 [INFO][14697] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:20.029388 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.078280 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:20.078288 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:20.182799 containerd[1567]: 2025-08-13 01:31:20.075 [INFO][14733] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Aug 13 01:31:20.182799 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.182799 containerd[1567]: 2025-08-13 01:31:20.076 [INFO][14733] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" iface="eth0" netns="/var/run/netns/cni-08d7454f-8df3-d68d-41ae-fea856fca51c" Aug 13 01:31:20.182799 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.182799 containerd[1567]: 2025-08-13 01:31:20.076 [INFO][14733] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" iface="eth0" netns="/var/run/netns/cni-08d7454f-8df3-d68d-41ae-fea856fca51c" Aug 13 01:31:20.182799 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.182799 containerd[1567]: 2025-08-13 01:31:20.083 [INFO][14733] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" after=7.126546ms iface="eth0" netns="/var/run/netns/cni-08d7454f-8df3-d68d-41ae-fea856fca51c" Aug 13 01:31:20.182799 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.182799 containerd[1567]: 2025-08-13 01:31:20.083 [INFO][14733] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Aug 13 01:31:20.182799 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.182799 containerd[1567]: 2025-08-13 01:31:20.083 [INFO][14733] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Aug 13 01:31:20.182799 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.182799 containerd[1567]: 2025-08-13 01:31:20.126 [INFO][14741] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" HandleID="k8s-pod-network.9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:20.182799 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.182799 containerd[1567]: 2025-08-13 01:31:20.127 [INFO][14741] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:20.182799 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.182799 containerd[1567]: 2025-08-13 01:31:20.127 [INFO][14741] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:20.182799 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.182799 containerd[1567]: 2025-08-13 01:31:20.174 [INFO][14741] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" HandleID="k8s-pod-network.9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:20.182799 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.182799 containerd[1567]: 2025-08-13 01:31:20.174 [INFO][14741] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" HandleID="k8s-pod-network.9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:20.182799 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.182799 containerd[1567]: 2025-08-13 01:31:20.176 [INFO][14741] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:20.182799 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.182799 containerd[1567]: 2025-08-13 01:31:20.179 [INFO][14733] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b" Aug 13 01:31:20.182799 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:20.186505 systemd[1]: run-netns-cni\x2d08d7454f\x2d8df3\x2dd68d\x2d41ae\x2dfea856fca51c.mount: Deactivated successfully. Aug 13 01:31:20.187693 containerd[1567]: time="2025-08-13T01:31:20.185693260Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b-lo: no space left on device" Aug 13 01:31:20.187910 kubelet[2720]: E0813 01:31:20.187874 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b-lo: no space left on device" Aug 13 01:31:20.188199 kubelet[2720]: E0813 01:31:20.187931 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:20.188199 kubelet[2720]: E0813 01:31:20.187954 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:20.188199 kubelet[2720]: E0813 01:31:20.188028 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:20.330150 containerd[1567]: time="2025-08-13T01:31:20.330058295Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:31:20.330150 containerd[1567]: time="2025-08-13T01:31:20.330140095Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:31:20.330311 containerd[1567]: time="2025-08-13T01:31:20.330168025Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:31:20.330311 containerd[1567]: time="2025-08-13T01:31:20.330190995Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:31:20.330311 containerd[1567]: time="2025-08-13T01:31:20.330212875Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:31:20.330311 containerd[1567]: time="2025-08-13T01:31:20.330234175Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:31:20.330311 containerd[1567]: time="2025-08-13T01:31:20.330255405Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:31:20.331034 containerd[1567]: time="2025-08-13T01:31:20.331012664Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:31:20.331103 containerd[1567]: time="2025-08-13T01:31:20.331059144Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log\"" error="write /var/log/pods/kube-system_kube-apiserver-172-234-25-215_5c228d00d203ca7627372d21a4f939a9/kube-apiserver/0.log: no space left on device" Aug 13 01:31:20.908347 containerd[1567]: time="2025-08-13T01:31:20.908083321Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:21.010386 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:21.010593 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.944 [INFO][14750] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21301 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.944 [INFO][14750] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.972 [INFO][14765] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" HandleID="k8s-pod-network.b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.972 [INFO][14765] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" HandleID="k8s-pod-network.b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5730), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:20.972452592 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.972 [INFO][14765] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.972 [INFO][14765] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.972 [INFO][14765] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.978 [INFO][14765] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" host="172-234-25-215" Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.982 [INFO][14765] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.985 [INFO][14765] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.986 [INFO][14765] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.989 [INFO][14765] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.989 [INFO][14765] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" host="172-234-25-215" Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.990 [INFO][14765] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:20.993 [INFO][14765] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" host="172-234-25-215" Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:21.002 [INFO][14765] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.28/26] block=192.168.62.0/26 handle="k8s-pod-network.b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" host="172-234-25-215" Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.031195 containerd[1567]: 2025-08-13 01:31:21.002 [INFO][14765] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.28/26] handle="k8s-pod-network.b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" host="172-234-25-215" Aug 13 01:31:21.031195 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.032092 containerd[1567]: 2025-08-13 01:31:21.002 [INFO][14765] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:21.032092 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.032092 containerd[1567]: 2025-08-13 01:31:21.002 [INFO][14765] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.28/26] IPv6=[] ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" HandleID="k8s-pod-network.b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:21.032092 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.032092 containerd[1567]: 2025-08-13 01:31:21.005 [INFO][14750] cni-plugin/k8s.go 418: Populated endpoint ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21301", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"9e7b2301291f8df68ff44cf75ecdf54a79e14c3cbe2b7334dd93757a61ce2d3b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.28/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:21.032092 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.032092 containerd[1567]: 2025-08-13 01:31:21.005 [INFO][14750] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.28/32] ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:21.032092 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.032092 containerd[1567]: 2025-08-13 01:31:21.005 [INFO][14750] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:21.032092 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.032092 containerd[1567]: 2025-08-13 01:31:21.009 [INFO][14750] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:21.032092 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.032092 containerd[1567]: 2025-08-13 01:31:21.010 [INFO][14750] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21301", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.28/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"62:94:fc:2f:0b:aa", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:21.032092 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.032401 containerd[1567]: 2025-08-13 01:31:21.021 [INFO][14750] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:21.032401 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.074546 kubelet[2720]: E0813 01:31:21.074062 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:31:21.075267 kubelet[2720]: E0813 01:31:21.075051 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-299933554-aTh- sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2568257827: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-b2ds8" podUID="567c4932-37b3-4053-b427-2472bc4c195b" Aug 13 01:31:21.101294 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:21.101307 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:21.197829 containerd[1567]: 2025-08-13 01:31:21.097 [INFO][14790] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Aug 13 01:31:21.197829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.197829 containerd[1567]: 2025-08-13 01:31:21.098 [INFO][14790] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" iface="eth0" netns="/var/run/netns/cni-bd57e4ce-3321-7f58-98cf-60c5028ff03c" Aug 13 01:31:21.197829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.197829 containerd[1567]: 2025-08-13 01:31:21.098 [INFO][14790] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" iface="eth0" netns="/var/run/netns/cni-bd57e4ce-3321-7f58-98cf-60c5028ff03c" Aug 13 01:31:21.197829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.197829 containerd[1567]: 2025-08-13 01:31:21.107 [INFO][14790] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" after=9.422743ms iface="eth0" netns="/var/run/netns/cni-bd57e4ce-3321-7f58-98cf-60c5028ff03c" Aug 13 01:31:21.197829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.197829 containerd[1567]: 2025-08-13 01:31:21.107 [INFO][14790] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Aug 13 01:31:21.197829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.197829 containerd[1567]: 2025-08-13 01:31:21.108 [INFO][14790] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Aug 13 01:31:21.197829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.197829 containerd[1567]: 2025-08-13 01:31:21.160 [INFO][14797] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" HandleID="k8s-pod-network.b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:21.197829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.197829 containerd[1567]: 2025-08-13 01:31:21.160 [INFO][14797] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:21.197829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.197829 containerd[1567]: 2025-08-13 01:31:21.160 [INFO][14797] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:21.197829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.197829 containerd[1567]: 2025-08-13 01:31:21.190 [INFO][14797] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" HandleID="k8s-pod-network.b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:21.197829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.197829 containerd[1567]: 2025-08-13 01:31:21.190 [INFO][14797] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" HandleID="k8s-pod-network.b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:21.197829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.197829 containerd[1567]: 2025-08-13 01:31:21.191 [INFO][14797] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:21.197829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.197829 containerd[1567]: 2025-08-13 01:31:21.193 [INFO][14790] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c" Aug 13 01:31:21.197829 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:21.202783 containerd[1567]: time="2025-08-13T01:31:21.199982713Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c-lo: no space left on device" Aug 13 01:31:21.201581 systemd[1]: run-netns-cni\x2dbd57e4ce\x2d3321\x2d7f58\x2d98cf\x2d60c5028ff03c.mount: Deactivated successfully. Aug 13 01:31:21.203296 kubelet[2720]: E0813 01:31:21.203240 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c-lo: no space left on device" Aug 13 01:31:21.203626 kubelet[2720]: E0813 01:31:21.203390 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:21.203626 kubelet[2720]: E0813 01:31:21.203413 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:21.204095 kubelet[2720]: E0813 01:31:21.204025 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:21.368607 containerd[1567]: time="2025-08-13T01:31:21.368555633Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:31:21.910511 containerd[1567]: time="2025-08-13T01:31:21.910234376Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:22.015291 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:22.016698 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:21.943 [INFO][14807] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21372 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:21.943 [INFO][14807] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:21.975 [INFO][14819] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" HandleID="k8s-pod-network.32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:21.976 [INFO][14819] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" HandleID="k8s-pod-network.32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024eff0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:21.975857816 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:21.976 [INFO][14819] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:21.976 [INFO][14819] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:21.976 [INFO][14819] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:21.982 [INFO][14819] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" host="172-234-25-215" Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:21.986 [INFO][14819] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:21.990 [INFO][14819] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:21.991 [INFO][14819] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:21.994 [INFO][14819] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:21.994 [INFO][14819] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" host="172-234-25-215" Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:21.995 [INFO][14819] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7 Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:22.000 [INFO][14819] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" host="172-234-25-215" Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:22.007 [INFO][14819] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.29/26] block=192.168.62.0/26 handle="k8s-pod-network.32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" host="172-234-25-215" Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.032702 containerd[1567]: 2025-08-13 01:31:22.007 [INFO][14819] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.29/26] handle="k8s-pod-network.32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" host="172-234-25-215" Aug 13 01:31:22.032702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.033773 containerd[1567]: 2025-08-13 01:31:22.007 [INFO][14819] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:22.033773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.033773 containerd[1567]: 2025-08-13 01:31:22.007 [INFO][14819] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.29/26] IPv6=[] ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" HandleID="k8s-pod-network.32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:22.033773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.033773 containerd[1567]: 2025-08-13 01:31:22.010 [INFO][14807] cni-plugin/k8s.go 418: Populated endpoint ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21372", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"b9e995c2bf91136c929f5cc67d4872a7b35fe6431c1ac25c5a475a75a8af5f7c", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.29/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:22.033773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.033773 containerd[1567]: 2025-08-13 01:31:22.011 [INFO][14807] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.29/32] ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:22.033773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.033773 containerd[1567]: 2025-08-13 01:31:22.011 [INFO][14807] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:22.033773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.033773 containerd[1567]: 2025-08-13 01:31:22.017 [INFO][14807] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:22.033773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.033773 containerd[1567]: 2025-08-13 01:31:22.017 [INFO][14807] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21372", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.29/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"d2:7d:be:9a:c6:5a", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:22.033773 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.034888 containerd[1567]: 2025-08-13 01:31:22.030 [INFO][14807] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:22.034888 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.108191 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:22.109604 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:22.244293 containerd[1567]: 2025-08-13 01:31:22.104 [INFO][14844] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Aug 13 01:31:22.244293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.244293 containerd[1567]: 2025-08-13 01:31:22.105 [INFO][14844] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" iface="eth0" netns="/var/run/netns/cni-9e61b4c0-6faa-9cce-8e3e-ffbf2cc75ee9" Aug 13 01:31:22.244293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.244293 containerd[1567]: 2025-08-13 01:31:22.106 [INFO][14844] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" iface="eth0" netns="/var/run/netns/cni-9e61b4c0-6faa-9cce-8e3e-ffbf2cc75ee9" Aug 13 01:31:22.244293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.244293 containerd[1567]: 2025-08-13 01:31:22.118 [INFO][14844] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" after=13.206345ms iface="eth0" netns="/var/run/netns/cni-9e61b4c0-6faa-9cce-8e3e-ffbf2cc75ee9" Aug 13 01:31:22.244293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.244293 containerd[1567]: 2025-08-13 01:31:22.119 [INFO][14844] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Aug 13 01:31:22.244293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.244293 containerd[1567]: 2025-08-13 01:31:22.119 [INFO][14844] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Aug 13 01:31:22.244293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.244293 containerd[1567]: 2025-08-13 01:31:22.184 [INFO][14856] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" HandleID="k8s-pod-network.32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:22.244293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.244293 containerd[1567]: 2025-08-13 01:31:22.184 [INFO][14856] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:22.244293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.244293 containerd[1567]: 2025-08-13 01:31:22.184 [INFO][14856] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:22.244293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.244293 containerd[1567]: 2025-08-13 01:31:22.238 [INFO][14856] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" HandleID="k8s-pod-network.32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:22.244293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.244293 containerd[1567]: 2025-08-13 01:31:22.238 [INFO][14856] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" HandleID="k8s-pod-network.32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:22.244293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.244293 containerd[1567]: 2025-08-13 01:31:22.239 [INFO][14856] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:22.244293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.244293 containerd[1567]: 2025-08-13 01:31:22.241 [INFO][14844] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7" Aug 13 01:31:22.244293 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:22.246644 containerd[1567]: time="2025-08-13T01:31:22.246605097Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7-lo: no space left on device" Aug 13 01:31:22.247739 kubelet[2720]: E0813 01:31:22.247621 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7-lo: no space left on device" Aug 13 01:31:22.248030 systemd[1]: run-netns-cni\x2d9e61b4c0\x2d6faa\x2d9cce\x2d8e3e\x2dffbf2cc75ee9.mount: Deactivated successfully. Aug 13 01:31:22.248976 kubelet[2720]: E0813 01:31:22.247669 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:22.248976 kubelet[2720]: E0813 01:31:22.248665 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:22.248976 kubelet[2720]: E0813 01:31:22.248761 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:22.911853 containerd[1567]: time="2025-08-13T01:31:22.911812292Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:23.011894 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:23.014824 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.943 [INFO][14866] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21439 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.943 [INFO][14866] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.969 [INFO][14878] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" HandleID="k8s-pod-network.db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.970 [INFO][14878] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" HandleID="k8s-pod-network.db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cd710), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:22.969956175 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.970 [INFO][14878] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.970 [INFO][14878] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.970 [INFO][14878] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.975 [INFO][14878] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" host="172-234-25-215" Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.979 [INFO][14878] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.983 [INFO][14878] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.984 [INFO][14878] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.988 [INFO][14878] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.988 [INFO][14878] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" host="172-234-25-215" Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.990 [INFO][14878] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9 Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:22.995 [INFO][14878] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" host="172-234-25-215" Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:23.002 [INFO][14878] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.30/26] block=192.168.62.0/26 handle="k8s-pod-network.db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" host="172-234-25-215" Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.034460 containerd[1567]: 2025-08-13 01:31:23.002 [INFO][14878] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.30/26] handle="k8s-pod-network.db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" host="172-234-25-215" Aug 13 01:31:23.034460 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.035119 containerd[1567]: 2025-08-13 01:31:23.002 [INFO][14878] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:23.035119 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.035119 containerd[1567]: 2025-08-13 01:31:23.002 [INFO][14878] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.30/26] IPv6=[] ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" HandleID="k8s-pod-network.db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:23.035119 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.035119 containerd[1567]: 2025-08-13 01:31:23.006 [INFO][14866] cni-plugin/k8s.go 418: Populated endpoint ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21439", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"32439afd5292b4ce60a8584d8dbb863201829829565bd7e1b0d272f1c29d78a7", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.30/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:23.035119 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.035119 containerd[1567]: 2025-08-13 01:31:23.006 [INFO][14866] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.30/32] ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:23.035119 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.035119 containerd[1567]: 2025-08-13 01:31:23.006 [INFO][14866] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:23.035119 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.035119 containerd[1567]: 2025-08-13 01:31:23.015 [INFO][14866] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:23.035119 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.035119 containerd[1567]: 2025-08-13 01:31:23.016 [INFO][14866] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21439", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.30/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ee:4c:d5:b8:bd:75", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:23.035119 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.035436 containerd[1567]: 2025-08-13 01:31:23.024 [INFO][14866] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:23.035436 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.094458 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:23.094466 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:23.230192 containerd[1567]: 2025-08-13 01:31:23.091 [INFO][14902] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Aug 13 01:31:23.230192 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.230192 containerd[1567]: 2025-08-13 01:31:23.091 [INFO][14902] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" iface="eth0" netns="/var/run/netns/cni-2b028935-be64-6d0d-f7ee-b25e6921ea6c" Aug 13 01:31:23.230192 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.230192 containerd[1567]: 2025-08-13 01:31:23.092 [INFO][14902] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" iface="eth0" netns="/var/run/netns/cni-2b028935-be64-6d0d-f7ee-b25e6921ea6c" Aug 13 01:31:23.230192 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.230192 containerd[1567]: 2025-08-13 01:31:23.100 [INFO][14902] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" after=8.738054ms iface="eth0" netns="/var/run/netns/cni-2b028935-be64-6d0d-f7ee-b25e6921ea6c" Aug 13 01:31:23.230192 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.230192 containerd[1567]: 2025-08-13 01:31:23.101 [INFO][14902] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Aug 13 01:31:23.230192 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.230192 containerd[1567]: 2025-08-13 01:31:23.101 [INFO][14902] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Aug 13 01:31:23.230192 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.230192 containerd[1567]: 2025-08-13 01:31:23.146 [INFO][14911] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" HandleID="k8s-pod-network.db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:23.230192 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.230192 containerd[1567]: 2025-08-13 01:31:23.147 [INFO][14911] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:23.230192 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.230192 containerd[1567]: 2025-08-13 01:31:23.148 [INFO][14911] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:23.230192 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.230192 containerd[1567]: 2025-08-13 01:31:23.220 [INFO][14911] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" HandleID="k8s-pod-network.db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:23.230192 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.230192 containerd[1567]: 2025-08-13 01:31:23.221 [INFO][14911] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" HandleID="k8s-pod-network.db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:23.230192 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.230192 containerd[1567]: 2025-08-13 01:31:23.222 [INFO][14911] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:23.230192 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.230192 containerd[1567]: 2025-08-13 01:31:23.226 [INFO][14902] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9" Aug 13 01:31:23.230192 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:23.236747 systemd[1]: run-netns-cni\x2d2b028935\x2dbe64\x2d6d0d\x2df7ee\x2db25e6921ea6c.mount: Deactivated successfully. Aug 13 01:31:23.237767 containerd[1567]: time="2025-08-13T01:31:23.237676853Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9-lo: no space left on device" Aug 13 01:31:23.239673 kubelet[2720]: E0813 01:31:23.239309 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9-lo: no space left on device" Aug 13 01:31:23.239673 kubelet[2720]: E0813 01:31:23.239365 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:23.239673 kubelet[2720]: E0813 01:31:23.239386 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:23.239673 kubelet[2720]: E0813 01:31:23.239434 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:23.914650 containerd[1567]: time="2025-08-13T01:31:23.914609126Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:24.024328 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:24.025083 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:23.953 [INFO][14921] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21511 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:23.953 [INFO][14921] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:23.985 [INFO][14935] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" HandleID="k8s-pod-network.0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:23.985 [INFO][14935] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" HandleID="k8s-pod-network.0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5950), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:23.985522826 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:23.985 [INFO][14935] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:23.985 [INFO][14935] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:23.985 [INFO][14935] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:23.991 [INFO][14935] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" host="172-234-25-215" Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:23.995 [INFO][14935] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:24.000 [INFO][14935] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:24.002 [INFO][14935] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:24.006 [INFO][14935] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:24.006 [INFO][14935] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" host="172-234-25-215" Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:24.007 [INFO][14935] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:24.009 [INFO][14935] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" host="172-234-25-215" Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:24.016 [INFO][14935] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.31/26] block=192.168.62.0/26 handle="k8s-pod-network.0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" host="172-234-25-215" Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.047928 containerd[1567]: 2025-08-13 01:31:24.016 [INFO][14935] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.31/26] handle="k8s-pod-network.0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" host="172-234-25-215" Aug 13 01:31:24.047928 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.048702 containerd[1567]: 2025-08-13 01:31:24.016 [INFO][14935] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:24.048702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.048702 containerd[1567]: 2025-08-13 01:31:24.016 [INFO][14935] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.31/26] IPv6=[] ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" HandleID="k8s-pod-network.0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:24.048702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.048702 containerd[1567]: 2025-08-13 01:31:24.019 [INFO][14921] cni-plugin/k8s.go 418: Populated endpoint ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21511", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"db2d6a8243644e66c639a9c7868ccfe91f825f4930ef107893297b4ffae52df9", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.31/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:24.048702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.048702 containerd[1567]: 2025-08-13 01:31:24.020 [INFO][14921] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.31/32] ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:24.048702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.048702 containerd[1567]: 2025-08-13 01:31:24.020 [INFO][14921] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:24.048702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.048702 containerd[1567]: 2025-08-13 01:31:24.025 [INFO][14921] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:24.048702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.048702 containerd[1567]: 2025-08-13 01:31:24.027 [INFO][14921] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21511", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.31/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"f6:cd:fe:6f:27:ad", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:24.048702 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.051413 containerd[1567]: 2025-08-13 01:31:24.036 [INFO][14921] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:24.051413 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.080751 kubelet[2720]: E0813 01:31:24.077751 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:31:24.081487 kubelet[2720]: E0813 01:31:24.081400 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-349524588-hiNR sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2074644924: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:31:24.134544 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:24.134556 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:24.268706 containerd[1567]: 2025-08-13 01:31:24.132 [INFO][14958] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Aug 13 01:31:24.268706 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.268706 containerd[1567]: 2025-08-13 01:31:24.132 [INFO][14958] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" iface="eth0" netns="/var/run/netns/cni-495f9414-a1c1-23ef-1542-1d83c3a653c8" Aug 13 01:31:24.268706 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.268706 containerd[1567]: 2025-08-13 01:31:24.132 [INFO][14958] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" iface="eth0" netns="/var/run/netns/cni-495f9414-a1c1-23ef-1542-1d83c3a653c8" Aug 13 01:31:24.268706 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.268706 containerd[1567]: 2025-08-13 01:31:24.139 [INFO][14958] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" after=7.103808ms iface="eth0" netns="/var/run/netns/cni-495f9414-a1c1-23ef-1542-1d83c3a653c8" Aug 13 01:31:24.268706 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.268706 containerd[1567]: 2025-08-13 01:31:24.140 [INFO][14958] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Aug 13 01:31:24.268706 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.268706 containerd[1567]: 2025-08-13 01:31:24.140 [INFO][14958] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Aug 13 01:31:24.268706 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.268706 containerd[1567]: 2025-08-13 01:31:24.211 [INFO][14967] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" HandleID="k8s-pod-network.0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:24.268706 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.268706 containerd[1567]: 2025-08-13 01:31:24.216 [INFO][14967] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:24.268706 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.268706 containerd[1567]: 2025-08-13 01:31:24.216 [INFO][14967] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:24.268706 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.268706 containerd[1567]: 2025-08-13 01:31:24.260 [INFO][14967] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" HandleID="k8s-pod-network.0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:24.268706 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.268706 containerd[1567]: 2025-08-13 01:31:24.261 [INFO][14967] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" HandleID="k8s-pod-network.0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:24.268706 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.268706 containerd[1567]: 2025-08-13 01:31:24.263 [INFO][14967] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:24.268706 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.268706 containerd[1567]: 2025-08-13 01:31:24.265 [INFO][14958] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f" Aug 13 01:31:24.268706 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:24.272512 systemd[1]: run-netns-cni\x2d495f9414\x2da1c1\x2d23ef\x2d1542\x2d1d83c3a653c8.mount: Deactivated successfully. Aug 13 01:31:24.274357 containerd[1567]: time="2025-08-13T01:31:24.272496608Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f-lo: no space left on device" Aug 13 01:31:24.274621 kubelet[2720]: E0813 01:31:24.274437 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f-lo: no space left on device" Aug 13 01:31:24.274621 kubelet[2720]: E0813 01:31:24.274535 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:24.274621 kubelet[2720]: E0813 01:31:24.274557 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:24.274708 kubelet[2720]: E0813 01:31:24.274628 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:24.368806 containerd[1567]: time="2025-08-13T01:31:24.368735440Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:31:24.570653 systemd[1]: Started sshd@51-172.234.25.215:22-147.75.109.163:59038.service - OpenSSH per-connection server daemon (147.75.109.163:59038). Aug 13 01:31:24.903577 sshd[14979]: Accepted publickey for core from 147.75.109.163 port 59038 ssh2: RSA SHA256:r4aVwGcRWirUnoa2gT2aTsV0OcegqRzu+Xc09vJuKwo Aug 13 01:31:24.905318 sshd-session[14979]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Aug 13 01:31:24.911458 systemd-logind[1543]: New session 52 of user core. Aug 13 01:31:24.914848 systemd[1]: Started session-52.scope - Session 52 of User core. Aug 13 01:31:24.916998 containerd[1567]: time="2025-08-13T01:31:24.916917392Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:25.053804 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:25.054941 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:24.959 [INFO][14983] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21585 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:24.960 [INFO][14983] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:24.991 [INFO][14994] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" HandleID="k8s-pod-network.0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:24.991 [INFO][14994] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" HandleID="k8s-pod-network.0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5020), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:24.991374545 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:24.991 [INFO][14994] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:24.991 [INFO][14994] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:24.992 [INFO][14994] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:25.001 [INFO][14994] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" host="172-234-25-215" Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:25.007 [INFO][14994] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:25.018 [INFO][14994] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:25.020 [INFO][14994] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:25.024 [INFO][14994] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:25.024 [INFO][14994] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" host="172-234-25-215" Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:25.026 [INFO][14994] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:25.032 [INFO][14994] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" host="172-234-25-215" Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:25.041 [INFO][14994] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.32/26] block=192.168.62.0/26 handle="k8s-pod-network.0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" host="172-234-25-215" Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.077133 containerd[1567]: 2025-08-13 01:31:25.041 [INFO][14994] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.32/26] handle="k8s-pod-network.0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" host="172-234-25-215" Aug 13 01:31:25.077133 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.079393 containerd[1567]: 2025-08-13 01:31:25.041 [INFO][14994] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:25.079393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.079393 containerd[1567]: 2025-08-13 01:31:25.041 [INFO][14994] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.32/26] IPv6=[] ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" HandleID="k8s-pod-network.0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:25.079393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.079393 containerd[1567]: 2025-08-13 01:31:25.045 [INFO][14983] cni-plugin/k8s.go 418: Populated endpoint ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21585", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0d0c0a7653d8fe6844f449401544161ffadf0cb32773656fb55c6fda3e61e22f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.32/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:25.079393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.079393 containerd[1567]: 2025-08-13 01:31:25.045 [INFO][14983] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.32/32] ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:25.079393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.079393 containerd[1567]: 2025-08-13 01:31:25.045 [INFO][14983] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:25.079393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.079393 containerd[1567]: 2025-08-13 01:31:25.055 [INFO][14983] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:25.079393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.079393 containerd[1567]: 2025-08-13 01:31:25.056 [INFO][14983] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21585", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.32/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"aa:57:82:b4:bf:7c", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:25.079393 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.079678 containerd[1567]: 2025-08-13 01:31:25.071 [INFO][14983] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:25.079678 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.174320 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:25.174328 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:25.275761 containerd[1567]: 2025-08-13 01:31:25.169 [INFO][15020] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Aug 13 01:31:25.275761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.275761 containerd[1567]: 2025-08-13 01:31:25.169 [INFO][15020] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" iface="eth0" netns="/var/run/netns/cni-a9f2aa9b-f26a-74cb-cf97-e4ac79754d7b" Aug 13 01:31:25.275761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.275761 containerd[1567]: 2025-08-13 01:31:25.170 [INFO][15020] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" iface="eth0" netns="/var/run/netns/cni-a9f2aa9b-f26a-74cb-cf97-e4ac79754d7b" Aug 13 01:31:25.275761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.275761 containerd[1567]: 2025-08-13 01:31:25.179 [INFO][15020] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" after=10.010591ms iface="eth0" netns="/var/run/netns/cni-a9f2aa9b-f26a-74cb-cf97-e4ac79754d7b" Aug 13 01:31:25.275761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.275761 containerd[1567]: 2025-08-13 01:31:25.179 [INFO][15020] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Aug 13 01:31:25.275761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.275761 containerd[1567]: 2025-08-13 01:31:25.179 [INFO][15020] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Aug 13 01:31:25.275761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.275761 containerd[1567]: 2025-08-13 01:31:25.222 [INFO][15032] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" HandleID="k8s-pod-network.0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:25.275761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.275761 containerd[1567]: 2025-08-13 01:31:25.223 [INFO][15032] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:25.275761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.275761 containerd[1567]: 2025-08-13 01:31:25.224 [INFO][15032] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:25.275761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.275761 containerd[1567]: 2025-08-13 01:31:25.266 [INFO][15032] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" HandleID="k8s-pod-network.0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:25.275761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.275761 containerd[1567]: 2025-08-13 01:31:25.267 [INFO][15032] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" HandleID="k8s-pod-network.0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:25.275761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.275761 containerd[1567]: 2025-08-13 01:31:25.268 [INFO][15032] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:25.275761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.275761 containerd[1567]: 2025-08-13 01:31:25.271 [INFO][15020] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba" Aug 13 01:31:25.275761 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:25.278535 systemd[1]: run-netns-cni\x2da9f2aa9b\x2df26a\x2d74cb\x2dcf97\x2de4ac79754d7b.mount: Deactivated successfully. Aug 13 01:31:25.280706 containerd[1567]: time="2025-08-13T01:31:25.280613653Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba-lo: no space left on device" Aug 13 01:31:25.281129 kubelet[2720]: E0813 01:31:25.281090 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba-lo: no space left on device" Aug 13 01:31:25.281129 kubelet[2720]: E0813 01:31:25.281140 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:25.281467 kubelet[2720]: E0813 01:31:25.281161 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:25.281467 kubelet[2720]: E0813 01:31:25.281210 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:25.288328 sshd[14981]: Connection closed by 147.75.109.163 port 59038 Aug 13 01:31:25.286868 sshd-session[14979]: pam_unix(sshd:session): session closed for user core Aug 13 01:31:25.291536 systemd-logind[1543]: Session 52 logged out. Waiting for processes to exit. Aug 13 01:31:25.291921 systemd[1]: sshd@51-172.234.25.215:22-147.75.109.163:59038.service: Deactivated successfully. Aug 13 01:31:25.295100 systemd[1]: session-52.scope: Deactivated successfully. Aug 13 01:31:25.299132 systemd-logind[1543]: Removed session 52. Aug 13 01:31:25.918532 containerd[1567]: time="2025-08-13T01:31:25.918499159Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:26.026106 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:26.027248 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:26.038150 containerd[1567]: time="2025-08-13T01:31:26.038112779Z" level=error msg="garbage collection failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device" Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:25.959 [INFO][15046] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21654 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:25.959 [INFO][15046] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:25.985 [INFO][15059] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" HandleID="k8s-pod-network.8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:25.986 [INFO][15059] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" HandleID="k8s-pod-network.8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002c5700), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:25.985857455 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:25.986 [INFO][15059] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:25.986 [INFO][15059] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:25.986 [INFO][15059] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:25.992 [INFO][15059] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" host="172-234-25-215" Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:25.995 [INFO][15059] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:25.999 [INFO][15059] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:26.001 [INFO][15059] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:26.005 [INFO][15059] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:26.005 [INFO][15059] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" host="172-234-25-215" Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:26.007 [INFO][15059] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:26.011 [INFO][15059] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" host="172-234-25-215" Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:26.016 [INFO][15059] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.33/26] block=192.168.62.0/26 handle="k8s-pod-network.8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" host="172-234-25-215" Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.044796 containerd[1567]: 2025-08-13 01:31:26.017 [INFO][15059] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.33/26] handle="k8s-pod-network.8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" host="172-234-25-215" Aug 13 01:31:26.044796 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.046428 containerd[1567]: 2025-08-13 01:31:26.017 [INFO][15059] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:26.046428 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.046428 containerd[1567]: 2025-08-13 01:31:26.017 [INFO][15059] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.33/26] IPv6=[] ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" HandleID="k8s-pod-network.8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:26.046428 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.046428 containerd[1567]: 2025-08-13 01:31:26.020 [INFO][15046] cni-plugin/k8s.go 418: Populated endpoint ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21654", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"0832073313e831b77511929b54586cf6932713f801b2b07a9b02967aa3397bba", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.33/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:26.046428 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.046428 containerd[1567]: 2025-08-13 01:31:26.020 [INFO][15046] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.33/32] ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:26.046428 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.046428 containerd[1567]: 2025-08-13 01:31:26.020 [INFO][15046] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:26.046428 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.046428 containerd[1567]: 2025-08-13 01:31:26.028 [INFO][15046] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:26.046428 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.046428 containerd[1567]: 2025-08-13 01:31:26.029 [INFO][15046] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21654", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.33/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"26:56:09:33:74:09", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:26.046428 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.046750 containerd[1567]: 2025-08-13 01:31:26.041 [INFO][15046] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:26.046750 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.114649 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:26.114660 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:26.209198 containerd[1567]: 2025-08-13 01:31:26.113 [INFO][15081] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Aug 13 01:31:26.209198 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.209198 containerd[1567]: 2025-08-13 01:31:26.113 [INFO][15081] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" iface="eth0" netns="/var/run/netns/cni-92e61f5f-53ea-dc5f-45c3-05b58d8ce12f" Aug 13 01:31:26.209198 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.209198 containerd[1567]: 2025-08-13 01:31:26.113 [INFO][15081] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" iface="eth0" netns="/var/run/netns/cni-92e61f5f-53ea-dc5f-45c3-05b58d8ce12f" Aug 13 01:31:26.209198 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.209198 containerd[1567]: 2025-08-13 01:31:26.122 [INFO][15081] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" after=9.316283ms iface="eth0" netns="/var/run/netns/cni-92e61f5f-53ea-dc5f-45c3-05b58d8ce12f" Aug 13 01:31:26.209198 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.209198 containerd[1567]: 2025-08-13 01:31:26.122 [INFO][15081] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Aug 13 01:31:26.209198 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.209198 containerd[1567]: 2025-08-13 01:31:26.123 [INFO][15081] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Aug 13 01:31:26.209198 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.209198 containerd[1567]: 2025-08-13 01:31:26.154 [INFO][15090] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" HandleID="k8s-pod-network.8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:26.209198 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.209198 containerd[1567]: 2025-08-13 01:31:26.155 [INFO][15090] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:26.209198 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.209198 containerd[1567]: 2025-08-13 01:31:26.155 [INFO][15090] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:26.209198 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.209198 containerd[1567]: 2025-08-13 01:31:26.197 [INFO][15090] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" HandleID="k8s-pod-network.8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:26.209198 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.209198 containerd[1567]: 2025-08-13 01:31:26.198 [INFO][15090] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" HandleID="k8s-pod-network.8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:26.209198 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.209198 containerd[1567]: 2025-08-13 01:31:26.201 [INFO][15090] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:26.209198 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.209198 containerd[1567]: 2025-08-13 01:31:26.205 [INFO][15081] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f" Aug 13 01:31:26.209198 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:26.216448 containerd[1567]: time="2025-08-13T01:31:26.215241263Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f-lo: no space left on device" Aug 13 01:31:26.215532 systemd[1]: run-netns-cni\x2d92e61f5f\x2d53ea\x2ddc5f\x2d45c3\x2d05b58d8ce12f.mount: Deactivated successfully. Aug 13 01:31:26.216873 kubelet[2720]: E0813 01:31:26.216829 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f-lo: no space left on device" Aug 13 01:31:26.216918 kubelet[2720]: E0813 01:31:26.216893 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:26.216918 kubelet[2720]: E0813 01:31:26.216913 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:26.217527 kubelet[2720]: E0813 01:31:26.216977 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:26.919914 containerd[1567]: time="2025-08-13T01:31:26.919875597Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:27.030797 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:27.031811 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:26.956 [INFO][15101] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21723 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:26.956 [INFO][15101] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:26.979 [INFO][15112] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" HandleID="k8s-pod-network.b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:26.979 [INFO][15112] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" HandleID="k8s-pod-network.b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f100), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:26.979272138 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:26.979 [INFO][15112] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:26.979 [INFO][15112] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:26.979 [INFO][15112] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:26.987 [INFO][15112] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" host="172-234-25-215" Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:26.992 [INFO][15112] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:26.997 [INFO][15112] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:27.000 [INFO][15112] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:27.005 [INFO][15112] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:27.005 [INFO][15112] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" host="172-234-25-215" Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:27.007 [INFO][15112] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:27.010 [INFO][15112] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" host="172-234-25-215" Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:27.020 [INFO][15112] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.34/26] block=192.168.62.0/26 handle="k8s-pod-network.b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" host="172-234-25-215" Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.052322 containerd[1567]: 2025-08-13 01:31:27.021 [INFO][15112] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.34/26] handle="k8s-pod-network.b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" host="172-234-25-215" Aug 13 01:31:27.052322 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.054369 containerd[1567]: 2025-08-13 01:31:27.021 [INFO][15112] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:27.054369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.054369 containerd[1567]: 2025-08-13 01:31:27.021 [INFO][15112] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.34/26] IPv6=[] ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" HandleID="k8s-pod-network.b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:27.054369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.054369 containerd[1567]: 2025-08-13 01:31:27.025 [INFO][15101] cni-plugin/k8s.go 418: Populated endpoint ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21723", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"8bfe99061b27a21d26b5995078751b6ce1664ae9eea5c4780ea5b5ef716d879f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.34/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:27.054369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.054369 containerd[1567]: 2025-08-13 01:31:27.025 [INFO][15101] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.34/32] ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:27.054369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.054369 containerd[1567]: 2025-08-13 01:31:27.025 [INFO][15101] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:27.054369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.054369 containerd[1567]: 2025-08-13 01:31:27.034 [INFO][15101] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:27.054369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.054369 containerd[1567]: 2025-08-13 01:31:27.035 [INFO][15101] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21723", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.34/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"b6:88:d8:cb:74:7e", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:27.054369 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.054681 containerd[1567]: 2025-08-13 01:31:27.046 [INFO][15101] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:27.054681 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.129754 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:27.129764 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:27.215307 containerd[1567]: 2025-08-13 01:31:27.124 [INFO][15133] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Aug 13 01:31:27.215307 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.215307 containerd[1567]: 2025-08-13 01:31:27.126 [INFO][15133] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" iface="eth0" netns="/var/run/netns/cni-b3811099-a29f-8ca1-41a2-d21e02d38df7" Aug 13 01:31:27.215307 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.215307 containerd[1567]: 2025-08-13 01:31:27.126 [INFO][15133] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" iface="eth0" netns="/var/run/netns/cni-b3811099-a29f-8ca1-41a2-d21e02d38df7" Aug 13 01:31:27.215307 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.215307 containerd[1567]: 2025-08-13 01:31:27.134 [INFO][15133] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" after=8.065346ms iface="eth0" netns="/var/run/netns/cni-b3811099-a29f-8ca1-41a2-d21e02d38df7" Aug 13 01:31:27.215307 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.215307 containerd[1567]: 2025-08-13 01:31:27.134 [INFO][15133] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Aug 13 01:31:27.215307 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.215307 containerd[1567]: 2025-08-13 01:31:27.134 [INFO][15133] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Aug 13 01:31:27.215307 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.215307 containerd[1567]: 2025-08-13 01:31:27.173 [INFO][15143] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" HandleID="k8s-pod-network.b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:27.215307 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.215307 containerd[1567]: 2025-08-13 01:31:27.174 [INFO][15143] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:27.215307 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.215307 containerd[1567]: 2025-08-13 01:31:27.174 [INFO][15143] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:27.215307 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.215307 containerd[1567]: 2025-08-13 01:31:27.208 [INFO][15143] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" HandleID="k8s-pod-network.b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:27.215307 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.215307 containerd[1567]: 2025-08-13 01:31:27.208 [INFO][15143] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" HandleID="k8s-pod-network.b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:27.215307 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.215307 containerd[1567]: 2025-08-13 01:31:27.210 [INFO][15143] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:27.215307 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.215307 containerd[1567]: 2025-08-13 01:31:27.212 [INFO][15133] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e" Aug 13 01:31:27.215307 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:27.219247 systemd[1]: run-netns-cni\x2db3811099\x2da29f\x2d8ca1\x2d41a2\x2dd21e02d38df7.mount: Deactivated successfully. Aug 13 01:31:27.221790 containerd[1567]: time="2025-08-13T01:31:27.221536002Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e-lo: no space left on device" Aug 13 01:31:27.222387 kubelet[2720]: E0813 01:31:27.221935 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e-lo: no space left on device" Aug 13 01:31:27.222387 kubelet[2720]: E0813 01:31:27.221981 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:27.222387 kubelet[2720]: E0813 01:31:27.222000 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:27.222387 kubelet[2720]: E0813 01:31:27.222046 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:27.368925 containerd[1567]: time="2025-08-13T01:31:27.368871521Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:31:27.495895 containerd[1567]: time="2025-08-13T01:31:27.495768898Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-controller-manager-172-234-25-215_96c354cf60eb29e83ce001f635cc0212/kube-controller-manager/0.log\"" error="write /var/log/pods/kube-system_kube-controller-manager-172-234-25-215_96c354cf60eb29e83ce001f635cc0212/kube-controller-manager/0.log: no space left on device" Aug 13 01:31:27.922241 containerd[1567]: time="2025-08-13T01:31:27.922145373Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:28.017860 containerd[1567]: time="2025-08-13T01:31:28.017693977Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/kube-system_kube-controller-manager-172-234-25-215_96c354cf60eb29e83ce001f635cc0212/kube-controller-manager/0.log\"" error="write /var/log/pods/kube-system_kube-controller-manager-172-234-25-215_96c354cf60eb29e83ce001f635cc0212/kube-controller-manager/0.log: no space left on device" Aug 13 01:31:28.038694 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:28.038946 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:27.958 [INFO][15152] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21794 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:27.958 [INFO][15152] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:27.984 [INFO][15165] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" HandleID="k8s-pod-network.890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:27.984 [INFO][15165] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" HandleID="k8s-pod-network.890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002cd8e0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:27.984261548 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:27.984 [INFO][15165] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:27.984 [INFO][15165] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:27.984 [INFO][15165] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:27.991 [INFO][15165] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" host="172-234-25-215" Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:27.996 [INFO][15165] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:28.007 [INFO][15165] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:28.012 [INFO][15165] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:28.016 [INFO][15165] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:28.016 [INFO][15165] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" host="172-234-25-215" Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:28.019 [INFO][15165] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21 Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:28.023 [INFO][15165] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" host="172-234-25-215" Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:28.031 [INFO][15165] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.35/26] block=192.168.62.0/26 handle="k8s-pod-network.890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" host="172-234-25-215" Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.053860 containerd[1567]: 2025-08-13 01:31:28.031 [INFO][15165] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.35/26] handle="k8s-pod-network.890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" host="172-234-25-215" Aug 13 01:31:28.053860 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.055834 containerd[1567]: 2025-08-13 01:31:28.031 [INFO][15165] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:28.055834 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.055834 containerd[1567]: 2025-08-13 01:31:28.031 [INFO][15165] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.35/26] IPv6=[] ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" HandleID="k8s-pod-network.890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:28.055834 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.055834 containerd[1567]: 2025-08-13 01:31:28.035 [INFO][15152] cni-plugin/k8s.go 418: Populated endpoint ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21794", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"b9a3fcaf2425f1892ab51d2b4d34b7111242db3adb95caceba183775a251ed5e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.35/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:28.055834 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.055834 containerd[1567]: 2025-08-13 01:31:28.035 [INFO][15152] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.35/32] ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:28.055834 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.055834 containerd[1567]: 2025-08-13 01:31:28.035 [INFO][15152] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:28.055834 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.055834 containerd[1567]: 2025-08-13 01:31:28.039 [INFO][15152] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:28.055834 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.055834 containerd[1567]: 2025-08-13 01:31:28.039 [INFO][15152] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21794", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.35/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"56:ce:b7:9b:20:22", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:28.055834 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.056901 containerd[1567]: 2025-08-13 01:31:28.049 [INFO][15152] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:28.056901 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.126888 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:28.127012 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:28.239495 containerd[1567]: 2025-08-13 01:31:28.123 [INFO][15188] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Aug 13 01:31:28.239495 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.239495 containerd[1567]: 2025-08-13 01:31:28.124 [INFO][15188] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" iface="eth0" netns="/var/run/netns/cni-58154f67-a9c4-0499-64b0-bd60baeef380" Aug 13 01:31:28.239495 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.239495 containerd[1567]: 2025-08-13 01:31:28.124 [INFO][15188] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" iface="eth0" netns="/var/run/netns/cni-58154f67-a9c4-0499-64b0-bd60baeef380" Aug 13 01:31:28.239495 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.239495 containerd[1567]: 2025-08-13 01:31:28.137 [INFO][15188] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" after=13.344126ms iface="eth0" netns="/var/run/netns/cni-58154f67-a9c4-0499-64b0-bd60baeef380" Aug 13 01:31:28.239495 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.239495 containerd[1567]: 2025-08-13 01:31:28.137 [INFO][15188] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Aug 13 01:31:28.239495 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.239495 containerd[1567]: 2025-08-13 01:31:28.138 [INFO][15188] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Aug 13 01:31:28.239495 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.239495 containerd[1567]: 2025-08-13 01:31:28.169 [INFO][15200] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" HandleID="k8s-pod-network.890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:28.239495 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.239495 containerd[1567]: 2025-08-13 01:31:28.170 [INFO][15200] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:28.239495 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.239495 containerd[1567]: 2025-08-13 01:31:28.170 [INFO][15200] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:28.239495 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.239495 containerd[1567]: 2025-08-13 01:31:28.232 [INFO][15200] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" HandleID="k8s-pod-network.890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:28.239495 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.239495 containerd[1567]: 2025-08-13 01:31:28.232 [INFO][15200] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" HandleID="k8s-pod-network.890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:28.239495 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.239495 containerd[1567]: 2025-08-13 01:31:28.234 [INFO][15200] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:28.239495 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.239495 containerd[1567]: 2025-08-13 01:31:28.236 [INFO][15188] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21" Aug 13 01:31:28.239495 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:28.243596 systemd[1]: run-netns-cni\x2d58154f67\x2da9c4\x2d0499\x2d64b0\x2dbd60baeef380.mount: Deactivated successfully. Aug 13 01:31:28.245227 containerd[1567]: time="2025-08-13T01:31:28.245087798Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21-lo: no space left on device" Aug 13 01:31:28.245990 kubelet[2720]: E0813 01:31:28.245909 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21-lo: no space left on device" Aug 13 01:31:28.247469 kubelet[2720]: E0813 01:31:28.246278 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:28.247469 kubelet[2720]: E0813 01:31:28.247070 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:28.247558 kubelet[2720]: E0813 01:31:28.247339 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:28.608696 kubelet[2720]: I0813 01:31:28.608545 2720 eviction_manager.go:376] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" Aug 13 01:31:28.608696 kubelet[2720]: I0813 01:31:28.608607 2720 container_gc.go:86] "Attempting to delete unused containers" Aug 13 01:31:28.611405 kubelet[2720]: I0813 01:31:28.611368 2720 image_gc_manager.go:447] "Attempting to delete unused images" Aug 13 01:31:28.623492 kubelet[2720]: I0813 01:31:28.623471 2720 eviction_manager.go:387] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" Aug 13 01:31:28.623626 kubelet[2720]: I0813 01:31:28.623599 2720 eviction_manager.go:405] "Eviction manager: pods ranked for eviction" pods=["kube-system/coredns-674b8bbfcf-jlpx4","kube-system/coredns-674b8bbfcf-b2ds8","calico-system/calico-kube-controllers-b57d47f84-lznkx","calico-system/calico-typha-5bf7f956b7-szw5v","calico-system/calico-node-2p8hj","kube-system/kube-controller-manager-172-234-25-215","kube-system/kube-proxy-czb9t","calico-system/csi-node-driver-8bzzs","kube-system/kube-apiserver-172-234-25-215","kube-system/kube-scheduler-172-234-25-215"] Aug 13 01:31:28.623686 kubelet[2720]: E0813 01:31:28.623641 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-jlpx4" Aug 13 01:31:28.623686 kubelet[2720]: E0813 01:31:28.623652 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/coredns-674b8bbfcf-b2ds8" Aug 13 01:31:28.623686 kubelet[2720]: E0813 01:31:28.623662 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:28.623686 kubelet[2720]: E0813 01:31:28.623675 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-typha-5bf7f956b7-szw5v" Aug 13 01:31:28.623686 kubelet[2720]: E0813 01:31:28.623688 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/calico-node-2p8hj" Aug 13 01:31:28.623797 kubelet[2720]: E0813 01:31:28.623696 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-controller-manager-172-234-25-215" Aug 13 01:31:28.623797 kubelet[2720]: E0813 01:31:28.623704 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-proxy-czb9t" Aug 13 01:31:28.623834 kubelet[2720]: E0813 01:31:28.623805 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="calico-system/csi-node-driver-8bzzs" Aug 13 01:31:28.623834 kubelet[2720]: E0813 01:31:28.623813 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-apiserver-172-234-25-215" Aug 13 01:31:28.623834 kubelet[2720]: E0813 01:31:28.623821 2720 eviction_manager.go:610] "Eviction manager: cannot evict a critical pod" pod="kube-system/kube-scheduler-172-234-25-215" Aug 13 01:31:28.623834 kubelet[2720]: I0813 01:31:28.623830 2720 eviction_manager.go:439] "Eviction manager: unable to evict any pods from the node" Aug 13 01:31:28.923859 containerd[1567]: time="2025-08-13T01:31:28.923770269Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:29.030199 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:29.031852 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:28.961 [INFO][15219] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21858 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:28.962 [INFO][15219] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:28.985 [INFO][15231] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" HandleID="k8s-pod-network.39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:28.985 [INFO][15231] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" HandleID="k8s-pod-network.39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f1d0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:28.985304346 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:28.985 [INFO][15231] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:28.985 [INFO][15231] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:28.985 [INFO][15231] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:28.992 [INFO][15231] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" host="172-234-25-215" Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:28.998 [INFO][15231] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:29.005 [INFO][15231] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:29.009 [INFO][15231] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:29.011 [INFO][15231] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:29.011 [INFO][15231] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" host="172-234-25-215" Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:29.012 [INFO][15231] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:29.016 [INFO][15231] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" host="172-234-25-215" Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:29.023 [INFO][15231] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.36/26] block=192.168.62.0/26 handle="k8s-pod-network.39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" host="172-234-25-215" Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.050947 containerd[1567]: 2025-08-13 01:31:29.023 [INFO][15231] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.36/26] handle="k8s-pod-network.39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" host="172-234-25-215" Aug 13 01:31:29.050947 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.053539 containerd[1567]: 2025-08-13 01:31:29.023 [INFO][15231] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:29.053539 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.053539 containerd[1567]: 2025-08-13 01:31:29.023 [INFO][15231] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.36/26] IPv6=[] ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" HandleID="k8s-pod-network.39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:29.053539 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.053539 containerd[1567]: 2025-08-13 01:31:29.027 [INFO][15219] cni-plugin/k8s.go 418: Populated endpoint ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21858", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"890c78dadfec4cfcdc3c5c031bcac0bcb57cec71eee881578509a9873c646b21", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.36/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:29.053539 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.053539 containerd[1567]: 2025-08-13 01:31:29.027 [INFO][15219] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.36/32] ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:29.053539 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.053539 containerd[1567]: 2025-08-13 01:31:29.027 [INFO][15219] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:29.053539 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.053539 containerd[1567]: 2025-08-13 01:31:29.033 [INFO][15219] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:29.053539 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.053539 containerd[1567]: 2025-08-13 01:31:29.033 [INFO][15219] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21858", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.36/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"fa:a4:80:e3:b7:88", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:29.053539 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.054530 containerd[1567]: 2025-08-13 01:31:29.043 [INFO][15219] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:29.054530 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.110121 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:29.110130 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:29.193985 containerd[1567]: 2025-08-13 01:31:29.108 [INFO][15252] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Aug 13 01:31:29.193985 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.193985 containerd[1567]: 2025-08-13 01:31:29.108 [INFO][15252] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" iface="eth0" netns="/var/run/netns/cni-17733ef9-2c78-6f42-9fa2-14438920e33f" Aug 13 01:31:29.193985 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.193985 containerd[1567]: 2025-08-13 01:31:29.109 [INFO][15252] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" iface="eth0" netns="/var/run/netns/cni-17733ef9-2c78-6f42-9fa2-14438920e33f" Aug 13 01:31:29.193985 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.193985 containerd[1567]: 2025-08-13 01:31:29.116 [INFO][15252] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" after=7.818416ms iface="eth0" netns="/var/run/netns/cni-17733ef9-2c78-6f42-9fa2-14438920e33f" Aug 13 01:31:29.193985 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.193985 containerd[1567]: 2025-08-13 01:31:29.116 [INFO][15252] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Aug 13 01:31:29.193985 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.193985 containerd[1567]: 2025-08-13 01:31:29.116 [INFO][15252] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Aug 13 01:31:29.193985 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.193985 containerd[1567]: 2025-08-13 01:31:29.148 [INFO][15264] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" HandleID="k8s-pod-network.39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:29.193985 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.193985 containerd[1567]: 2025-08-13 01:31:29.148 [INFO][15264] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:29.193985 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.193985 containerd[1567]: 2025-08-13 01:31:29.149 [INFO][15264] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:29.193985 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.193985 containerd[1567]: 2025-08-13 01:31:29.188 [INFO][15264] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" HandleID="k8s-pod-network.39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:29.193985 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.193985 containerd[1567]: 2025-08-13 01:31:29.188 [INFO][15264] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" HandleID="k8s-pod-network.39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:29.193985 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.193985 containerd[1567]: 2025-08-13 01:31:29.189 [INFO][15264] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:29.193985 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.193985 containerd[1567]: 2025-08-13 01:31:29.191 [INFO][15252] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b" Aug 13 01:31:29.193985 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:29.197510 containerd[1567]: time="2025-08-13T01:31:29.195682449Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b-lo: no space left on device" Aug 13 01:31:29.197575 kubelet[2720]: E0813 01:31:29.196977 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b-lo: no space left on device" Aug 13 01:31:29.197575 kubelet[2720]: E0813 01:31:29.197028 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:29.197575 kubelet[2720]: E0813 01:31:29.197050 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:29.197575 kubelet[2720]: E0813 01:31:29.197098 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:29.197766 systemd[1]: run-netns-cni\x2d17733ef9\x2d2c78\x2d6f42\x2d9fa2\x2d14438920e33f.mount: Deactivated successfully. Aug 13 01:31:29.926019 containerd[1567]: time="2025-08-13T01:31:29.925968015Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:30.037067 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:30.038544 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:29.965 [INFO][15272] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21912 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:29.965 [INFO][15272] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:29.991 [INFO][15285] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" HandleID="k8s-pod-network.268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:29.991 [INFO][15285] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" HandleID="k8s-pod-network.268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5920), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:29.991381265 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:29.991 [INFO][15285] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:29.991 [INFO][15285] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:29.991 [INFO][15285] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:30.004 [INFO][15285] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" host="172-234-25-215" Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:30.009 [INFO][15285] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:30.014 [INFO][15285] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:30.016 [INFO][15285] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:30.018 [INFO][15285] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:30.018 [INFO][15285] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" host="172-234-25-215" Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:30.020 [INFO][15285] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:30.023 [INFO][15285] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" host="172-234-25-215" Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:30.030 [INFO][15285] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.37/26] block=192.168.62.0/26 handle="k8s-pod-network.268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" host="172-234-25-215" Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.055743 containerd[1567]: 2025-08-13 01:31:30.030 [INFO][15285] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.37/26] handle="k8s-pod-network.268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" host="172-234-25-215" Aug 13 01:31:30.055743 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.057837 containerd[1567]: 2025-08-13 01:31:30.030 [INFO][15285] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:30.057837 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.057837 containerd[1567]: 2025-08-13 01:31:30.030 [INFO][15285] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.37/26] IPv6=[] ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" HandleID="k8s-pod-network.268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:30.057837 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.057837 containerd[1567]: 2025-08-13 01:31:30.033 [INFO][15272] cni-plugin/k8s.go 418: Populated endpoint ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21912", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"39d26023b8af875b6ec3de8086c0357102ef81b843abe872988c7920ba60458b", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.37/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:30.057837 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.057837 containerd[1567]: 2025-08-13 01:31:30.033 [INFO][15272] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.37/32] ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:30.057837 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.057837 containerd[1567]: 2025-08-13 01:31:30.033 [INFO][15272] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:30.057837 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.057837 containerd[1567]: 2025-08-13 01:31:30.039 [INFO][15272] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:30.057837 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.057837 containerd[1567]: 2025-08-13 01:31:30.040 [INFO][15272] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21912", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.37/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"4a:63:39:e2:91:53", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:30.057837 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.058151 containerd[1567]: 2025-08-13 01:31:30.048 [INFO][15272] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:30.058151 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.151333 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:30.151600 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:30.172828 containerd[1567]: time="2025-08-13T01:31:30.172777150Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-typha-5bf7f956b7-szw5v_d0c18708-6c9b-42ce-bbc4-7fe0748e1776/calico-typha/0.log\"" error="write /var/log/pods/calico-system_calico-typha-5bf7f956b7-szw5v_d0c18708-6c9b-42ce-bbc4-7fe0748e1776/calico-typha/0.log: no space left on device" Aug 13 01:31:30.300889 containerd[1567]: 2025-08-13 01:31:30.146 [INFO][15307] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Aug 13 01:31:30.300889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.300889 containerd[1567]: 2025-08-13 01:31:30.147 [INFO][15307] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" iface="eth0" netns="/var/run/netns/cni-38d371cb-b182-e479-f7a2-ff8bfe1b4149" Aug 13 01:31:30.300889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.300889 containerd[1567]: 2025-08-13 01:31:30.148 [INFO][15307] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" iface="eth0" netns="/var/run/netns/cni-38d371cb-b182-e479-f7a2-ff8bfe1b4149" Aug 13 01:31:30.300889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.300889 containerd[1567]: 2025-08-13 01:31:30.158 [INFO][15307] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" after=10.93507ms iface="eth0" netns="/var/run/netns/cni-38d371cb-b182-e479-f7a2-ff8bfe1b4149" Aug 13 01:31:30.300889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.300889 containerd[1567]: 2025-08-13 01:31:30.158 [INFO][15307] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Aug 13 01:31:30.300889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.300889 containerd[1567]: 2025-08-13 01:31:30.158 [INFO][15307] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Aug 13 01:31:30.300889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.300889 containerd[1567]: 2025-08-13 01:31:30.206 [INFO][15317] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" HandleID="k8s-pod-network.268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:30.300889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.300889 containerd[1567]: 2025-08-13 01:31:30.207 [INFO][15317] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:30.300889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.300889 containerd[1567]: 2025-08-13 01:31:30.207 [INFO][15317] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:30.300889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.300889 containerd[1567]: 2025-08-13 01:31:30.291 [INFO][15317] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" HandleID="k8s-pod-network.268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:30.300889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.300889 containerd[1567]: 2025-08-13 01:31:30.292 [INFO][15317] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" HandleID="k8s-pod-network.268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:30.300889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.300889 containerd[1567]: 2025-08-13 01:31:30.293 [INFO][15317] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:30.300889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.300889 containerd[1567]: 2025-08-13 01:31:30.296 [INFO][15307] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f" Aug 13 01:31:30.300889 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:30.304935 systemd[1]: run-netns-cni\x2d38d371cb\x2db182\x2de479\x2df7a2\x2dff8bfe1b4149.mount: Deactivated successfully. Aug 13 01:31:30.306284 containerd[1567]: time="2025-08-13T01:31:30.305556836Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f-lo: no space left on device" Aug 13 01:31:30.307261 kubelet[2720]: E0813 01:31:30.307193 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f-lo: no space left on device" Aug 13 01:31:30.307261 kubelet[2720]: E0813 01:31:30.307255 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:30.308959 kubelet[2720]: E0813 01:31:30.307276 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:30.308959 kubelet[2720]: E0813 01:31:30.307329 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:30.369010 containerd[1567]: time="2025-08-13T01:31:30.368955399Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:31:30.928505 containerd[1567]: time="2025-08-13T01:31:30.928147391Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:31.038805 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:31.040982 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:30.967 [INFO][15328] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 21988 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:30.967 [INFO][15328] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:30.993 [INFO][15341] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" HandleID="k8s-pod-network.7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:30.993 [INFO][15341] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" HandleID="k8s-pod-network.7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d56a0), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:30.99348572 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:30.993 [INFO][15341] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:30.993 [INFO][15341] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:30.994 [INFO][15341] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:31.002 [INFO][15341] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" host="172-234-25-215" Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:31.007 [INFO][15341] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:31.012 [INFO][15341] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:31.014 [INFO][15341] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:31.017 [INFO][15341] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:31.017 [INFO][15341] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" host="172-234-25-215" Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:31.018 [INFO][15341] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6 Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:31.023 [INFO][15341] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" host="172-234-25-215" Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:31.030 [INFO][15341] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.38/26] block=192.168.62.0/26 handle="k8s-pod-network.7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" host="172-234-25-215" Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.066294 containerd[1567]: 2025-08-13 01:31:31.030 [INFO][15341] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.38/26] handle="k8s-pod-network.7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" host="172-234-25-215" Aug 13 01:31:31.066294 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.067524 containerd[1567]: 2025-08-13 01:31:31.030 [INFO][15341] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:31.067524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.067524 containerd[1567]: 2025-08-13 01:31:31.030 [INFO][15341] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.38/26] IPv6=[] ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" HandleID="k8s-pod-network.7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:31.067524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.067524 containerd[1567]: 2025-08-13 01:31:31.033 [INFO][15328] cni-plugin/k8s.go 418: Populated endpoint ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21988", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"268e2fa2eea2d21c480b6d100e3897a9b831d9293da6cb997731f39972e1ed9f", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.38/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:31.067524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.067524 containerd[1567]: 2025-08-13 01:31:31.033 [INFO][15328] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.38/32] ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:31.067524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.067524 containerd[1567]: 2025-08-13 01:31:31.033 [INFO][15328] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:31.067524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.067524 containerd[1567]: 2025-08-13 01:31:31.042 [INFO][15328] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:31.067524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.067524 containerd[1567]: 2025-08-13 01:31:31.042 [INFO][15328] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"21988", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.38/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"82:90:5e:e8:d7:d7", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:31.067524 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.067895 containerd[1567]: 2025-08-13 01:31:31.058 [INFO][15328] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:31.067895 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.139691 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:31.139703 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:31.224182 containerd[1567]: 2025-08-13 01:31:31.135 [INFO][15363] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Aug 13 01:31:31.224182 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.224182 containerd[1567]: 2025-08-13 01:31:31.136 [INFO][15363] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" iface="eth0" netns="/var/run/netns/cni-a3970eed-64b6-f51f-56bf-51374d906fc7" Aug 13 01:31:31.224182 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.224182 containerd[1567]: 2025-08-13 01:31:31.137 [INFO][15363] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" iface="eth0" netns="/var/run/netns/cni-a3970eed-64b6-f51f-56bf-51374d906fc7" Aug 13 01:31:31.224182 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.224182 containerd[1567]: 2025-08-13 01:31:31.144 [INFO][15363] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" after=7.803155ms iface="eth0" netns="/var/run/netns/cni-a3970eed-64b6-f51f-56bf-51374d906fc7" Aug 13 01:31:31.224182 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.224182 containerd[1567]: 2025-08-13 01:31:31.144 [INFO][15363] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Aug 13 01:31:31.224182 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.224182 containerd[1567]: 2025-08-13 01:31:31.144 [INFO][15363] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Aug 13 01:31:31.224182 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.224182 containerd[1567]: 2025-08-13 01:31:31.176 [INFO][15373] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" HandleID="k8s-pod-network.7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:31.224182 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.224182 containerd[1567]: 2025-08-13 01:31:31.178 [INFO][15373] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:31.224182 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.224182 containerd[1567]: 2025-08-13 01:31:31.178 [INFO][15373] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:31.224182 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.224182 containerd[1567]: 2025-08-13 01:31:31.216 [INFO][15373] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" HandleID="k8s-pod-network.7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:31.224182 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.224182 containerd[1567]: 2025-08-13 01:31:31.216 [INFO][15373] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" HandleID="k8s-pod-network.7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:31.224182 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.224182 containerd[1567]: 2025-08-13 01:31:31.218 [INFO][15373] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:31.224182 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.224182 containerd[1567]: 2025-08-13 01:31:31.221 [INFO][15363] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6" Aug 13 01:31:31.224182 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:31.226857 containerd[1567]: time="2025-08-13T01:31:31.226804561Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6-lo: no space left on device" Aug 13 01:31:31.227758 kubelet[2720]: E0813 01:31:31.227050 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6-lo: no space left on device" Aug 13 01:31:31.227952 kubelet[2720]: E0813 01:31:31.227876 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:31.227952 kubelet[2720]: E0813 01:31:31.227905 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:31.228077 kubelet[2720]: E0813 01:31:31.228040 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:31.228497 systemd[1]: run-netns-cni\x2da3970eed\x2d64b6\x2df51f\x2d56bf\x2d51374d906fc7.mount: Deactivated successfully. Aug 13 01:31:31.929299 containerd[1567]: time="2025-08-13T01:31:31.929051418Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:32.024898 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:32.026507 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:31.964 [INFO][15382] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 22061 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:31.965 [INFO][15382] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:31.985 [INFO][15395] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" HandleID="k8s-pod-network.eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:31.985 [INFO][15395] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" HandleID="k8s-pod-network.eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f020), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:31.985731004 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:31.985 [INFO][15395] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:31.986 [INFO][15395] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:31.986 [INFO][15395] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:31.991 [INFO][15395] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" host="172-234-25-215" Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:31.995 [INFO][15395] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:32.001 [INFO][15395] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:32.003 [INFO][15395] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:32.006 [INFO][15395] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:32.006 [INFO][15395] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" host="172-234-25-215" Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:32.009 [INFO][15395] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:32.012 [INFO][15395] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" host="172-234-25-215" Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:32.018 [INFO][15395] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.39/26] block=192.168.62.0/26 handle="k8s-pod-network.eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" host="172-234-25-215" Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042054 containerd[1567]: 2025-08-13 01:31:32.018 [INFO][15395] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.39/26] handle="k8s-pod-network.eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" host="172-234-25-215" Aug 13 01:31:32.042054 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042762 containerd[1567]: 2025-08-13 01:31:32.018 [INFO][15395] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:32.042762 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042762 containerd[1567]: 2025-08-13 01:31:32.018 [INFO][15395] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.39/26] IPv6=[] ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" HandleID="k8s-pod-network.eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:32.042762 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042762 containerd[1567]: 2025-08-13 01:31:32.020 [INFO][15382] cni-plugin/k8s.go 418: Populated endpoint ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"22061", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"7d011f24d5524fc326be9cfba8dce5a566468943a7f75fd1dcc15dc09d8668a6", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.39/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:32.042762 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042762 containerd[1567]: 2025-08-13 01:31:32.021 [INFO][15382] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.39/32] ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:32.042762 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042762 containerd[1567]: 2025-08-13 01:31:32.021 [INFO][15382] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:32.042762 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042762 containerd[1567]: 2025-08-13 01:31:32.026 [INFO][15382] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:32.042762 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.042762 containerd[1567]: 2025-08-13 01:31:32.028 [INFO][15382] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"22061", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.39/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"46:dc:75:f5:86:7f", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:32.042762 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.043057 containerd[1567]: 2025-08-13 01:31:32.036 [INFO][15382] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:32.043057 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.116697 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:32.116706 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:32.239590 containerd[1567]: 2025-08-13 01:31:32.108 [INFO][15418] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Aug 13 01:31:32.239590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.239590 containerd[1567]: 2025-08-13 01:31:32.109 [INFO][15418] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" iface="eth0" netns="/var/run/netns/cni-e017266e-088f-bb37-77c9-5899fe8a9d4a" Aug 13 01:31:32.239590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.239590 containerd[1567]: 2025-08-13 01:31:32.110 [INFO][15418] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" iface="eth0" netns="/var/run/netns/cni-e017266e-088f-bb37-77c9-5899fe8a9d4a" Aug 13 01:31:32.239590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.239590 containerd[1567]: 2025-08-13 01:31:32.124 [INFO][15418] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" after=14.468893ms iface="eth0" netns="/var/run/netns/cni-e017266e-088f-bb37-77c9-5899fe8a9d4a" Aug 13 01:31:32.239590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.239590 containerd[1567]: 2025-08-13 01:31:32.124 [INFO][15418] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Aug 13 01:31:32.239590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.239590 containerd[1567]: 2025-08-13 01:31:32.124 [INFO][15418] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Aug 13 01:31:32.239590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.239590 containerd[1567]: 2025-08-13 01:31:32.180 [INFO][15427] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" HandleID="k8s-pod-network.eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:32.239590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.239590 containerd[1567]: 2025-08-13 01:31:32.182 [INFO][15427] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:32.239590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.239590 containerd[1567]: 2025-08-13 01:31:32.182 [INFO][15427] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:32.239590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.239590 containerd[1567]: 2025-08-13 01:31:32.232 [INFO][15427] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" HandleID="k8s-pod-network.eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:32.239590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.239590 containerd[1567]: 2025-08-13 01:31:32.233 [INFO][15427] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" HandleID="k8s-pod-network.eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:32.239590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.239590 containerd[1567]: 2025-08-13 01:31:32.234 [INFO][15427] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:32.239590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.239590 containerd[1567]: 2025-08-13 01:31:32.237 [INFO][15418] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf" Aug 13 01:31:32.239590 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:32.243078 systemd[1]: run-netns-cni\x2de017266e\x2d088f\x2dbb37\x2d77c9\x2d5899fe8a9d4a.mount: Deactivated successfully. Aug 13 01:31:32.243275 containerd[1567]: time="2025-08-13T01:31:32.243227840Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf-lo: no space left on device" Aug 13 01:31:32.243789 kubelet[2720]: E0813 01:31:32.243389 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf-lo: no space left on device" Aug 13 01:31:32.243789 kubelet[2720]: E0813 01:31:32.243635 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:32.243789 kubelet[2720]: E0813 01:31:32.243653 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:32.243789 kubelet[2720]: E0813 01:31:32.243696 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:32.932155 containerd[1567]: time="2025-08-13T01:31:32.932112482Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:33.074968 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:33.075701 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:32.981 [INFO][15437] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 22129 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:32.981 [INFO][15437] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.019 [INFO][15450] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" HandleID="k8s-pod-network.48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.019 [INFO][15450] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" HandleID="k8s-pod-network.48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f100), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:33.019397162 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.019 [INFO][15450] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.019 [INFO][15450] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.019 [INFO][15450] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.028 [INFO][15450] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" host="172-234-25-215" Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.033 [INFO][15450] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.041 [INFO][15450] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.043 [INFO][15450] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.047 [INFO][15450] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.048 [INFO][15450] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" host="172-234-25-215" Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.050 [INFO][15450] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020 Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.056 [INFO][15450] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" host="172-234-25-215" Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.068 [INFO][15450] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.40/26] block=192.168.62.0/26 handle="k8s-pod-network.48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" host="172-234-25-215" Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.096659 containerd[1567]: 2025-08-13 01:31:33.068 [INFO][15450] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.40/26] handle="k8s-pod-network.48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" host="172-234-25-215" Aug 13 01:31:33.096659 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.097589 containerd[1567]: 2025-08-13 01:31:33.068 [INFO][15450] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:33.097589 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.097589 containerd[1567]: 2025-08-13 01:31:33.068 [INFO][15450] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.40/26] IPv6=[] ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" HandleID="k8s-pod-network.48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:33.097589 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.097589 containerd[1567]: 2025-08-13 01:31:33.071 [INFO][15437] cni-plugin/k8s.go 418: Populated endpoint ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"22129", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"eaf063e96ce1283c592f9d6cd45c2f3cc0bdb44df3c319b7069e74489d72bcdf", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.40/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:33.097589 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.097589 containerd[1567]: 2025-08-13 01:31:33.071 [INFO][15437] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.40/32] ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:33.097589 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.097589 containerd[1567]: 2025-08-13 01:31:33.071 [INFO][15437] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:33.097589 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.097589 containerd[1567]: 2025-08-13 01:31:33.075 [INFO][15437] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:33.097589 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.097589 containerd[1567]: 2025-08-13 01:31:33.076 [INFO][15437] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"22129", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.40/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ae:53:9c:b8:8a:31", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:33.097589 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.097911 containerd[1567]: 2025-08-13 01:31:33.087 [INFO][15437] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:33.097911 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.164749 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:33.164759 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:33.249530 containerd[1567]: 2025-08-13 01:31:33.159 [INFO][15475] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Aug 13 01:31:33.249530 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.249530 containerd[1567]: 2025-08-13 01:31:33.160 [INFO][15475] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" iface="eth0" netns="/var/run/netns/cni-0c7074bf-7220-ce9a-820d-656cc4b04216" Aug 13 01:31:33.249530 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.249530 containerd[1567]: 2025-08-13 01:31:33.160 [INFO][15475] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" iface="eth0" netns="/var/run/netns/cni-0c7074bf-7220-ce9a-820d-656cc4b04216" Aug 13 01:31:33.249530 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.249530 containerd[1567]: 2025-08-13 01:31:33.171 [INFO][15475] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" after=10.71142ms iface="eth0" netns="/var/run/netns/cni-0c7074bf-7220-ce9a-820d-656cc4b04216" Aug 13 01:31:33.249530 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.249530 containerd[1567]: 2025-08-13 01:31:33.171 [INFO][15475] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Aug 13 01:31:33.249530 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.249530 containerd[1567]: 2025-08-13 01:31:33.171 [INFO][15475] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Aug 13 01:31:33.249530 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.249530 containerd[1567]: 2025-08-13 01:31:33.210 [INFO][15484] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" HandleID="k8s-pod-network.48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:33.249530 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.249530 containerd[1567]: 2025-08-13 01:31:33.211 [INFO][15484] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:33.249530 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.249530 containerd[1567]: 2025-08-13 01:31:33.211 [INFO][15484] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:33.249530 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.249530 containerd[1567]: 2025-08-13 01:31:33.242 [INFO][15484] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" HandleID="k8s-pod-network.48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:33.249530 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.249530 containerd[1567]: 2025-08-13 01:31:33.242 [INFO][15484] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" HandleID="k8s-pod-network.48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:33.249530 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.249530 containerd[1567]: 2025-08-13 01:31:33.244 [INFO][15484] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:33.249530 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.249530 containerd[1567]: 2025-08-13 01:31:33.247 [INFO][15475] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020" Aug 13 01:31:33.249530 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:33.252150 containerd[1567]: time="2025-08-13T01:31:33.251707394Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020-lo: no space left on device" Aug 13 01:31:33.252613 kubelet[2720]: E0813 01:31:33.252326 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020-lo: no space left on device" Aug 13 01:31:33.252613 kubelet[2720]: E0813 01:31:33.252380 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:33.252613 kubelet[2720]: E0813 01:31:33.252399 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:33.252613 kubelet[2720]: E0813 01:31:33.252442 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:33.254203 systemd[1]: run-netns-cni\x2d0c7074bf\x2d7220\x2dce9a\x2d820d\x2d656cc4b04216.mount: Deactivated successfully. Aug 13 01:31:33.371145 containerd[1567]: time="2025-08-13T01:31:33.371083674Z" level=error msg="Fail to write \"stderr\" log to log file \"/var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log\"" error="write /var/log/pods/calico-system_calico-node-2p8hj_4f334296-e8c0-427e-a090-d8fbd3582f26/calico-node/0.log: no space left on device" Aug 13 01:31:33.933896 containerd[1567]: time="2025-08-13T01:31:33.933850239Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:34.044489 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:34.048451 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:33.973 [INFO][15495] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 22194 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:33.973 [INFO][15495] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:33.997 [INFO][15508] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" HandleID="k8s-pod-network.e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:33.997 [INFO][15508] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" HandleID="k8s-pod-network.e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc00024f050), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:33.997287822 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:33.997 [INFO][15508] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:33.997 [INFO][15508] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:33.997 [INFO][15508] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:34.003 [INFO][15508] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" host="172-234-25-215" Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:34.007 [INFO][15508] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:34.012 [INFO][15508] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:34.014 [INFO][15508] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:34.016 [INFO][15508] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:34.018 [INFO][15508] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" host="172-234-25-215" Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:34.020 [INFO][15508] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291 Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:34.024 [INFO][15508] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" host="172-234-25-215" Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:34.033 [INFO][15508] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.41/26] block=192.168.62.0/26 handle="k8s-pod-network.e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" host="172-234-25-215" Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.074416 containerd[1567]: 2025-08-13 01:31:34.033 [INFO][15508] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.41/26] handle="k8s-pod-network.e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" host="172-234-25-215" Aug 13 01:31:34.074416 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.077553 containerd[1567]: 2025-08-13 01:31:34.033 [INFO][15508] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:34.077553 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.077553 containerd[1567]: 2025-08-13 01:31:34.033 [INFO][15508] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.41/26] IPv6=[] ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" HandleID="k8s-pod-network.e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:34.077553 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.077553 containerd[1567]: 2025-08-13 01:31:34.037 [INFO][15495] cni-plugin/k8s.go 418: Populated endpoint ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"22194", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"48d8e138d2d07389094a9d08e0b14b5cc396e66042eb4618b89e4efedb136020", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.41/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:34.077553 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.077553 containerd[1567]: 2025-08-13 01:31:34.037 [INFO][15495] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.41/32] ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:34.077553 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.077553 containerd[1567]: 2025-08-13 01:31:34.037 [INFO][15495] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:34.077553 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.077553 containerd[1567]: 2025-08-13 01:31:34.047 [INFO][15495] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:34.077553 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.077553 containerd[1567]: 2025-08-13 01:31:34.049 [INFO][15495] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"22194", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.41/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"ae:29:80:67:9e:47", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:34.077553 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.078348 containerd[1567]: 2025-08-13 01:31:34.067 [INFO][15495] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:34.078348 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.139473 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:34.139483 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:34.246786 containerd[1567]: 2025-08-13 01:31:34.136 [INFO][15531] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Aug 13 01:31:34.246786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.246786 containerd[1567]: 2025-08-13 01:31:34.136 [INFO][15531] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" iface="eth0" netns="/var/run/netns/cni-a6fd9e47-3b3d-bfcf-d598-4b30686d2ae5" Aug 13 01:31:34.246786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.246786 containerd[1567]: 2025-08-13 01:31:34.137 [INFO][15531] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" iface="eth0" netns="/var/run/netns/cni-a6fd9e47-3b3d-bfcf-d598-4b30686d2ae5" Aug 13 01:31:34.246786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.246786 containerd[1567]: 2025-08-13 01:31:34.147 [INFO][15531] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" after=11.113389ms iface="eth0" netns="/var/run/netns/cni-a6fd9e47-3b3d-bfcf-d598-4b30686d2ae5" Aug 13 01:31:34.246786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.246786 containerd[1567]: 2025-08-13 01:31:34.147 [INFO][15531] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Aug 13 01:31:34.246786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.246786 containerd[1567]: 2025-08-13 01:31:34.148 [INFO][15531] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Aug 13 01:31:34.246786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.246786 containerd[1567]: 2025-08-13 01:31:34.181 [INFO][15540] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" HandleID="k8s-pod-network.e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:34.246786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.246786 containerd[1567]: 2025-08-13 01:31:34.181 [INFO][15540] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:34.246786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.246786 containerd[1567]: 2025-08-13 01:31:34.181 [INFO][15540] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:34.246786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.246786 containerd[1567]: 2025-08-13 01:31:34.240 [INFO][15540] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" HandleID="k8s-pod-network.e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:34.246786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.246786 containerd[1567]: 2025-08-13 01:31:34.240 [INFO][15540] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" HandleID="k8s-pod-network.e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:34.246786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.246786 containerd[1567]: 2025-08-13 01:31:34.241 [INFO][15540] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:34.246786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.246786 containerd[1567]: 2025-08-13 01:31:34.243 [INFO][15531] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291" Aug 13 01:31:34.246786 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:34.251605 containerd[1567]: time="2025-08-13T01:31:34.251495974Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291-lo: no space left on device" Aug 13 01:31:34.252602 kubelet[2720]: E0813 01:31:34.252211 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291-lo: no space left on device" Aug 13 01:31:34.252602 kubelet[2720]: E0813 01:31:34.252275 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:34.252602 kubelet[2720]: E0813 01:31:34.252295 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:34.252602 kubelet[2720]: E0813 01:31:34.252353 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:34.252272 systemd[1]: run-netns-cni\x2da6fd9e47\x2d3b3d\x2dbfcf\x2dd598\x2d4b30686d2ae5.mount: Deactivated successfully. Aug 13 01:31:34.935490 containerd[1567]: time="2025-08-13T01:31:34.935400347Z" level=info msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,}" Aug 13 01:31:35.039928 systemd-networkd[1449]: cali84aa0ce63ef: Link UP Aug 13 01:31:35.041487 systemd-networkd[1449]: cali84aa0ce63ef: Gained carrier Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:34.974 [INFO][15552] cni-plugin/plugin.go 340: Calico CNI found existing endpoint: &{{WorkloadEndpoint projectcalico.org/v3} {172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0 calico-kube-controllers-b57d47f84- calico-system ffb60df5-70ce-4ebe-ad70-a3fd8de49793 22259 0 2025-08-13 01:25:39 +0000 UTC map[app.kubernetes.io/name:calico-kube-controllers k8s-app:calico-kube-controllers pod-template-hash:b57d47f84 projectcalico.org/namespace:calico-system projectcalico.org/orchestrator:k8s projectcalico.org/serviceaccount:calico-kube-controllers] map[] [] [] []} {k8s 172-234-25-215 e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291 calico-kube-controllers-b57d47f84-lznkx eth0 calico-kube-controllers [] [] [kns.calico-system ksa.calico-system.calico-kube-controllers] cali84aa0ce63ef [] [] }} ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-" Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:34.974 [INFO][15552] cni-plugin/k8s.go 74: Extracted identifiers for CmdAddK8s ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.000 [INFO][15562] ipam/ipam_plugin.go 225: Calico CNI IPAM request count IPv4=1 IPv6=0 ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" HandleID="k8s-pod-network.38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.000 [INFO][15562] ipam/ipam_plugin.go 265: Auto assigning IP ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" HandleID="k8s-pod-network.38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" assignArgs=ipam.AutoAssignArgs{Num4:1, Num6:0, HandleID:(*string)(0xc0002d5650), Attrs:map[string]string{"namespace":"calico-system", "node":"172-234-25-215", "pod":"calico-kube-controllers-b57d47f84-lznkx", "timestamp":"2025-08-13 01:31:35.000404776 +0000 UTC"}, Hostname:"172-234-25-215", IPv4Pools:[]net.IPNet{}, IPv6Pools:[]net.IPNet{}, MaxBlocksPerHost:0, HostReservedAttrIPv4s:(*ipam.HostReservedAttr)(nil), HostReservedAttrIPv6s:(*ipam.HostReservedAttr)(nil), IntendedUse:"Workload"} Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.000 [INFO][15562] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.000 [INFO][15562] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.000 [INFO][15562] ipam/ipam.go 110: Auto-assign 1 ipv4, 0 ipv6 addrs for host '172-234-25-215' Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.006 [INFO][15562] ipam/ipam.go 691: Looking up existing affinities for host handle="k8s-pod-network.38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" host="172-234-25-215" Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.010 [INFO][15562] ipam/ipam.go 394: Looking up existing affinities for host host="172-234-25-215" Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.014 [INFO][15562] ipam/ipam.go 511: Trying affinity for 192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.016 [INFO][15562] ipam/ipam.go 158: Attempting to load block cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.021 [INFO][15562] ipam/ipam.go 235: Affinity is confirmed and block has been loaded cidr=192.168.62.0/26 host="172-234-25-215" Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.021 [INFO][15562] ipam/ipam.go 1220: Attempting to assign 1 addresses from block block=192.168.62.0/26 handle="k8s-pod-network.38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" host="172-234-25-215" Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.023 [INFO][15562] ipam/ipam.go 1764: Creating new handle: k8s-pod-network.38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.027 [INFO][15562] ipam/ipam.go 1243: Writing block in order to claim IPs block=192.168.62.0/26 handle="k8s-pod-network.38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" host="172-234-25-215" Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.032 [INFO][15562] ipam/ipam.go 1256: Successfully claimed IPs: [192.168.62.42/26] block=192.168.62.0/26 handle="k8s-pod-network.38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" host="172-234-25-215" Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.062869 containerd[1567]: 2025-08-13 01:31:35.032 [INFO][15562] ipam/ipam.go 878: Auto-assigned 1 out of 1 IPv4s: [192.168.62.42/26] handle="k8s-pod-network.38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" host="172-234-25-215" Aug 13 01:31:35.062869 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.063560 containerd[1567]: 2025-08-13 01:31:35.032 [INFO][15562] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:35.063560 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.063560 containerd[1567]: 2025-08-13 01:31:35.032 [INFO][15562] ipam/ipam_plugin.go 283: Calico CNI IPAM assigned addresses IPv4=[192.168.62.42/26] IPv6=[] ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" HandleID="k8s-pod-network.38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:35.063560 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.063560 containerd[1567]: 2025-08-13 01:31:35.035 [INFO][15552] cni-plugin/k8s.go 418: Populated endpoint ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"22259", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"e0fb1aaeb61481457cab8c31a0cd45b6b5997cd4cfc433fee0acd30b99b57291", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.42/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:35.063560 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.063560 containerd[1567]: 2025-08-13 01:31:35.035 [INFO][15552] cni-plugin/k8s.go 419: Calico CNI using IPs: [192.168.62.42/32] ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:35.063560 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.063560 containerd[1567]: 2025-08-13 01:31:35.035 [INFO][15552] cni-plugin/dataplane_linux.go 69: Setting the host side veth name to cali84aa0ce63ef ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:35.063560 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.063560 containerd[1567]: 2025-08-13 01:31:35.042 [INFO][15552] cni-plugin/dataplane_linux.go 508: Disabling IPv4 forwarding ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:35.063560 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.063560 containerd[1567]: 2025-08-13 01:31:35.042 [INFO][15552] cni-plugin/k8s.go 446: Added Mac, interface name, and active container ID to endpoint ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" endpoint=&v3.WorkloadEndpoint{TypeMeta:v1.TypeMeta{Kind:"WorkloadEndpoint", APIVersion:"projectcalico.org/v3"}, ObjectMeta:v1.ObjectMeta{Name:"172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0", GenerateName:"calico-kube-controllers-b57d47f84-", Namespace:"calico-system", SelfLink:"", UID:"ffb60df5-70ce-4ebe-ad70-a3fd8de49793", ResourceVersion:"22259", Generation:0, CreationTimestamp:time.Date(2025, time.August, 13, 1, 25, 39, 0, time.Local), DeletionTimestamp:, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/name":"calico-kube-controllers", "k8s-app":"calico-kube-controllers", "pod-template-hash":"b57d47f84", "projectcalico.org/namespace":"calico-system", "projectcalico.org/orchestrator":"k8s", "projectcalico.org/serviceaccount":"calico-kube-controllers"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v3.WorkloadEndpointSpec{Orchestrator:"k8s", Workload:"", Node:"172-234-25-215", ContainerID:"38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e", Pod:"calico-kube-controllers-b57d47f84-lznkx", Endpoint:"eth0", ServiceAccountName:"calico-kube-controllers", IPNetworks:[]string{"192.168.62.42/32"}, IPNATs:[]v3.IPNAT(nil), IPv4Gateway:"", IPv6Gateway:"", Profiles:[]string{"kns.calico-system", "ksa.calico-system.calico-kube-controllers"}, InterfaceName:"cali84aa0ce63ef", MAC:"be:3e:f0:bd:11:5f", Ports:[]v3.WorkloadEndpointPort(nil), AllowSpoofedSourcePrefixes:[]string(nil), QoSControls:(*v3.QoSControls)(nil)}} Aug 13 01:31:35.063560 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.063893 containerd[1567]: 2025-08-13 01:31:35.054 [INFO][15552] cni-plugin/k8s.go 532: Wrote updated endpoint to datastore ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Namespace="calico-system" Pod="calico-kube-controllers-b57d47f84-lznkx" WorkloadEndpoint="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:35.063893 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.074428 kubelet[2720]: E0813 01:31:35.074376 2720 dns.go:153] "Nameserver limits exceeded" err="Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 172.232.0.9 172.232.0.19 172.232.0.20" Aug 13 01:31:35.075946 kubelet[2720]: E0813 01:31:35.075844 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"coredns\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": ErrImagePull: failed to pull and unpack image \\\"registry.k8s.io/coredns/coredns:v1.12.0\\\": failed to prepare extraction snapshot \\\"extract-349524588-hiNR sha256:0c46b3b43cce082ce5ec34ffc6c294e0417d17b9d1e6f366d114b6aadc4f96dc\\\": failed to create prepare snapshot dir: failed to create temp dir: mkdir /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-2074644924: no space left on device\"" pod="kube-system/coredns-674b8bbfcf-jlpx4" podUID="e3a3eaf4-0523-46b9-8d19-1303011f36f8" Aug 13 01:31:35.116308 systemd-networkd[1449]: cali84aa0ce63ef: Link DOWN Aug 13 01:31:35.116316 systemd-networkd[1449]: cali84aa0ce63ef: Lost carrier Aug 13 01:31:35.196207 containerd[1567]: 2025-08-13 01:31:35.114 [INFO][15586] cni-plugin/k8s.go 640: Cleaning up netns ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Aug 13 01:31:35.196207 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.196207 containerd[1567]: 2025-08-13 01:31:35.115 [INFO][15586] cni-plugin/dataplane_linux.go 559: Deleting workload's device in netns. ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" iface="eth0" netns="/var/run/netns/cni-5dfc33ae-e5cb-d09a-7435-bb22bee19096" Aug 13 01:31:35.196207 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.196207 containerd[1567]: 2025-08-13 01:31:35.115 [INFO][15586] cni-plugin/dataplane_linux.go 570: Entered netns, deleting veth. ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" iface="eth0" netns="/var/run/netns/cni-5dfc33ae-e5cb-d09a-7435-bb22bee19096" Aug 13 01:31:35.196207 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.196207 containerd[1567]: 2025-08-13 01:31:35.122 [INFO][15586] cni-plugin/dataplane_linux.go 604: Deleted device in netns. ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" after=6.774447ms iface="eth0" netns="/var/run/netns/cni-5dfc33ae-e5cb-d09a-7435-bb22bee19096" Aug 13 01:31:35.196207 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.196207 containerd[1567]: 2025-08-13 01:31:35.122 [INFO][15586] cni-plugin/k8s.go 647: Releasing IP address(es) ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Aug 13 01:31:35.196207 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.196207 containerd[1567]: 2025-08-13 01:31:35.122 [INFO][15586] cni-plugin/utils.go 188: Calico CNI releasing IP address ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Aug 13 01:31:35.196207 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.196207 containerd[1567]: 2025-08-13 01:31:35.161 [INFO][15593] ipam/ipam_plugin.go 412: Releasing address using handleID ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" HandleID="k8s-pod-network.38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:35.196207 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.196207 containerd[1567]: 2025-08-13 01:31:35.161 [INFO][15593] ipam/ipam_plugin.go 353: About to acquire host-wide IPAM lock. Aug 13 01:31:35.196207 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.196207 containerd[1567]: 2025-08-13 01:31:35.161 [INFO][15593] ipam/ipam_plugin.go 368: Acquired host-wide IPAM lock. Aug 13 01:31:35.196207 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.196207 containerd[1567]: 2025-08-13 01:31:35.189 [INFO][15593] ipam/ipam_plugin.go 431: Released address using handleID ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" HandleID="k8s-pod-network.38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:35.196207 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.196207 containerd[1567]: 2025-08-13 01:31:35.189 [INFO][15593] ipam/ipam_plugin.go 440: Releasing address using workloadID ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" HandleID="k8s-pod-network.38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Workload="172--234--25--215-k8s-calico--kube--controllers--b57d47f84--lznkx-eth0" Aug 13 01:31:35.196207 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.196207 containerd[1567]: 2025-08-13 01:31:35.191 [INFO][15593] ipam/ipam_plugin.go 374: Released host-wide IPAM lock. Aug 13 01:31:35.196207 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.196207 containerd[1567]: 2025-08-13 01:31:35.193 [INFO][15586] cni-plugin/k8s.go 653: Teardown processing complete. ContainerID="38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e" Aug 13 01:31:35.196207 containerd[1567]: Failed to write to log, write /var/log/calico/cni/cni.log: no space left on device Aug 13 01:31:35.200379 containerd[1567]: time="2025-08-13T01:31:35.200295179Z" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:calico-kube-controllers-b57d47f84-lznkx,Uid:ffb60df5-70ce-4ebe-ad70-a3fd8de49793,Namespace:calico-system,Attempt:0,} failed, error" error="rpc error: code = Unknown desc = failed to setup network for sandbox \"38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e-lo: no space left on device" Aug 13 01:31:35.200801 kubelet[2720]: E0813 01:31:35.200751 2720 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e-lo: no space left on device" Aug 13 01:31:35.200870 kubelet[2720]: E0813 01:31:35.200834 2720 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:35.200870 kubelet[2720]: E0813 01:31:35.200856 2720 kuberuntime_manager.go:1252] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to setup network for sandbox \"38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e\": failed to set network \"cni-loopback\" cached result: write /var/lib/cni/results/cni-loopback-38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e-lo: no space left on device" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" Aug 13 01:31:35.200971 kubelet[2720]: E0813 01:31:35.200936 2720 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"calico-kube-controllers-b57d47f84-lznkx_calico-system(ffb60df5-70ce-4ebe-ad70-a3fd8de49793)\\\": rpc error: code = Unknown desc = failed to setup network for sandbox \\\"38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e\\\": failed to set network \\\"cni-loopback\\\" cached result: write /var/lib/cni/results/cni-loopback-38fe69f51badfafb09a5447424c66d7e291bdf2d089c38179b9cc0404f6a526e-lo: no space left on device\"" pod="calico-system/calico-kube-controllers-b57d47f84-lznkx" podUID="ffb60df5-70ce-4ebe-ad70-a3fd8de49793" Aug 13 01:31:35.201670 systemd[1]: run-netns-cni\x2d5dfc33ae\x2de5cb\x2dd09a\x2d7435\x2dbb22bee19096.mount: Deactivated successfully.