Jul 15 10:53:43.068019 kernel: Linux version 6.6.98-flatcar (build@pony-truck.infra.kinvolk.io) (x86_64-cros-linux-gnu-gcc (Gentoo Hardened 14.2.1_p20241221 p7) 14.2.1 20241221, GNU ld (Gentoo 2.43 p3) 2.43.1) #1 SMP PREEMPT_DYNAMIC Tue Jul 15 09:10:16 -00 2025 Jul 15 10:53:43.068047 kernel: Command line: BOOT_IMAGE=/flatcar/vmlinuz-a mount.usr=/dev/mapper/usr verity.usr=PARTUUID=7130c94a-213a-4e5a-8e26-6cce9662f132 rootflags=rw mount.usrflags=ro consoleblank=0 root=LABEL=ROOT console=ttyS0,115200n8 console=tty0 flatcar.first_boot=detected flatcar.oem.id=openstack flatcar.autologin verity.usrhash=5af33b6d30f308631e4a91e3c355c3478a77b3d8f9ea9ec9a25f78d7867f681f Jul 15 10:53:43.068058 kernel: BIOS-provided physical RAM map: Jul 15 10:53:43.068065 kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable Jul 15 10:53:43.068073 kernel: BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved Jul 15 10:53:43.068082 kernel: BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved Jul 15 10:53:43.068091 kernel: BIOS-e820: [mem 0x0000000000100000-0x00000000bffdcfff] usable Jul 15 10:53:43.068099 kernel: BIOS-e820: [mem 0x00000000bffdd000-0x00000000bfffffff] reserved Jul 15 10:53:43.068107 kernel: BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved Jul 15 10:53:43.068114 kernel: BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved Jul 15 10:53:43.068122 kernel: BIOS-e820: [mem 0x0000000100000000-0x000000013fffffff] usable Jul 15 10:53:43.068129 kernel: BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved Jul 15 10:53:43.068137 kernel: NX (Execute Disable) protection: active Jul 15 10:53:43.068145 kernel: APIC: Static calls initialized Jul 15 10:53:43.068155 kernel: SMBIOS 3.0.0 present. Jul 15 10:53:43.068164 kernel: DMI: OpenStack Foundation OpenStack Nova, BIOS 1.16.3-debian-1.16.3-2 04/01/2014 Jul 15 10:53:43.068172 kernel: Hypervisor detected: KVM Jul 15 10:53:43.068180 kernel: kvm-clock: Using msrs 4b564d01 and 4b564d00 Jul 15 10:53:43.068188 kernel: kvm-clock: using sched offset of 3562944427 cycles Jul 15 10:53:43.068198 kernel: clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns Jul 15 10:53:43.068207 kernel: tsc: Detected 1996.249 MHz processor Jul 15 10:53:43.068215 kernel: e820: update [mem 0x00000000-0x00000fff] usable ==> reserved Jul 15 10:53:43.068224 kernel: e820: remove [mem 0x000a0000-0x000fffff] usable Jul 15 10:53:43.068232 kernel: last_pfn = 0x140000 max_arch_pfn = 0x400000000 Jul 15 10:53:43.068241 kernel: MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs Jul 15 10:53:43.068249 kernel: x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT Jul 15 10:53:43.068257 kernel: last_pfn = 0xbffdd max_arch_pfn = 0x400000000 Jul 15 10:53:43.068266 kernel: ACPI: Early table checksum verification disabled Jul 15 10:53:43.068276 kernel: ACPI: RSDP 0x00000000000F51E0 000014 (v00 BOCHS ) Jul 15 10:53:43.068284 kernel: ACPI: RSDT 0x00000000BFFE1B65 000030 (v01 BOCHS BXPC 00000001 BXPC 00000001) Jul 15 10:53:43.068292 kernel: ACPI: FACP 0x00000000BFFE1A49 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) Jul 15 10:53:43.068301 kernel: ACPI: DSDT 0x00000000BFFE0040 001A09 (v01 BOCHS BXPC 00000001 BXPC 00000001) Jul 15 10:53:43.068309 kernel: ACPI: FACS 0x00000000BFFE0000 000040 Jul 15 10:53:43.068318 kernel: ACPI: APIC 0x00000000BFFE1ABD 000080 (v03 BOCHS BXPC 00000001 BXPC 00000001) Jul 15 10:53:43.068326 kernel: ACPI: WAET 0x00000000BFFE1B3D 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) Jul 15 10:53:43.068334 kernel: ACPI: Reserving FACP table memory at [mem 0xbffe1a49-0xbffe1abc] Jul 15 10:53:43.068343 kernel: ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe1a48] Jul 15 10:53:43.068353 kernel: ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] Jul 15 10:53:43.068361 kernel: ACPI: Reserving APIC table memory at [mem 0xbffe1abd-0xbffe1b3c] Jul 15 10:53:43.068369 kernel: ACPI: Reserving WAET table memory at [mem 0xbffe1b3d-0xbffe1b64] Jul 15 10:53:43.068403 kernel: No NUMA configuration found Jul 15 10:53:43.068412 kernel: Faking a node at [mem 0x0000000000000000-0x000000013fffffff] Jul 15 10:53:43.068421 kernel: NODE_DATA(0) allocated [mem 0x13fffa000-0x13fffffff] Jul 15 10:53:43.068432 kernel: Zone ranges: Jul 15 10:53:43.068441 kernel: DMA [mem 0x0000000000001000-0x0000000000ffffff] Jul 15 10:53:43.068449 kernel: DMA32 [mem 0x0000000001000000-0x00000000ffffffff] Jul 15 10:53:43.068458 kernel: Normal [mem 0x0000000100000000-0x000000013fffffff] Jul 15 10:53:43.068466 kernel: Movable zone start for each node Jul 15 10:53:43.068475 kernel: Early memory node ranges Jul 15 10:53:43.068483 kernel: node 0: [mem 0x0000000000001000-0x000000000009efff] Jul 15 10:53:43.068492 kernel: node 0: [mem 0x0000000000100000-0x00000000bffdcfff] Jul 15 10:53:43.068503 kernel: node 0: [mem 0x0000000100000000-0x000000013fffffff] Jul 15 10:53:43.068512 kernel: Initmem setup node 0 [mem 0x0000000000001000-0x000000013fffffff] Jul 15 10:53:43.068520 kernel: On node 0, zone DMA: 1 pages in unavailable ranges Jul 15 10:53:43.068529 kernel: On node 0, zone DMA: 97 pages in unavailable ranges Jul 15 10:53:43.068538 kernel: On node 0, zone Normal: 35 pages in unavailable ranges Jul 15 10:53:43.068546 kernel: ACPI: PM-Timer IO Port: 0x608 Jul 15 10:53:43.068555 kernel: ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) Jul 15 10:53:43.068564 kernel: IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 Jul 15 10:53:43.068572 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) Jul 15 10:53:43.068583 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) Jul 15 10:53:43.068592 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) Jul 15 10:53:43.068600 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) Jul 15 10:53:43.068609 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) Jul 15 10:53:43.068618 kernel: ACPI: Using ACPI (MADT) for SMP configuration information Jul 15 10:53:43.068626 kernel: smpboot: Allowing 2 CPUs, 0 hotplug CPUs Jul 15 10:53:43.068635 kernel: kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write() Jul 15 10:53:43.068644 kernel: [mem 0xc0000000-0xfeffbfff] available for PCI devices Jul 15 10:53:43.068652 kernel: Booting paravirtualized kernel on KVM Jul 15 10:53:43.068663 kernel: clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns Jul 15 10:53:43.068672 kernel: setup_percpu: NR_CPUS:512 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1 Jul 15 10:53:43.068680 kernel: percpu: Embedded 58 pages/cpu s197096 r8192 d32280 u1048576 Jul 15 10:53:43.068689 kernel: pcpu-alloc: s197096 r8192 d32280 u1048576 alloc=1*2097152 Jul 15 10:53:43.068697 kernel: pcpu-alloc: [0] 0 1 Jul 15 10:53:43.068706 kernel: kvm-guest: PV spinlocks disabled, no host support Jul 15 10:53:43.068716 kernel: Kernel command line: rootflags=rw mount.usrflags=ro BOOT_IMAGE=/flatcar/vmlinuz-a mount.usr=/dev/mapper/usr verity.usr=PARTUUID=7130c94a-213a-4e5a-8e26-6cce9662f132 rootflags=rw mount.usrflags=ro consoleblank=0 root=LABEL=ROOT console=ttyS0,115200n8 console=tty0 flatcar.first_boot=detected flatcar.oem.id=openstack flatcar.autologin verity.usrhash=5af33b6d30f308631e4a91e3c355c3478a77b3d8f9ea9ec9a25f78d7867f681f Jul 15 10:53:43.070459 kernel: Unknown kernel command line parameters "BOOT_IMAGE=/flatcar/vmlinuz-a", will be passed to user space. Jul 15 10:53:43.070473 kernel: Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) Jul 15 10:53:43.070483 kernel: Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Jul 15 10:53:43.070491 kernel: Fallback order for Node 0: 0 Jul 15 10:53:43.070500 kernel: Built 1 zonelists, mobility grouping on. Total pages: 1031901 Jul 15 10:53:43.070509 kernel: Policy zone: Normal Jul 15 10:53:43.070518 kernel: mem auto-init: stack:off, heap alloc:off, heap free:off Jul 15 10:53:43.070527 kernel: software IO TLB: area num 2. Jul 15 10:53:43.070536 kernel: Memory: 3964168K/4193772K available (14336K kernel code, 2295K rwdata, 22872K rodata, 43504K init, 1572K bss, 229344K reserved, 0K cma-reserved) Jul 15 10:53:43.070545 kernel: SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 Jul 15 10:53:43.070556 kernel: ftrace: allocating 37944 entries in 149 pages Jul 15 10:53:43.070564 kernel: ftrace: allocated 149 pages with 4 groups Jul 15 10:53:43.070573 kernel: Dynamic Preempt: voluntary Jul 15 10:53:43.070582 kernel: rcu: Preemptible hierarchical RCU implementation. Jul 15 10:53:43.070591 kernel: rcu: RCU event tracing is enabled. Jul 15 10:53:43.070600 kernel: rcu: RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=2. Jul 15 10:53:43.070609 kernel: Trampoline variant of Tasks RCU enabled. Jul 15 10:53:43.070618 kernel: Rude variant of Tasks RCU enabled. Jul 15 10:53:43.070627 kernel: Tracing variant of Tasks RCU enabled. Jul 15 10:53:43.070638 kernel: rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. Jul 15 10:53:43.070647 kernel: rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 Jul 15 10:53:43.070655 kernel: NR_IRQS: 33024, nr_irqs: 440, preallocated irqs: 16 Jul 15 10:53:43.070665 kernel: rcu: srcu_init: Setting srcu_struct sizes based on contention. Jul 15 10:53:43.070673 kernel: Console: colour VGA+ 80x25 Jul 15 10:53:43.070682 kernel: printk: console [tty0] enabled Jul 15 10:53:43.070691 kernel: printk: console [ttyS0] enabled Jul 15 10:53:43.070699 kernel: ACPI: Core revision 20230628 Jul 15 10:53:43.070708 kernel: APIC: Switch to symmetric I/O mode setup Jul 15 10:53:43.070717 kernel: x2apic enabled Jul 15 10:53:43.070728 kernel: APIC: Switched APIC routing to: physical x2apic Jul 15 10:53:43.070737 kernel: ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 Jul 15 10:53:43.070746 kernel: tsc: Marking TSC unstable due to TSCs unsynchronized Jul 15 10:53:43.070755 kernel: Calibrating delay loop (skipped) preset value.. 3992.49 BogoMIPS (lpj=1996249) Jul 15 10:53:43.070763 kernel: Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 Jul 15 10:53:43.070772 kernel: Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 Jul 15 10:53:43.070781 kernel: Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization Jul 15 10:53:43.070790 kernel: Spectre V2 : Mitigation: Retpolines Jul 15 10:53:43.070799 kernel: Spectre V2 : Spectre v2 / SpectreRSB: Filling RSB on context switch and VMEXIT Jul 15 10:53:43.070809 kernel: Speculative Store Bypass: Vulnerable Jul 15 10:53:43.070818 kernel: x86/fpu: x87 FPU will use FXSAVE Jul 15 10:53:43.070827 kernel: Freeing SMP alternatives memory: 32K Jul 15 10:53:43.070836 kernel: pid_max: default: 32768 minimum: 301 Jul 15 10:53:43.070851 kernel: LSM: initializing lsm=lockdown,capability,landlock,selinux,integrity Jul 15 10:53:43.070862 kernel: landlock: Up and running. Jul 15 10:53:43.070871 kernel: SELinux: Initializing. Jul 15 10:53:43.070881 kernel: Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear) Jul 15 10:53:43.070890 kernel: Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear) Jul 15 10:53:43.070899 kernel: smpboot: CPU0: AMD Intel Core i7 9xx (Nehalem Class Core i7) (family: 0x6, model: 0x1a, stepping: 0x3) Jul 15 10:53:43.070908 kernel: RCU Tasks: Setting shift to 1 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=2. Jul 15 10:53:43.070920 kernel: RCU Tasks Rude: Setting shift to 1 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=2. Jul 15 10:53:43.070929 kernel: RCU Tasks Trace: Setting shift to 1 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=2. Jul 15 10:53:43.070938 kernel: Performance Events: AMD PMU driver. Jul 15 10:53:43.070947 kernel: ... version: 0 Jul 15 10:53:43.070957 kernel: ... bit width: 48 Jul 15 10:53:43.070967 kernel: ... generic registers: 4 Jul 15 10:53:43.070976 kernel: ... value mask: 0000ffffffffffff Jul 15 10:53:43.070985 kernel: ... max period: 00007fffffffffff Jul 15 10:53:43.070995 kernel: ... fixed-purpose events: 0 Jul 15 10:53:43.071004 kernel: ... event mask: 000000000000000f Jul 15 10:53:43.071013 kernel: signal: max sigframe size: 1440 Jul 15 10:53:43.071022 kernel: rcu: Hierarchical SRCU implementation. Jul 15 10:53:43.071031 kernel: rcu: Max phase no-delay instances is 400. Jul 15 10:53:43.071040 kernel: smp: Bringing up secondary CPUs ... Jul 15 10:53:43.071051 kernel: smpboot: x86: Booting SMP configuration: Jul 15 10:53:43.071060 kernel: .... node #0, CPUs: #1 Jul 15 10:53:43.071069 kernel: smp: Brought up 1 node, 2 CPUs Jul 15 10:53:43.071078 kernel: smpboot: Max logical packages: 2 Jul 15 10:53:43.071087 kernel: smpboot: Total of 2 processors activated (7984.99 BogoMIPS) Jul 15 10:53:43.071097 kernel: devtmpfs: initialized Jul 15 10:53:43.071106 kernel: x86/mm: Memory block size: 128MB Jul 15 10:53:43.071115 kernel: clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns Jul 15 10:53:43.071124 kernel: futex hash table entries: 512 (order: 3, 32768 bytes, linear) Jul 15 10:53:43.071133 kernel: pinctrl core: initialized pinctrl subsystem Jul 15 10:53:43.071144 kernel: NET: Registered PF_NETLINK/PF_ROUTE protocol family Jul 15 10:53:43.071154 kernel: audit: initializing netlink subsys (disabled) Jul 15 10:53:43.071163 kernel: audit: type=2000 audit(1752576821.900:1): state=initialized audit_enabled=0 res=1 Jul 15 10:53:43.071172 kernel: thermal_sys: Registered thermal governor 'step_wise' Jul 15 10:53:43.071181 kernel: thermal_sys: Registered thermal governor 'user_space' Jul 15 10:53:43.071190 kernel: cpuidle: using governor menu Jul 15 10:53:43.071200 kernel: acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 Jul 15 10:53:43.071209 kernel: dca service started, version 1.12.1 Jul 15 10:53:43.071218 kernel: PCI: Using configuration type 1 for base access Jul 15 10:53:43.071229 kernel: kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. Jul 15 10:53:43.071238 kernel: HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages Jul 15 10:53:43.071247 kernel: HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page Jul 15 10:53:43.071256 kernel: ACPI: Added _OSI(Module Device) Jul 15 10:53:43.071265 kernel: ACPI: Added _OSI(Processor Device) Jul 15 10:53:43.071275 kernel: ACPI: Added _OSI(Processor Aggregator Device) Jul 15 10:53:43.071284 kernel: ACPI: 1 ACPI AML tables successfully acquired and loaded Jul 15 10:53:43.071293 kernel: ACPI: _OSC evaluation for CPUs failed, trying _PDC Jul 15 10:53:43.071302 kernel: ACPI: Interpreter enabled Jul 15 10:53:43.071313 kernel: ACPI: PM: (supports S0 S3 S5) Jul 15 10:53:43.071322 kernel: ACPI: Using IOAPIC for interrupt routing Jul 15 10:53:43.071331 kernel: PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug Jul 15 10:53:43.071340 kernel: PCI: Using E820 reservations for host bridge windows Jul 15 10:53:43.071349 kernel: ACPI: Enabled 2 GPEs in block 00 to 0F Jul 15 10:53:43.071359 kernel: ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) Jul 15 10:53:43.071551 kernel: acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] Jul 15 10:53:43.071654 kernel: acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] Jul 15 10:53:43.071754 kernel: acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended configuration space under this bridge Jul 15 10:53:43.071768 kernel: acpiphp: Slot [3] registered Jul 15 10:53:43.071778 kernel: acpiphp: Slot [4] registered Jul 15 10:53:43.071787 kernel: acpiphp: Slot [5] registered Jul 15 10:53:43.071796 kernel: acpiphp: Slot [6] registered Jul 15 10:53:43.071805 kernel: acpiphp: Slot [7] registered Jul 15 10:53:43.071814 kernel: acpiphp: Slot [8] registered Jul 15 10:53:43.071823 kernel: acpiphp: Slot [9] registered Jul 15 10:53:43.071836 kernel: acpiphp: Slot [10] registered Jul 15 10:53:43.071845 kernel: acpiphp: Slot [11] registered Jul 15 10:53:43.071854 kernel: acpiphp: Slot [12] registered Jul 15 10:53:43.071863 kernel: acpiphp: Slot [13] registered Jul 15 10:53:43.071872 kernel: acpiphp: Slot [14] registered Jul 15 10:53:43.071881 kernel: acpiphp: Slot [15] registered Jul 15 10:53:43.071890 kernel: acpiphp: Slot [16] registered Jul 15 10:53:43.071899 kernel: acpiphp: Slot [17] registered Jul 15 10:53:43.071908 kernel: acpiphp: Slot [18] registered Jul 15 10:53:43.071919 kernel: acpiphp: Slot [19] registered Jul 15 10:53:43.071928 kernel: acpiphp: Slot [20] registered Jul 15 10:53:43.071937 kernel: acpiphp: Slot [21] registered Jul 15 10:53:43.071946 kernel: acpiphp: Slot [22] registered Jul 15 10:53:43.071955 kernel: acpiphp: Slot [23] registered Jul 15 10:53:43.071977 kernel: acpiphp: Slot [24] registered Jul 15 10:53:43.071986 kernel: acpiphp: Slot [25] registered Jul 15 10:53:43.071995 kernel: acpiphp: Slot [26] registered Jul 15 10:53:43.072004 kernel: acpiphp: Slot [27] registered Jul 15 10:53:43.072013 kernel: acpiphp: Slot [28] registered Jul 15 10:53:43.072025 kernel: acpiphp: Slot [29] registered Jul 15 10:53:43.072033 kernel: acpiphp: Slot [30] registered Jul 15 10:53:43.072043 kernel: acpiphp: Slot [31] registered Jul 15 10:53:43.072052 kernel: PCI host bridge to bus 0000:00 Jul 15 10:53:43.072246 kernel: pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] Jul 15 10:53:43.072368 kernel: pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] Jul 15 10:53:43.073559 kernel: pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] Jul 15 10:53:43.073644 kernel: pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] Jul 15 10:53:43.073732 kernel: pci_bus 0000:00: root bus resource [mem 0xc000000000-0xc07fffffff window] Jul 15 10:53:43.073813 kernel: pci_bus 0000:00: root bus resource [bus 00-ff] Jul 15 10:53:43.073929 kernel: pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 Jul 15 10:53:43.074044 kernel: pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 Jul 15 10:53:43.074148 kernel: pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 Jul 15 10:53:43.074242 kernel: pci 0000:00:01.1: reg 0x20: [io 0xc120-0xc12f] Jul 15 10:53:43.076431 kernel: pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] Jul 15 10:53:43.076543 kernel: pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] Jul 15 10:53:43.076641 kernel: pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] Jul 15 10:53:43.076735 kernel: pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] Jul 15 10:53:43.076840 kernel: pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 Jul 15 10:53:43.076935 kernel: pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI Jul 15 10:53:43.077029 kernel: pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB Jul 15 10:53:43.077139 kernel: pci 0000:00:02.0: [1af4:1050] type 00 class 0x030000 Jul 15 10:53:43.077238 kernel: pci 0000:00:02.0: reg 0x10: [mem 0xfe000000-0xfe7fffff pref] Jul 15 10:53:43.077332 kernel: pci 0000:00:02.0: reg 0x18: [mem 0xc000000000-0xc000003fff 64bit pref] Jul 15 10:53:43.078482 kernel: pci 0000:00:02.0: reg 0x20: [mem 0xfeb90000-0xfeb90fff] Jul 15 10:53:43.078585 kernel: pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfeb8ffff pref] Jul 15 10:53:43.078685 kernel: pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff] Jul 15 10:53:43.078796 kernel: pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000 Jul 15 10:53:43.078892 kernel: pci 0000:00:03.0: reg 0x10: [io 0xc080-0xc0bf] Jul 15 10:53:43.078986 kernel: pci 0000:00:03.0: reg 0x14: [mem 0xfeb91000-0xfeb91fff] Jul 15 10:53:43.079083 kernel: pci 0000:00:03.0: reg 0x20: [mem 0xc000004000-0xc000007fff 64bit pref] Jul 15 10:53:43.079177 kernel: pci 0000:00:03.0: reg 0x30: [mem 0xfeb00000-0xfeb7ffff pref] Jul 15 10:53:43.079280 kernel: pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 Jul 15 10:53:43.079375 kernel: pci 0000:00:04.0: reg 0x10: [io 0xc000-0xc07f] Jul 15 10:53:43.079565 kernel: pci 0000:00:04.0: reg 0x14: [mem 0xfeb92000-0xfeb92fff] Jul 15 10:53:43.079660 kernel: pci 0000:00:04.0: reg 0x20: [mem 0xc000008000-0xc00000bfff 64bit pref] Jul 15 10:53:43.079765 kernel: pci 0000:00:05.0: [1af4:1002] type 00 class 0x00ff00 Jul 15 10:53:43.079862 kernel: pci 0000:00:05.0: reg 0x10: [io 0xc0c0-0xc0ff] Jul 15 10:53:43.079956 kernel: pci 0000:00:05.0: reg 0x20: [mem 0xc00000c000-0xc00000ffff 64bit pref] Jul 15 10:53:43.080075 kernel: pci 0000:00:06.0: [1af4:1005] type 00 class 0x00ff00 Jul 15 10:53:43.080172 kernel: pci 0000:00:06.0: reg 0x10: [io 0xc100-0xc11f] Jul 15 10:53:43.080272 kernel: pci 0000:00:06.0: reg 0x14: [mem 0xfeb93000-0xfeb93fff] Jul 15 10:53:43.080367 kernel: pci 0000:00:06.0: reg 0x20: [mem 0xc000010000-0xc000013fff 64bit pref] Jul 15 10:53:43.082338 kernel: ACPI: PCI: Interrupt link LNKA configured for IRQ 10 Jul 15 10:53:43.082352 kernel: ACPI: PCI: Interrupt link LNKB configured for IRQ 10 Jul 15 10:53:43.082362 kernel: ACPI: PCI: Interrupt link LNKC configured for IRQ 11 Jul 15 10:53:43.082372 kernel: ACPI: PCI: Interrupt link LNKD configured for IRQ 11 Jul 15 10:53:43.082397 kernel: ACPI: PCI: Interrupt link LNKS configured for IRQ 9 Jul 15 10:53:43.082407 kernel: iommu: Default domain type: Translated Jul 15 10:53:43.084405 kernel: iommu: DMA domain TLB invalidation policy: lazy mode Jul 15 10:53:43.084424 kernel: PCI: Using ACPI for IRQ routing Jul 15 10:53:43.084434 kernel: PCI: pci_cache_line_size set to 64 bytes Jul 15 10:53:43.084444 kernel: e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff] Jul 15 10:53:43.084453 kernel: e820: reserve RAM buffer [mem 0xbffdd000-0xbfffffff] Jul 15 10:53:43.084573 kernel: pci 0000:00:02.0: vgaarb: setting as boot VGA device Jul 15 10:53:43.084672 kernel: pci 0000:00:02.0: vgaarb: bridge control possible Jul 15 10:53:43.084767 kernel: pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none Jul 15 10:53:43.084781 kernel: vgaarb: loaded Jul 15 10:53:43.084794 kernel: clocksource: Switched to clocksource kvm-clock Jul 15 10:53:43.084804 kernel: VFS: Disk quotas dquot_6.6.0 Jul 15 10:53:43.084813 kernel: VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) Jul 15 10:53:43.084822 kernel: pnp: PnP ACPI init Jul 15 10:53:43.084930 kernel: pnp 00:03: [dma 2] Jul 15 10:53:43.084946 kernel: pnp: PnP ACPI: found 5 devices Jul 15 10:53:43.084955 kernel: clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns Jul 15 10:53:43.084965 kernel: NET: Registered PF_INET protocol family Jul 15 10:53:43.084974 kernel: IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear) Jul 15 10:53:43.084987 kernel: tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear) Jul 15 10:53:43.084997 kernel: Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) Jul 15 10:53:43.085006 kernel: TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear) Jul 15 10:53:43.085016 kernel: TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear) Jul 15 10:53:43.085025 kernel: TCP: Hash tables configured (established 32768 bind 32768) Jul 15 10:53:43.085035 kernel: UDP hash table entries: 2048 (order: 4, 65536 bytes, linear) Jul 15 10:53:43.085044 kernel: UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear) Jul 15 10:53:43.085053 kernel: NET: Registered PF_UNIX/PF_LOCAL protocol family Jul 15 10:53:43.085064 kernel: NET: Registered PF_XDP protocol family Jul 15 10:53:43.085153 kernel: pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] Jul 15 10:53:43.085236 kernel: pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] Jul 15 10:53:43.085320 kernel: pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] Jul 15 10:53:43.085464 kernel: pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] Jul 15 10:53:43.085549 kernel: pci_bus 0000:00: resource 8 [mem 0xc000000000-0xc07fffffff window] Jul 15 10:53:43.085656 kernel: pci 0000:00:01.0: PIIX3: Enabling Passive Release Jul 15 10:53:43.085766 kernel: pci 0000:00:00.0: Limiting direct PCI/PCI transfers Jul 15 10:53:43.085786 kernel: PCI: CLS 0 bytes, default 64 Jul 15 10:53:43.085795 kernel: PCI-DMA: Using software bounce buffering for IO (SWIOTLB) Jul 15 10:53:43.085805 kernel: software IO TLB: mapped [mem 0x00000000bbfdd000-0x00000000bffdd000] (64MB) Jul 15 10:53:43.085814 kernel: Initialise system trusted keyrings Jul 15 10:53:43.085823 kernel: workingset: timestamp_bits=39 max_order=20 bucket_order=0 Jul 15 10:53:43.085833 kernel: Key type asymmetric registered Jul 15 10:53:43.085842 kernel: Asymmetric key parser 'x509' registered Jul 15 10:53:43.085851 kernel: Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251) Jul 15 10:53:43.085861 kernel: io scheduler mq-deadline registered Jul 15 10:53:43.085872 kernel: io scheduler kyber registered Jul 15 10:53:43.085882 kernel: io scheduler bfq registered Jul 15 10:53:43.085891 kernel: ioatdma: Intel(R) QuickData Technology Driver 5.00 Jul 15 10:53:43.085902 kernel: ACPI: \_SB_.LNKB: Enabled at IRQ 10 Jul 15 10:53:43.085911 kernel: ACPI: \_SB_.LNKC: Enabled at IRQ 11 Jul 15 10:53:43.085920 kernel: ACPI: \_SB_.LNKD: Enabled at IRQ 11 Jul 15 10:53:43.085930 kernel: ACPI: \_SB_.LNKA: Enabled at IRQ 10 Jul 15 10:53:43.085939 kernel: Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled Jul 15 10:53:43.085948 kernel: 00:00: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A Jul 15 10:53:43.085961 kernel: random: crng init done Jul 15 10:53:43.085970 kernel: i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 Jul 15 10:53:43.085980 kernel: serio: i8042 KBD port at 0x60,0x64 irq 1 Jul 15 10:53:43.085989 kernel: serio: i8042 AUX port at 0x60,0x64 irq 12 Jul 15 10:53:43.086081 kernel: rtc_cmos 00:04: RTC can wake from S4 Jul 15 10:53:43.086096 kernel: input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0 Jul 15 10:53:43.086178 kernel: rtc_cmos 00:04: registered as rtc0 Jul 15 10:53:43.086261 kernel: rtc_cmos 00:04: setting system clock to 2025-07-15T10:53:42 UTC (1752576822) Jul 15 10:53:43.086349 kernel: rtc_cmos 00:04: alarms up to one day, y3k, 242 bytes nvram Jul 15 10:53:43.086363 kernel: amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled Jul 15 10:53:43.086373 kernel: NET: Registered PF_INET6 protocol family Jul 15 10:53:43.087420 kernel: Segment Routing with IPv6 Jul 15 10:53:43.087432 kernel: In-situ OAM (IOAM) with IPv6 Jul 15 10:53:43.087442 kernel: NET: Registered PF_PACKET protocol family Jul 15 10:53:43.087451 kernel: Key type dns_resolver registered Jul 15 10:53:43.087461 kernel: IPI shorthand broadcast: enabled Jul 15 10:53:43.087470 kernel: sched_clock: Marking stable (1003007569, 168539228)->(1204433355, -32886558) Jul 15 10:53:43.087483 kernel: registered taskstats version 1 Jul 15 10:53:43.087493 kernel: Loading compiled-in X.509 certificates Jul 15 10:53:43.087503 kernel: Loaded X.509 cert 'Kinvolk GmbH: Module signing key for 6.6.98-flatcar: b24d4d02c910a1872551e9ac8f9af93a6b380d4d' Jul 15 10:53:43.087512 kernel: Key type .fscrypt registered Jul 15 10:53:43.087521 kernel: Key type fscrypt-provisioning registered Jul 15 10:53:43.087530 kernel: ima: No TPM chip found, activating TPM-bypass! Jul 15 10:53:43.087540 kernel: ima: Allocated hash algorithm: sha1 Jul 15 10:53:43.087549 kernel: ima: No architecture policies found Jul 15 10:53:43.087558 kernel: clk: Disabling unused clocks Jul 15 10:53:43.087570 kernel: Freeing unused kernel image (initmem) memory: 43504K Jul 15 10:53:43.087579 kernel: Write protecting the kernel read-only data: 38912k Jul 15 10:53:43.087588 kernel: Freeing unused kernel image (rodata/data gap) memory: 1704K Jul 15 10:53:43.087598 kernel: Run /init as init process Jul 15 10:53:43.087607 kernel: with arguments: Jul 15 10:53:43.087617 kernel: /init Jul 15 10:53:43.087625 kernel: with environment: Jul 15 10:53:43.087634 kernel: HOME=/ Jul 15 10:53:43.087643 kernel: TERM=linux Jul 15 10:53:43.087655 kernel: BOOT_IMAGE=/flatcar/vmlinuz-a Jul 15 10:53:43.087667 systemd[1]: Successfully made /usr/ read-only. Jul 15 10:53:43.087681 systemd[1]: systemd 256.8 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL -ACL +BLKID +CURL +ELFUTILS -FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 -PWQUALITY -P11KIT -QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP -SYSVINIT +LIBARCHIVE) Jul 15 10:53:43.087692 systemd[1]: Detected virtualization kvm. Jul 15 10:53:43.087702 systemd[1]: Detected architecture x86-64. Jul 15 10:53:43.087712 systemd[1]: Running in initrd. Jul 15 10:53:43.087722 systemd[1]: No hostname configured, using default hostname. Jul 15 10:53:43.087734 systemd[1]: Hostname set to . Jul 15 10:53:43.087744 systemd[1]: Initializing machine ID from VM UUID. Jul 15 10:53:43.087754 systemd[1]: Queued start job for default target initrd.target. Jul 15 10:53:43.087764 systemd[1]: Started clevis-luks-askpass.path - Forward Password Requests to Clevis Directory Watch. Jul 15 10:53:43.087775 systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch. Jul 15 10:53:43.087785 systemd[1]: Expecting device dev-disk-by\x2dlabel-EFI\x2dSYSTEM.device - /dev/disk/by-label/EFI-SYSTEM... Jul 15 10:53:43.087796 systemd[1]: Expecting device dev-disk-by\x2dlabel-OEM.device - /dev/disk/by-label/OEM... Jul 15 10:53:43.087815 systemd[1]: Expecting device dev-disk-by\x2dlabel-ROOT.device - /dev/disk/by-label/ROOT... Jul 15 10:53:43.087829 systemd[1]: Expecting device dev-disk-by\x2dpartlabel-USR\x2dA.device - /dev/disk/by-partlabel/USR-A... Jul 15 10:53:43.087840 systemd[1]: Expecting device dev-disk-by\x2dpartuuid-7130c94a\x2d213a\x2d4e5a\x2d8e26\x2d6cce9662f132.device - /dev/disk/by-partuuid/7130c94a-213a-4e5a-8e26-6cce9662f132... Jul 15 10:53:43.087850 systemd[1]: Expecting device dev-mapper-usr.device - /dev/mapper/usr... Jul 15 10:53:43.087861 systemd[1]: Reached target cryptsetup-pre.target - Local Encrypted Volumes (Pre). Jul 15 10:53:43.087873 systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes. Jul 15 10:53:43.087883 systemd[1]: Reached target paths.target - Path Units. Jul 15 10:53:43.087894 systemd[1]: Reached target slices.target - Slice Units. Jul 15 10:53:43.087904 systemd[1]: Reached target swap.target - Swaps. Jul 15 10:53:43.087914 systemd[1]: Reached target timers.target - Timer Units. Jul 15 10:53:43.087925 systemd[1]: Listening on iscsid.socket - Open-iSCSI iscsid Socket. Jul 15 10:53:43.087935 systemd[1]: Listening on iscsiuio.socket - Open-iSCSI iscsiuio Socket. Jul 15 10:53:43.087946 systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log). Jul 15 10:53:43.087956 systemd[1]: Listening on systemd-journald.socket - Journal Sockets. Jul 15 10:53:43.087981 systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket. Jul 15 10:53:43.087992 systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket. Jul 15 10:53:43.088002 systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket. Jul 15 10:53:43.088012 systemd[1]: Reached target sockets.target - Socket Units. Jul 15 10:53:43.088022 systemd[1]: Starting ignition-setup-pre.service - Ignition env setup... Jul 15 10:53:43.088032 systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes... Jul 15 10:53:43.088042 systemd[1]: Finished network-cleanup.service - Network Cleanup. Jul 15 10:53:43.088053 systemd[1]: Starting systemd-fsck-usr.service... Jul 15 10:53:43.088063 systemd[1]: Starting systemd-journald.service - Journal Service... Jul 15 10:53:43.088076 systemd[1]: Starting systemd-modules-load.service - Load Kernel Modules... Jul 15 10:53:43.088087 systemd[1]: Starting systemd-vconsole-setup.service - Virtual Console Setup... Jul 15 10:53:43.088097 systemd[1]: Finished ignition-setup-pre.service - Ignition env setup. Jul 15 10:53:43.088130 systemd-journald[185]: Collecting audit messages is disabled. Jul 15 10:53:43.088160 systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes. Jul 15 10:53:43.088171 systemd[1]: Finished systemd-fsck-usr.service. Jul 15 10:53:43.088182 systemd[1]: Starting systemd-tmpfiles-setup-dev-early.service - Create Static Device Nodes in /dev gracefully... Jul 15 10:53:43.088194 systemd-journald[185]: Journal started Jul 15 10:53:43.088220 systemd-journald[185]: Runtime Journal (/run/log/journal/360f0ca0f20744b8ad97daf7d2f8a5e0) is 8M, max 78.3M, 70.3M free. Jul 15 10:53:43.073178 systemd-modules-load[186]: Inserted module 'overlay' Jul 15 10:53:43.093452 systemd[1]: Started systemd-journald.service - Journal Service. Jul 15 10:53:43.094418 systemd[1]: Finished systemd-tmpfiles-setup-dev-early.service - Create Static Device Nodes in /dev gracefully. Jul 15 10:53:43.144052 kernel: bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this. Jul 15 10:53:43.144077 kernel: Bridge firewalling registered Jul 15 10:53:43.101568 systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev... Jul 15 10:53:43.116052 systemd-modules-load[186]: Inserted module 'br_netfilter' Jul 15 10:53:43.152522 systemd[1]: Starting systemd-tmpfiles-setup.service - Create System Files and Directories... Jul 15 10:53:43.153594 systemd[1]: Finished systemd-modules-load.service - Load Kernel Modules. Jul 15 10:53:43.157974 systemd[1]: Finished systemd-vconsole-setup.service - Virtual Console Setup. Jul 15 10:53:43.159476 systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev. Jul 15 10:53:43.165534 systemd[1]: Starting dracut-cmdline-ask.service - dracut ask for additional cmdline parameters... Jul 15 10:53:43.167449 systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables... Jul 15 10:53:43.169427 systemd[1]: Finished systemd-tmpfiles-setup.service - Create System Files and Directories. Jul 15 10:53:43.180347 systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables. Jul 15 10:53:43.185509 systemd[1]: Starting systemd-resolved.service - Network Name Resolution... Jul 15 10:53:43.186167 systemd[1]: Finished dracut-cmdline-ask.service - dracut ask for additional cmdline parameters. Jul 15 10:53:43.187507 systemd[1]: Starting dracut-cmdline.service - dracut cmdline hook... Jul 15 10:53:43.207400 dracut-cmdline[222]: dracut-dracut-053 Jul 15 10:53:43.207400 dracut-cmdline[222]: Using kernel command line parameters: rd.driver.pre=btrfs rootflags=rw mount.usrflags=ro BOOT_IMAGE=/flatcar/vmlinuz-a mount.usr=/dev/mapper/usr verity.usr=PARTUUID=7130c94a-213a-4e5a-8e26-6cce9662f132 rootflags=rw mount.usrflags=ro consoleblank=0 root=LABEL=ROOT console=ttyS0,115200n8 console=tty0 flatcar.first_boot=detected flatcar.oem.id=openstack flatcar.autologin verity.usrhash=5af33b6d30f308631e4a91e3c355c3478a77b3d8f9ea9ec9a25f78d7867f681f Jul 15 10:53:43.227148 systemd-resolved[221]: Positive Trust Anchors: Jul 15 10:53:43.227865 systemd-resolved[221]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d Jul 15 10:53:43.227910 systemd-resolved[221]: Negative trust anchors: home.arpa 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 170.0.0.192.in-addr.arpa 171.0.0.192.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa ipv4only.arpa resolver.arpa corp home internal intranet lan local private test Jul 15 10:53:43.231211 systemd-resolved[221]: Defaulting to hostname 'linux'. Jul 15 10:53:43.232244 systemd[1]: Started systemd-resolved.service - Network Name Resolution. Jul 15 10:53:43.234032 systemd[1]: Reached target nss-lookup.target - Host and Network Name Lookups. Jul 15 10:53:43.286408 kernel: SCSI subsystem initialized Jul 15 10:53:43.296404 kernel: Loading iSCSI transport class v2.0-870. Jul 15 10:53:43.307434 kernel: iscsi: registered transport (tcp) Jul 15 10:53:43.330006 kernel: iscsi: registered transport (qla4xxx) Jul 15 10:53:43.330068 kernel: QLogic iSCSI HBA Driver Jul 15 10:53:43.381908 systemd[1]: Finished dracut-cmdline.service - dracut cmdline hook. Jul 15 10:53:43.388608 systemd[1]: Starting dracut-pre-udev.service - dracut pre-udev hook... Jul 15 10:53:43.440637 kernel: device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. Jul 15 10:53:43.440761 kernel: device-mapper: uevent: version 1.0.3 Jul 15 10:53:43.444670 kernel: device-mapper: ioctl: 4.48.0-ioctl (2023-03-01) initialised: dm-devel@redhat.com Jul 15 10:53:43.513529 kernel: raid6: sse2x4 gen() 5178 MB/s Jul 15 10:53:43.531491 kernel: raid6: sse2x2 gen() 6057 MB/s Jul 15 10:53:43.549768 kernel: raid6: sse2x1 gen() 9758 MB/s Jul 15 10:53:43.549846 kernel: raid6: using algorithm sse2x1 gen() 9758 MB/s Jul 15 10:53:43.568882 kernel: raid6: .... xor() 7394 MB/s, rmw enabled Jul 15 10:53:43.568948 kernel: raid6: using ssse3x2 recovery algorithm Jul 15 10:53:43.591795 kernel: xor: measuring software checksum speed Jul 15 10:53:43.591858 kernel: prefetch64-sse : 18498 MB/sec Jul 15 10:53:43.593110 kernel: generic_sse : 16366 MB/sec Jul 15 10:53:43.593172 kernel: xor: using function: prefetch64-sse (18498 MB/sec) Jul 15 10:53:43.766459 kernel: Btrfs loaded, zoned=no, fsverity=no Jul 15 10:53:43.783822 systemd[1]: Finished dracut-pre-udev.service - dracut pre-udev hook. Jul 15 10:53:43.790701 systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files... Jul 15 10:53:43.839835 systemd-udevd[405]: Using default interface naming scheme 'v255'. Jul 15 10:53:43.850361 systemd[1]: Started systemd-udevd.service - Rule-based Manager for Device Events and Files. Jul 15 10:53:43.860607 systemd[1]: Starting dracut-pre-trigger.service - dracut pre-trigger hook... Jul 15 10:53:43.878256 dracut-pre-trigger[420]: rd.md=0: removing MD RAID activation Jul 15 10:53:43.920165 systemd[1]: Finished dracut-pre-trigger.service - dracut pre-trigger hook. Jul 15 10:53:43.927624 systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices... Jul 15 10:53:43.975943 systemd[1]: Finished systemd-udev-trigger.service - Coldplug All udev Devices. Jul 15 10:53:43.983549 systemd[1]: Starting dracut-initqueue.service - dracut initqueue hook... Jul 15 10:53:44.001892 systemd[1]: Finished dracut-initqueue.service - dracut initqueue hook. Jul 15 10:53:44.004049 systemd[1]: Reached target remote-fs-pre.target - Preparation for Remote File Systems. Jul 15 10:53:44.006451 systemd[1]: Reached target remote-cryptsetup.target - Remote Encrypted Volumes. Jul 15 10:53:44.006958 systemd[1]: Reached target remote-fs.target - Remote File Systems. Jul 15 10:53:44.014609 systemd[1]: Starting dracut-pre-mount.service - dracut pre-mount hook... Jul 15 10:53:44.034611 systemd[1]: Finished dracut-pre-mount.service - dracut pre-mount hook. Jul 15 10:53:44.077415 kernel: virtio_blk virtio2: 2/0/0 default/read/poll queues Jul 15 10:53:44.112291 kernel: virtio_blk virtio2: [vda] 20971520 512-byte logical blocks (10.7 GB/10.0 GiB) Jul 15 10:53:44.112758 systemd[1]: dracut-cmdline-ask.service: Deactivated successfully. Jul 15 10:53:44.112903 systemd[1]: Stopped dracut-cmdline-ask.service - dracut ask for additional cmdline parameters. Jul 15 10:53:44.114761 systemd[1]: Stopping dracut-cmdline-ask.service - dracut ask for additional cmdline parameters... Jul 15 10:53:44.115985 systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. Jul 15 10:53:44.127479 kernel: GPT:Primary header thinks Alt. header is not at the end of the disk. Jul 15 10:53:44.127511 kernel: GPT:17805311 != 20971519 Jul 15 10:53:44.127524 kernel: GPT:Alternate GPT header not at the end of the disk. Jul 15 10:53:44.127536 kernel: GPT:17805311 != 20971519 Jul 15 10:53:44.127547 kernel: GPT: Use GNU Parted to correct GPT errors. Jul 15 10:53:44.127559 kernel: vda: vda1 vda2 vda3 vda4 vda6 vda7 vda9 Jul 15 10:53:44.127570 kernel: libata version 3.00 loaded. Jul 15 10:53:44.116118 systemd[1]: Stopped systemd-vconsole-setup.service - Virtual Console Setup. Jul 15 10:53:44.118664 systemd[1]: Stopping systemd-vconsole-setup.service - Virtual Console Setup... Jul 15 10:53:44.132747 kernel: ata_piix 0000:00:01.1: version 2.13 Jul 15 10:53:44.133704 systemd[1]: Starting systemd-vconsole-setup.service - Virtual Console Setup... Jul 15 10:53:44.134632 systemd[1]: run-credentials-systemd\x2dvconsole\x2dsetup.service.mount: Deactivated successfully. Jul 15 10:53:44.138473 kernel: scsi host0: ata_piix Jul 15 10:53:44.138640 kernel: scsi host1: ata_piix Jul 15 10:53:44.139763 kernel: ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc120 irq 14 Jul 15 10:53:44.145422 kernel: ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc128 irq 15 Jul 15 10:53:44.174790 kernel: BTRFS: device label OEM devid 1 transid 11 /dev/vda6 scanned by (udev-worker) (481) Jul 15 10:53:44.185405 kernel: BTRFS: device fsid 72bb124d-7a3e-423d-a96c-4fa37107854d devid 1 transid 36 /dev/vda3 scanned by (udev-worker) (457) Jul 15 10:53:44.212742 systemd[1]: Found device dev-disk-by\x2dlabel-EFI\x2dSYSTEM.device - /dev/disk/by-label/EFI-SYSTEM. Jul 15 10:53:44.218026 systemd[1]: Finished systemd-vconsole-setup.service - Virtual Console Setup. Jul 15 10:53:44.229662 systemd[1]: Found device dev-disk-by\x2dlabel-ROOT.device - /dev/disk/by-label/ROOT. Jul 15 10:53:44.238521 systemd[1]: Found device dev-disk-by\x2dpartlabel-USR\x2dA.device - /dev/disk/by-partlabel/USR-A. Jul 15 10:53:44.239093 systemd[1]: Found device dev-disk-by\x2dpartuuid-7130c94a\x2d213a\x2d4e5a\x2d8e26\x2d6cce9662f132.device - /dev/disk/by-partuuid/7130c94a-213a-4e5a-8e26-6cce9662f132. Jul 15 10:53:44.250850 systemd[1]: Found device dev-disk-by\x2dlabel-OEM.device - /dev/disk/by-label/OEM. Jul 15 10:53:44.260533 systemd[1]: Starting disk-uuid.service - Generate new UUID for disk GPT if necessary... Jul 15 10:53:44.263205 systemd[1]: Starting dracut-cmdline-ask.service - dracut ask for additional cmdline parameters... Jul 15 10:53:44.275224 disk-uuid[509]: Primary Header is updated. Jul 15 10:53:44.275224 disk-uuid[509]: Secondary Entries is updated. Jul 15 10:53:44.275224 disk-uuid[509]: Secondary Header is updated. Jul 15 10:53:44.280596 systemd[1]: Finished dracut-cmdline-ask.service - dracut ask for additional cmdline parameters. Jul 15 10:53:44.285540 kernel: vda: vda1 vda2 vda3 vda4 vda6 vda7 vda9 Jul 15 10:53:45.301476 kernel: vda: vda1 vda2 vda3 vda4 vda6 vda7 vda9 Jul 15 10:53:45.303840 disk-uuid[517]: The operation has completed successfully. Jul 15 10:53:45.388196 systemd[1]: disk-uuid.service: Deactivated successfully. Jul 15 10:53:45.388372 systemd[1]: Finished disk-uuid.service - Generate new UUID for disk GPT if necessary. Jul 15 10:53:45.444520 systemd[1]: Starting verity-setup.service - Verity Setup for /dev/mapper/usr... Jul 15 10:53:45.448814 sh[530]: Success Jul 15 10:53:45.474429 kernel: device-mapper: verity: sha256 using implementation "sha256-ssse3" Jul 15 10:53:45.582053 systemd[1]: Found device dev-mapper-usr.device - /dev/mapper/usr. Jul 15 10:53:45.588574 systemd[1]: Mounting sysusr-usr.mount - /sysusr/usr... Jul 15 10:53:45.594514 systemd[1]: Finished verity-setup.service - Verity Setup for /dev/mapper/usr. Jul 15 10:53:45.620514 kernel: BTRFS info (device dm-0): first mount of filesystem 72bb124d-7a3e-423d-a96c-4fa37107854d Jul 15 10:53:45.620585 kernel: BTRFS info (device dm-0): using crc32c (crc32c-intel) checksum algorithm Jul 15 10:53:45.622571 kernel: BTRFS warning (device dm-0): 'nologreplay' is deprecated, use 'rescue=nologreplay' instead Jul 15 10:53:45.624675 kernel: BTRFS info (device dm-0): disabling log replay at mount time Jul 15 10:53:45.627602 kernel: BTRFS info (device dm-0): using free space tree Jul 15 10:53:45.643767 systemd[1]: Mounted sysusr-usr.mount - /sysusr/usr. Jul 15 10:53:45.645422 systemd[1]: afterburn-network-kargs.service - Afterburn Initrd Setup Network Kernel Arguments was skipped because no trigger condition checks were met. Jul 15 10:53:45.650635 systemd[1]: Starting ignition-setup.service - Ignition (setup)... Jul 15 10:53:45.654596 systemd[1]: Starting parse-ip-for-networkd.service - Write systemd-networkd units from cmdline... Jul 15 10:53:45.672157 kernel: BTRFS info (device vda6): first mount of filesystem c20e5e0c-dae6-45ce-95fe-16a1362e2f71 Jul 15 10:53:45.672214 kernel: BTRFS info (device vda6): using crc32c (crc32c-intel) checksum algorithm Jul 15 10:53:45.672229 kernel: BTRFS info (device vda6): using free space tree Jul 15 10:53:45.679414 kernel: BTRFS info (device vda6): auto enabling async discard Jul 15 10:53:45.686412 kernel: BTRFS info (device vda6): last unmount of filesystem c20e5e0c-dae6-45ce-95fe-16a1362e2f71 Jul 15 10:53:45.695727 systemd[1]: Finished ignition-setup.service - Ignition (setup). Jul 15 10:53:45.702569 systemd[1]: Starting ignition-fetch-offline.service - Ignition (fetch-offline)... Jul 15 10:53:45.823162 systemd[1]: Finished parse-ip-for-networkd.service - Write systemd-networkd units from cmdline. Jul 15 10:53:45.832709 systemd[1]: Starting systemd-networkd.service - Network Configuration... Jul 15 10:53:45.875069 ignition[613]: Ignition 2.20.0 Jul 15 10:53:45.875080 ignition[613]: Stage: fetch-offline Jul 15 10:53:45.875120 ignition[613]: no configs at "/usr/lib/ignition/base.d" Jul 15 10:53:45.875130 ignition[613]: no config dir at "/usr/lib/ignition/base.platform.d/openstack" Jul 15 10:53:45.875233 ignition[613]: parsed url from cmdline: "" Jul 15 10:53:45.878850 systemd-networkd[711]: lo: Link UP Jul 15 10:53:45.875237 ignition[613]: no config URL provided Jul 15 10:53:45.878854 systemd-networkd[711]: lo: Gained carrier Jul 15 10:53:45.875243 ignition[613]: reading system config file "/usr/lib/ignition/user.ign" Jul 15 10:53:45.879692 systemd[1]: Finished ignition-fetch-offline.service - Ignition (fetch-offline). Jul 15 10:53:45.875251 ignition[613]: no config at "/usr/lib/ignition/user.ign" Jul 15 10:53:45.881633 systemd-networkd[711]: Enumeration completed Jul 15 10:53:45.875256 ignition[613]: failed to fetch config: resource requires networking Jul 15 10:53:45.882234 systemd[1]: Started systemd-networkd.service - Network Configuration. Jul 15 10:53:45.875456 ignition[613]: Ignition finished successfully Jul 15 10:53:45.882288 systemd-networkd[711]: eth0: found matching network '/usr/lib/systemd/network/zz-default.network', based on potentially unpredictable interface name. Jul 15 10:53:45.882292 systemd-networkd[711]: eth0: Configuring with /usr/lib/systemd/network/zz-default.network. Jul 15 10:53:45.884229 systemd[1]: Reached target network.target - Network. Jul 15 10:53:45.884429 systemd-networkd[711]: eth0: Link UP Jul 15 10:53:45.884433 systemd-networkd[711]: eth0: Gained carrier Jul 15 10:53:45.884442 systemd-networkd[711]: eth0: found matching network '/usr/lib/systemd/network/zz-default.network', based on potentially unpredictable interface name. Jul 15 10:53:45.890606 systemd[1]: Starting ignition-fetch.service - Ignition (fetch)... Jul 15 10:53:45.897573 systemd-networkd[711]: eth0: DHCPv4 address 172.24.4.51/24, gateway 172.24.4.1 acquired from 172.24.4.1 Jul 15 10:53:45.907207 ignition[720]: Ignition 2.20.0 Jul 15 10:53:45.907220 ignition[720]: Stage: fetch Jul 15 10:53:45.907440 ignition[720]: no configs at "/usr/lib/ignition/base.d" Jul 15 10:53:45.907452 ignition[720]: no config dir at "/usr/lib/ignition/base.platform.d/openstack" Jul 15 10:53:45.907547 ignition[720]: parsed url from cmdline: "" Jul 15 10:53:45.907550 ignition[720]: no config URL provided Jul 15 10:53:45.907556 ignition[720]: reading system config file "/usr/lib/ignition/user.ign" Jul 15 10:53:45.907564 ignition[720]: no config at "/usr/lib/ignition/user.ign" Jul 15 10:53:45.907647 ignition[720]: GET http://169.254.169.254/openstack/latest/user_data: attempt #1 Jul 15 10:53:45.907775 ignition[720]: config drive ("/dev/disk/by-label/config-2") not found. Waiting... Jul 15 10:53:45.907818 ignition[720]: config drive ("/dev/disk/by-label/CONFIG-2") not found. Waiting... Jul 15 10:53:46.193226 ignition[720]: GET result: OK Jul 15 10:53:46.193332 ignition[720]: parsing config with SHA512: 76b4390d21c1b2ed3d8c7bb0b8cf77e91280fd613adb77958c3f139a6ddcdeeae4b2e4e2673328a902a0f1941c831b9eaa32ee0442973fe3d860fc9651a5bc28 Jul 15 10:53:46.212580 unknown[720]: fetched base config from "system" Jul 15 10:53:46.212606 unknown[720]: fetched base config from "system" Jul 15 10:53:46.213082 ignition[720]: fetch: fetch complete Jul 15 10:53:46.212621 unknown[720]: fetched user config from "openstack" Jul 15 10:53:46.213095 ignition[720]: fetch: fetch passed Jul 15 10:53:46.216793 systemd[1]: Finished ignition-fetch.service - Ignition (fetch). Jul 15 10:53:46.213187 ignition[720]: Ignition finished successfully Jul 15 10:53:46.225721 systemd[1]: Starting ignition-kargs.service - Ignition (kargs)... Jul 15 10:53:46.270004 ignition[727]: Ignition 2.20.0 Jul 15 10:53:46.270032 ignition[727]: Stage: kargs Jul 15 10:53:46.270498 ignition[727]: no configs at "/usr/lib/ignition/base.d" Jul 15 10:53:46.270526 ignition[727]: no config dir at "/usr/lib/ignition/base.platform.d/openstack" Jul 15 10:53:46.274844 systemd[1]: Finished ignition-kargs.service - Ignition (kargs). Jul 15 10:53:46.272189 ignition[727]: kargs: kargs passed Jul 15 10:53:46.272283 ignition[727]: Ignition finished successfully Jul 15 10:53:46.293435 systemd[1]: Starting ignition-disks.service - Ignition (disks)... Jul 15 10:53:46.320919 ignition[733]: Ignition 2.20.0 Jul 15 10:53:46.320939 ignition[733]: Stage: disks Jul 15 10:53:46.321333 ignition[733]: no configs at "/usr/lib/ignition/base.d" Jul 15 10:53:46.321359 ignition[733]: no config dir at "/usr/lib/ignition/base.platform.d/openstack" Jul 15 10:53:46.323077 ignition[733]: disks: disks passed Jul 15 10:53:46.327621 systemd[1]: Finished ignition-disks.service - Ignition (disks). Jul 15 10:53:46.323171 ignition[733]: Ignition finished successfully Jul 15 10:53:46.329357 systemd[1]: Reached target initrd-root-device.target - Initrd Root Device. Jul 15 10:53:46.332119 systemd[1]: Reached target local-fs-pre.target - Preparation for Local File Systems. Jul 15 10:53:46.334629 systemd[1]: Reached target local-fs.target - Local File Systems. Jul 15 10:53:46.337149 systemd[1]: Reached target sysinit.target - System Initialization. Jul 15 10:53:46.339578 systemd[1]: Reached target basic.target - Basic System. Jul 15 10:53:46.349782 systemd[1]: Starting systemd-fsck-root.service - File System Check on /dev/disk/by-label/ROOT... Jul 15 10:53:46.386427 systemd-fsck[742]: ROOT: clean, 14/1628000 files, 120691/1617920 blocks Jul 15 10:53:46.397654 systemd[1]: Finished systemd-fsck-root.service - File System Check on /dev/disk/by-label/ROOT. Jul 15 10:53:46.407647 systemd[1]: Mounting sysroot.mount - /sysroot... Jul 15 10:53:46.545446 kernel: EXT4-fs (vda9): mounted filesystem 94753c30-8339-4199-9a73-c3199acfdab5 r/w with ordered data mode. Quota mode: none. Jul 15 10:53:46.548075 systemd[1]: Mounted sysroot.mount - /sysroot. Jul 15 10:53:46.550204 systemd[1]: Reached target initrd-root-fs.target - Initrd Root File System. Jul 15 10:53:46.557536 systemd[1]: Mounting sysroot-oem.mount - /sysroot/oem... Jul 15 10:53:46.561605 systemd[1]: Mounting sysroot-usr.mount - /sysroot/usr... Jul 15 10:53:46.562321 systemd[1]: flatcar-metadata-hostname.service - Flatcar Metadata Hostname Agent was skipped because no trigger condition checks were met. Jul 15 10:53:46.564796 systemd[1]: Starting flatcar-openstack-hostname.service - Flatcar OpenStack Metadata Hostname Agent... Jul 15 10:53:46.566350 systemd[1]: ignition-remount-sysroot.service - Remount /sysroot read-write for Ignition was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/sysroot). Jul 15 10:53:46.566401 systemd[1]: Reached target ignition-diskful.target - Ignition Boot Disk Setup. Jul 15 10:53:46.576407 kernel: BTRFS: device label OEM devid 1 transid 12 /dev/vda6 scanned by mount (750) Jul 15 10:53:46.582478 kernel: BTRFS info (device vda6): first mount of filesystem c20e5e0c-dae6-45ce-95fe-16a1362e2f71 Jul 15 10:53:46.582510 kernel: BTRFS info (device vda6): using crc32c (crc32c-intel) checksum algorithm Jul 15 10:53:46.582523 kernel: BTRFS info (device vda6): using free space tree Jul 15 10:53:46.585262 systemd[1]: Mounted sysroot-usr.mount - /sysroot/usr. Jul 15 10:53:46.595425 kernel: BTRFS info (device vda6): auto enabling async discard Jul 15 10:53:46.596612 systemd[1]: Starting initrd-setup-root.service - Root filesystem setup... Jul 15 10:53:46.610339 systemd[1]: Mounted sysroot-oem.mount - /sysroot/oem. Jul 15 10:53:46.711560 initrd-setup-root[779]: cut: /sysroot/etc/passwd: No such file or directory Jul 15 10:53:46.720566 initrd-setup-root[786]: cut: /sysroot/etc/group: No such file or directory Jul 15 10:53:46.727443 initrd-setup-root[793]: cut: /sysroot/etc/shadow: No such file or directory Jul 15 10:53:46.731948 initrd-setup-root[800]: cut: /sysroot/etc/gshadow: No such file or directory Jul 15 10:53:46.821205 systemd[1]: Finished initrd-setup-root.service - Root filesystem setup. Jul 15 10:53:46.827522 systemd[1]: Starting ignition-mount.service - Ignition (mount)... Jul 15 10:53:46.830513 systemd[1]: Starting sysroot-boot.service - /sysroot/boot... Jul 15 10:53:46.836635 systemd[1]: sysroot-oem.mount: Deactivated successfully. Jul 15 10:53:46.837962 kernel: BTRFS info (device vda6): last unmount of filesystem c20e5e0c-dae6-45ce-95fe-16a1362e2f71 Jul 15 10:53:46.867987 systemd[1]: Finished sysroot-boot.service - /sysroot/boot. Jul 15 10:53:46.875144 ignition[867]: INFO : Ignition 2.20.0 Jul 15 10:53:46.875144 ignition[867]: INFO : Stage: mount Jul 15 10:53:46.876283 ignition[867]: INFO : no configs at "/usr/lib/ignition/base.d" Jul 15 10:53:46.876283 ignition[867]: INFO : no config dir at "/usr/lib/ignition/base.platform.d/openstack" Jul 15 10:53:46.876283 ignition[867]: INFO : mount: mount passed Jul 15 10:53:46.876283 ignition[867]: INFO : Ignition finished successfully Jul 15 10:53:46.877634 systemd[1]: Finished ignition-mount.service - Ignition (mount). Jul 15 10:53:47.060753 systemd-networkd[711]: eth0: Gained IPv6LL Jul 15 10:53:53.771169 coreos-metadata[752]: Jul 15 10:53:53.770 WARN failed to locate config-drive, using the metadata service API instead Jul 15 10:53:53.830099 coreos-metadata[752]: Jul 15 10:53:53.828 INFO Fetching http://169.254.169.254/latest/meta-data/hostname: Attempt #1 Jul 15 10:53:53.845979 coreos-metadata[752]: Jul 15 10:53:53.845 INFO Fetch successful Jul 15 10:53:53.848111 coreos-metadata[752]: Jul 15 10:53:53.848 INFO wrote hostname ci-4230-2-100-7a68a02958.novalocal to /sysroot/etc/hostname Jul 15 10:53:53.857198 systemd[1]: flatcar-openstack-hostname.service: Deactivated successfully. Jul 15 10:53:53.858002 systemd[1]: Finished flatcar-openstack-hostname.service - Flatcar OpenStack Metadata Hostname Agent. Jul 15 10:53:53.874820 systemd[1]: Starting ignition-files.service - Ignition (files)... Jul 15 10:53:53.916888 systemd[1]: Mounting sysroot-oem.mount - /sysroot/oem... Jul 15 10:53:53.960555 kernel: BTRFS: device label OEM devid 1 transid 13 /dev/vda6 scanned by mount (884) Jul 15 10:53:53.969120 kernel: BTRFS info (device vda6): first mount of filesystem c20e5e0c-dae6-45ce-95fe-16a1362e2f71 Jul 15 10:53:53.969215 kernel: BTRFS info (device vda6): using crc32c (crc32c-intel) checksum algorithm Jul 15 10:53:53.973160 kernel: BTRFS info (device vda6): using free space tree Jul 15 10:53:53.987516 kernel: BTRFS info (device vda6): auto enabling async discard Jul 15 10:53:53.994217 systemd[1]: Mounted sysroot-oem.mount - /sysroot/oem. Jul 15 10:53:54.046635 ignition[902]: INFO : Ignition 2.20.0 Jul 15 10:53:54.046635 ignition[902]: INFO : Stage: files Jul 15 10:53:54.050607 ignition[902]: INFO : no configs at "/usr/lib/ignition/base.d" Jul 15 10:53:54.050607 ignition[902]: INFO : no config dir at "/usr/lib/ignition/base.platform.d/openstack" Jul 15 10:53:54.050607 ignition[902]: DEBUG : files: compiled without relabeling support, skipping Jul 15 10:53:54.057072 ignition[902]: INFO : files: ensureUsers: op(1): [started] creating or modifying user "core" Jul 15 10:53:54.057072 ignition[902]: DEBUG : files: ensureUsers: op(1): executing: "usermod" "--root" "/sysroot" "core" Jul 15 10:53:54.057072 ignition[902]: INFO : files: ensureUsers: op(1): [finished] creating or modifying user "core" Jul 15 10:53:54.063768 ignition[902]: INFO : files: ensureUsers: op(2): [started] adding ssh keys to user "core" Jul 15 10:53:54.063768 ignition[902]: INFO : files: ensureUsers: op(2): [finished] adding ssh keys to user "core" Jul 15 10:53:54.061113 unknown[902]: wrote ssh authorized keys file for user: core Jul 15 10:53:54.073205 ignition[902]: INFO : files: createFilesystemsFiles: createFiles: op(3): [started] writing file "/sysroot/etc/flatcar/update.conf" Jul 15 10:53:54.073205 ignition[902]: INFO : files: createFilesystemsFiles: createFiles: op(3): [finished] writing file "/sysroot/etc/flatcar/update.conf" Jul 15 10:53:54.073205 ignition[902]: INFO : files: op(4): [started] processing unit "coreos-metadata.service" Jul 15 10:53:54.073205 ignition[902]: INFO : files: op(4): op(5): [started] writing systemd drop-in "20-clct-provider-override.conf" at "/sysroot/etc/systemd/system/coreos-metadata.service.d/20-clct-provider-override.conf" Jul 15 10:53:54.073205 ignition[902]: INFO : files: op(4): op(5): [finished] writing systemd drop-in "20-clct-provider-override.conf" at "/sysroot/etc/systemd/system/coreos-metadata.service.d/20-clct-provider-override.conf" Jul 15 10:53:54.073205 ignition[902]: INFO : files: op(4): [finished] processing unit "coreos-metadata.service" Jul 15 10:53:54.073205 ignition[902]: INFO : files: op(6): [started] processing unit "coreos-metadata-sshkeys@.service" Jul 15 10:53:54.073205 ignition[902]: INFO : files: op(6): op(7): [started] writing systemd drop-in "20-clct-provider-override.conf" at "/sysroot/etc/systemd/system/coreos-metadata-sshkeys@.service.d/20-clct-provider-override.conf" Jul 15 10:53:54.073205 ignition[902]: INFO : files: op(6): op(7): [finished] writing systemd drop-in "20-clct-provider-override.conf" at "/sysroot/etc/systemd/system/coreos-metadata-sshkeys@.service.d/20-clct-provider-override.conf" Jul 15 10:53:54.073205 ignition[902]: INFO : files: op(6): [finished] processing unit "coreos-metadata-sshkeys@.service" Jul 15 10:53:54.073205 ignition[902]: INFO : files: op(8): [started] processing unit "etcd-member.service" Jul 15 10:53:54.073205 ignition[902]: INFO : files: op(8): op(9): [started] writing systemd drop-in "20-clct-etcd-member.conf" at "/sysroot/etc/systemd/system/etcd-member.service.d/20-clct-etcd-member.conf" Jul 15 10:53:54.073205 ignition[902]: INFO : files: op(8): op(9): [finished] writing systemd drop-in "20-clct-etcd-member.conf" at "/sysroot/etc/systemd/system/etcd-member.service.d/20-clct-etcd-member.conf" Jul 15 10:53:54.073205 ignition[902]: INFO : files: op(8): [finished] processing unit "etcd-member.service" Jul 15 10:53:54.073205 ignition[902]: INFO : files: op(a): [started] setting preset to enabled for "coreos-metadata-sshkeys@.service " Jul 15 10:53:54.073205 ignition[902]: INFO : files: op(a): [finished] setting preset to enabled for "coreos-metadata-sshkeys@.service " Jul 15 10:53:54.096081 ignition[902]: INFO : files: op(b): [started] setting preset to enabled for "etcd-member.service" Jul 15 10:53:54.096081 ignition[902]: INFO : files: op(b): [finished] setting preset to enabled for "etcd-member.service" Jul 15 10:53:54.096081 ignition[902]: INFO : files: createResultFile: createFiles: op(c): [started] writing file "/sysroot/etc/.ignition-result.json" Jul 15 10:53:54.096081 ignition[902]: INFO : files: createResultFile: createFiles: op(c): [finished] writing file "/sysroot/etc/.ignition-result.json" Jul 15 10:53:54.096081 ignition[902]: INFO : files: files passed Jul 15 10:53:54.096081 ignition[902]: INFO : Ignition finished successfully Jul 15 10:53:54.083853 systemd[1]: Finished ignition-files.service - Ignition (files). Jul 15 10:53:54.098722 systemd[1]: Starting ignition-quench.service - Ignition (record completion)... Jul 15 10:53:54.108730 systemd[1]: Starting initrd-setup-root-after-ignition.service - Root filesystem completion... Jul 15 10:53:54.116293 systemd[1]: ignition-quench.service: Deactivated successfully. Jul 15 10:53:54.117421 systemd[1]: Finished ignition-quench.service - Ignition (record completion). Jul 15 10:53:54.124846 initrd-setup-root-after-ignition[931]: grep: /sysroot/etc/flatcar/enabled-sysext.conf: No such file or directory Jul 15 10:53:54.124846 initrd-setup-root-after-ignition[931]: grep: /sysroot/usr/share/flatcar/enabled-sysext.conf: No such file or directory Jul 15 10:53:54.127326 initrd-setup-root-after-ignition[935]: grep: /sysroot/etc/flatcar/enabled-sysext.conf: No such file or directory Jul 15 10:53:54.127541 systemd[1]: Finished initrd-setup-root-after-ignition.service - Root filesystem completion. Jul 15 10:53:54.129274 systemd[1]: Reached target ignition-complete.target - Ignition Complete. Jul 15 10:53:54.134586 systemd[1]: Starting initrd-parse-etc.service - Mountpoints Configured in the Real Root... Jul 15 10:53:54.157916 systemd[1]: initrd-parse-etc.service: Deactivated successfully. Jul 15 10:53:54.158042 systemd[1]: Finished initrd-parse-etc.service - Mountpoints Configured in the Real Root. Jul 15 10:53:54.159320 systemd[1]: Reached target initrd-fs.target - Initrd File Systems. Jul 15 10:53:54.160447 systemd[1]: Reached target initrd.target - Initrd Default Target. Jul 15 10:53:54.161708 systemd[1]: dracut-mount.service - dracut mount hook was skipped because no trigger condition checks were met. Jul 15 10:53:54.167735 systemd[1]: Starting dracut-pre-pivot.service - dracut pre-pivot and cleanup hook... Jul 15 10:53:54.181985 systemd[1]: Finished dracut-pre-pivot.service - dracut pre-pivot and cleanup hook. Jul 15 10:53:54.193721 systemd[1]: Starting initrd-cleanup.service - Cleaning Up and Shutting Down Daemons... Jul 15 10:53:54.205696 systemd[1]: Stopped target nss-lookup.target - Host and Network Name Lookups. Jul 15 10:53:54.206592 systemd[1]: Stopped target remote-cryptsetup.target - Remote Encrypted Volumes. Jul 15 10:53:54.208674 systemd[1]: Stopped target timers.target - Timer Units. Jul 15 10:53:54.219247 systemd[1]: dracut-pre-pivot.service: Deactivated successfully. Jul 15 10:53:54.219460 systemd[1]: Stopped dracut-pre-pivot.service - dracut pre-pivot and cleanup hook. Jul 15 10:53:54.221565 systemd[1]: Stopped target initrd.target - Initrd Default Target. Jul 15 10:53:54.222586 systemd[1]: Stopped target basic.target - Basic System. Jul 15 10:53:54.224545 systemd[1]: Stopped target ignition-complete.target - Ignition Complete. Jul 15 10:53:54.226195 systemd[1]: Stopped target ignition-diskful.target - Ignition Boot Disk Setup. Jul 15 10:53:54.227711 systemd[1]: Stopped target initrd-root-device.target - Initrd Root Device. Jul 15 10:53:54.229742 systemd[1]: Stopped target remote-fs.target - Remote File Systems. Jul 15 10:53:54.231691 systemd[1]: Stopped target remote-fs-pre.target - Preparation for Remote File Systems. Jul 15 10:53:54.233704 systemd[1]: Stopped target sysinit.target - System Initialization. Jul 15 10:53:54.235593 systemd[1]: Stopped target local-fs.target - Local File Systems. Jul 15 10:53:54.237595 systemd[1]: Stopped target swap.target - Swaps. Jul 15 10:53:54.239406 systemd[1]: dracut-pre-mount.service: Deactivated successfully. Jul 15 10:53:54.239667 systemd[1]: Stopped dracut-pre-mount.service - dracut pre-mount hook. Jul 15 10:53:54.241768 systemd[1]: Stopped target cryptsetup.target - Local Encrypted Volumes. Jul 15 10:53:54.242776 systemd[1]: Stopped target cryptsetup-pre.target - Local Encrypted Volumes (Pre). Jul 15 10:53:54.244461 systemd[1]: clevis-luks-askpass.path: Deactivated successfully. Jul 15 10:53:54.244602 systemd[1]: Stopped clevis-luks-askpass.path - Forward Password Requests to Clevis Directory Watch. Jul 15 10:53:54.246450 systemd[1]: dracut-initqueue.service: Deactivated successfully. Jul 15 10:53:54.246653 systemd[1]: Stopped dracut-initqueue.service - dracut initqueue hook. Jul 15 10:53:54.249408 systemd[1]: initrd-setup-root-after-ignition.service: Deactivated successfully. Jul 15 10:53:54.249544 systemd[1]: Stopped initrd-setup-root-after-ignition.service - Root filesystem completion. Jul 15 10:53:54.250446 systemd[1]: ignition-files.service: Deactivated successfully. Jul 15 10:53:54.250566 systemd[1]: Stopped ignition-files.service - Ignition (files). Jul 15 10:53:54.259516 systemd[1]: Stopping ignition-mount.service - Ignition (mount)... Jul 15 10:53:54.263596 systemd[1]: Stopping sysroot-boot.service - /sysroot/boot... Jul 15 10:53:54.264673 systemd[1]: systemd-udev-trigger.service: Deactivated successfully. Jul 15 10:53:54.264821 systemd[1]: Stopped systemd-udev-trigger.service - Coldplug All udev Devices. Jul 15 10:53:54.266656 systemd[1]: dracut-pre-trigger.service: Deactivated successfully. Jul 15 10:53:54.266820 systemd[1]: Stopped dracut-pre-trigger.service - dracut pre-trigger hook. Jul 15 10:53:54.278127 ignition[956]: INFO : Ignition 2.20.0 Jul 15 10:53:54.278127 ignition[956]: INFO : Stage: umount Jul 15 10:53:54.278127 ignition[956]: INFO : no configs at "/usr/lib/ignition/base.d" Jul 15 10:53:54.278127 ignition[956]: INFO : no config dir at "/usr/lib/ignition/base.platform.d/openstack" Jul 15 10:53:54.278611 systemd[1]: initrd-cleanup.service: Deactivated successfully. Jul 15 10:53:54.284768 ignition[956]: INFO : umount: umount passed Jul 15 10:53:54.284768 ignition[956]: INFO : Ignition finished successfully Jul 15 10:53:54.278731 systemd[1]: Finished initrd-cleanup.service - Cleaning Up and Shutting Down Daemons. Jul 15 10:53:54.280836 systemd[1]: ignition-mount.service: Deactivated successfully. Jul 15 10:53:54.280952 systemd[1]: Stopped ignition-mount.service - Ignition (mount). Jul 15 10:53:54.284154 systemd[1]: ignition-disks.service: Deactivated successfully. Jul 15 10:53:54.284215 systemd[1]: Stopped ignition-disks.service - Ignition (disks). Jul 15 10:53:54.285253 systemd[1]: ignition-kargs.service: Deactivated successfully. Jul 15 10:53:54.285314 systemd[1]: Stopped ignition-kargs.service - Ignition (kargs). Jul 15 10:53:54.286758 systemd[1]: ignition-fetch.service: Deactivated successfully. Jul 15 10:53:54.286809 systemd[1]: Stopped ignition-fetch.service - Ignition (fetch). Jul 15 10:53:54.287293 systemd[1]: Stopped target network.target - Network. Jul 15 10:53:54.287756 systemd[1]: ignition-fetch-offline.service: Deactivated successfully. Jul 15 10:53:54.287814 systemd[1]: Stopped ignition-fetch-offline.service - Ignition (fetch-offline). Jul 15 10:53:54.288512 systemd[1]: Stopped target paths.target - Path Units. Jul 15 10:53:54.289415 systemd[1]: systemd-ask-password-console.path: Deactivated successfully. Jul 15 10:53:54.291466 systemd[1]: Stopped systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch. Jul 15 10:53:54.292246 systemd[1]: Stopped target slices.target - Slice Units. Jul 15 10:53:54.294190 systemd[1]: Stopped target sockets.target - Socket Units. Jul 15 10:53:54.295213 systemd[1]: iscsid.socket: Deactivated successfully. Jul 15 10:53:54.295257 systemd[1]: Closed iscsid.socket - Open-iSCSI iscsid Socket. Jul 15 10:53:54.296228 systemd[1]: iscsiuio.socket: Deactivated successfully. Jul 15 10:53:54.296262 systemd[1]: Closed iscsiuio.socket - Open-iSCSI iscsiuio Socket. Jul 15 10:53:54.297250 systemd[1]: ignition-setup.service: Deactivated successfully. Jul 15 10:53:54.297296 systemd[1]: Stopped ignition-setup.service - Ignition (setup). Jul 15 10:53:54.298239 systemd[1]: ignition-setup-pre.service: Deactivated successfully. Jul 15 10:53:54.298294 systemd[1]: Stopped ignition-setup-pre.service - Ignition env setup. Jul 15 10:53:54.299328 systemd[1]: Stopping systemd-networkd.service - Network Configuration... Jul 15 10:53:54.301852 systemd[1]: Stopping systemd-resolved.service - Network Name Resolution... Jul 15 10:53:54.307556 systemd[1]: systemd-resolved.service: Deactivated successfully. Jul 15 10:53:54.308611 systemd[1]: Stopped systemd-resolved.service - Network Name Resolution. Jul 15 10:53:54.312585 systemd[1]: sysroot-boot.mount: Deactivated successfully. Jul 15 10:53:54.312683 systemd[1]: run-credentials-systemd\x2dresolved.service.mount: Deactivated successfully. Jul 15 10:53:54.313222 systemd[1]: systemd-networkd.service: Deactivated successfully. Jul 15 10:53:54.313706 systemd[1]: Stopped systemd-networkd.service - Network Configuration. Jul 15 10:53:54.318105 systemd[1]: run-credentials-systemd\x2dnetworkd.service.mount: Deactivated successfully. Jul 15 10:53:54.318376 systemd[1]: sysroot-boot.service: Deactivated successfully. Jul 15 10:53:54.318530 systemd[1]: Stopped sysroot-boot.service - /sysroot/boot. Jul 15 10:53:54.320938 systemd[1]: systemd-networkd.socket: Deactivated successfully. Jul 15 10:53:54.321031 systemd[1]: Closed systemd-networkd.socket - Network Service Netlink Socket. Jul 15 10:53:54.322086 systemd[1]: initrd-setup-root.service: Deactivated successfully. Jul 15 10:53:54.322138 systemd[1]: Stopped initrd-setup-root.service - Root filesystem setup. Jul 15 10:53:54.328774 systemd[1]: Stopping network-cleanup.service - Network Cleanup... Jul 15 10:53:54.329315 systemd[1]: parse-ip-for-networkd.service: Deactivated successfully. Jul 15 10:53:54.329394 systemd[1]: Stopped parse-ip-for-networkd.service - Write systemd-networkd units from cmdline. Jul 15 10:53:54.329994 systemd[1]: systemd-sysctl.service: Deactivated successfully. Jul 15 10:53:54.330043 systemd[1]: Stopped systemd-sysctl.service - Apply Kernel Variables. Jul 15 10:53:54.330698 systemd[1]: systemd-modules-load.service: Deactivated successfully. Jul 15 10:53:54.330762 systemd[1]: Stopped systemd-modules-load.service - Load Kernel Modules. Jul 15 10:53:54.331698 systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. Jul 15 10:53:54.331745 systemd[1]: Stopped systemd-tmpfiles-setup.service - Create System Files and Directories. Jul 15 10:53:54.333461 systemd[1]: Stopping systemd-udevd.service - Rule-based Manager for Device Events and Files... Jul 15 10:53:54.335569 systemd[1]: run-credentials-systemd\x2dsysctl.service.mount: Deactivated successfully. Jul 15 10:53:54.335637 systemd[1]: run-credentials-systemd\x2dtmpfiles\x2dsetup.service.mount: Deactivated successfully. Jul 15 10:53:54.344973 systemd[1]: systemd-udevd.service: Deactivated successfully. Jul 15 10:53:54.345176 systemd[1]: Stopped systemd-udevd.service - Rule-based Manager for Device Events and Files. Jul 15 10:53:54.346725 systemd[1]: systemd-udevd-control.socket: Deactivated successfully. Jul 15 10:53:54.346797 systemd[1]: Closed systemd-udevd-control.socket - udev Control Socket. Jul 15 10:53:54.347482 systemd[1]: systemd-udevd-kernel.socket: Deactivated successfully. Jul 15 10:53:54.347519 systemd[1]: Closed systemd-udevd-kernel.socket - udev Kernel Socket. Jul 15 10:53:54.348734 systemd[1]: dracut-pre-udev.service: Deactivated successfully. Jul 15 10:53:54.348787 systemd[1]: Stopped dracut-pre-udev.service - dracut pre-udev hook. Jul 15 10:53:54.350913 systemd[1]: dracut-cmdline.service: Deactivated successfully. Jul 15 10:53:54.350984 systemd[1]: Stopped dracut-cmdline.service - dracut cmdline hook. Jul 15 10:53:54.353017 systemd[1]: dracut-cmdline-ask.service: Deactivated successfully. Jul 15 10:53:54.353078 systemd[1]: Stopped dracut-cmdline-ask.service - dracut ask for additional cmdline parameters. Jul 15 10:53:54.359571 systemd[1]: Starting initrd-udevadm-cleanup-db.service - Cleanup udev Database... Jul 15 10:53:54.360363 systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. Jul 15 10:53:54.360516 systemd[1]: Stopped systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev. Jul 15 10:53:54.364234 systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. Jul 15 10:53:54.364307 systemd[1]: Stopped systemd-tmpfiles-setup-dev-early.service - Create Static Device Nodes in /dev gracefully. Jul 15 10:53:54.365641 systemd[1]: kmod-static-nodes.service: Deactivated successfully. Jul 15 10:53:54.365692 systemd[1]: Stopped kmod-static-nodes.service - Create List of Static Device Nodes. Jul 15 10:53:54.367200 systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. Jul 15 10:53:54.367263 systemd[1]: Stopped systemd-vconsole-setup.service - Virtual Console Setup. Jul 15 10:53:54.369266 systemd[1]: run-credentials-systemd\x2dtmpfiles\x2dsetup\x2ddev.service.mount: Deactivated successfully. Jul 15 10:53:54.369330 systemd[1]: run-credentials-systemd\x2dvconsole\x2dsetup.service.mount: Deactivated successfully. Jul 15 10:53:54.369763 systemd[1]: network-cleanup.service: Deactivated successfully. Jul 15 10:53:54.369880 systemd[1]: Stopped network-cleanup.service - Network Cleanup. Jul 15 10:53:54.371178 systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. Jul 15 10:53:54.371271 systemd[1]: Finished initrd-udevadm-cleanup-db.service - Cleanup udev Database. Jul 15 10:53:54.373033 systemd[1]: Reached target initrd-switch-root.target - Switch Root. Jul 15 10:53:54.382802 systemd[1]: Starting initrd-switch-root.service - Switch Root... Jul 15 10:53:54.390832 systemd[1]: Switching root. Jul 15 10:53:54.432144 systemd-journald[185]: Journal stopped Jul 15 10:53:55.849656 systemd-journald[185]: Received SIGTERM from PID 1 (systemd). Jul 15 10:53:55.849754 kernel: SELinux: policy capability network_peer_controls=1 Jul 15 10:53:55.849771 kernel: SELinux: policy capability open_perms=1 Jul 15 10:53:55.849802 kernel: SELinux: policy capability extended_socket_class=1 Jul 15 10:53:55.849820 kernel: SELinux: policy capability always_check_network=0 Jul 15 10:53:55.849832 kernel: SELinux: policy capability cgroup_seclabel=1 Jul 15 10:53:55.849844 kernel: SELinux: policy capability nnp_nosuid_transition=1 Jul 15 10:53:55.849856 kernel: SELinux: policy capability genfs_seclabel_symlinks=0 Jul 15 10:53:55.849868 kernel: SELinux: policy capability ioctl_skip_cloexec=0 Jul 15 10:53:55.849879 kernel: audit: type=1403 audit(1752576834.624:2): auid=4294967295 ses=4294967295 lsm=selinux res=1 Jul 15 10:53:55.849902 systemd[1]: Successfully loaded SELinux policy in 82.350ms. Jul 15 10:53:55.849925 systemd[1]: Relabeled /dev/, /dev/shm/, /run/ in 30.665ms. Jul 15 10:53:55.849947 systemd[1]: systemd 256.8 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL -ACL +BLKID +CURL +ELFUTILS -FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 -PWQUALITY -P11KIT -QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP -SYSVINIT +LIBARCHIVE) Jul 15 10:53:55.849961 systemd[1]: Detected virtualization kvm. Jul 15 10:53:55.849974 systemd[1]: Detected architecture x86-64. Jul 15 10:53:55.849987 systemd[1]: Detected first boot. Jul 15 10:53:55.850001 systemd[1]: Hostname set to . Jul 15 10:53:55.850014 systemd[1]: Initializing machine ID from VM UUID. Jul 15 10:53:55.850027 zram_generator::config[1001]: No configuration found. Jul 15 10:53:55.850040 kernel: Guest personality initialized and is inactive Jul 15 10:53:55.850060 kernel: VMCI host device registered (name=vmci, major=10, minor=125) Jul 15 10:53:55.850073 kernel: Initialized host personality Jul 15 10:53:55.850084 kernel: NET: Registered PF_VSOCK protocol family Jul 15 10:53:55.850097 systemd[1]: Populated /etc with preset unit settings. Jul 15 10:53:55.850110 systemd[1]: run-credentials-systemd\x2djournald.service.mount: Deactivated successfully. Jul 15 10:53:55.850123 systemd[1]: initrd-switch-root.service: Deactivated successfully. Jul 15 10:53:55.850136 systemd[1]: Stopped initrd-switch-root.service - Switch Root. Jul 15 10:53:55.850157 systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. Jul 15 10:53:55.850178 systemd[1]: Created slice system-addon\x2dconfig.slice - Slice /system/addon-config. Jul 15 10:53:55.850192 systemd[1]: Created slice system-addon\x2drun.slice - Slice /system/addon-run. Jul 15 10:53:55.850205 systemd[1]: Created slice system-coreos\x2dmetadata\x2dsshkeys.slice - Slice /system/coreos-metadata-sshkeys. Jul 15 10:53:55.850218 systemd[1]: Created slice system-getty.slice - Slice /system/getty. Jul 15 10:53:55.850231 systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe. Jul 15 10:53:55.850244 systemd[1]: Created slice system-serial\x2dgetty.slice - Slice /system/serial-getty. Jul 15 10:53:55.850257 systemd[1]: Created slice system-system\x2dcloudinit.slice - Slice /system/system-cloudinit. Jul 15 10:53:55.850270 systemd[1]: Created slice system-systemd\x2dfsck.slice - Slice /system/systemd-fsck. Jul 15 10:53:55.850283 systemd[1]: Created slice user.slice - User and Session Slice. Jul 15 10:53:55.850311 systemd[1]: Started clevis-luks-askpass.path - Forward Password Requests to Clevis Directory Watch. Jul 15 10:53:55.850329 systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch. Jul 15 10:53:55.850342 systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch. Jul 15 10:53:55.850356 systemd[1]: Set up automount boot.automount - Boot partition Automount Point. Jul 15 10:53:55.850369 systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point. Jul 15 10:53:55.851558 systemd[1]: Expecting device dev-disk-by\x2dlabel-OEM.device - /dev/disk/by-label/OEM... Jul 15 10:53:55.851591 systemd[1]: Expecting device dev-ttyS0.device - /dev/ttyS0... Jul 15 10:53:55.851605 systemd[1]: Reached target cryptsetup-pre.target - Local Encrypted Volumes (Pre). Jul 15 10:53:55.851619 systemd[1]: Stopped target initrd-switch-root.target - Switch Root. Jul 15 10:53:55.851632 systemd[1]: Stopped target initrd-fs.target - Initrd File Systems. Jul 15 10:53:55.851647 systemd[1]: Stopped target initrd-root-fs.target - Initrd Root File System. Jul 15 10:53:55.851660 systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes. Jul 15 10:53:55.851673 systemd[1]: Reached target remote-cryptsetup.target - Remote Encrypted Volumes. Jul 15 10:53:55.851693 systemd[1]: Reached target remote-fs.target - Remote File Systems. Jul 15 10:53:55.851706 systemd[1]: Reached target slices.target - Slice Units. Jul 15 10:53:55.851730 systemd[1]: Reached target swap.target - Swaps. Jul 15 10:53:55.851743 systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes. Jul 15 10:53:55.851757 systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket. Jul 15 10:53:55.851770 systemd[1]: Listening on systemd-creds.socket - Credential Encryption/Decryption. Jul 15 10:53:55.851784 systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket. Jul 15 10:53:55.851797 systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket. Jul 15 10:53:55.851810 systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket. Jul 15 10:53:55.851824 systemd[1]: Listening on systemd-userdbd.socket - User Database Manager Socket. Jul 15 10:53:55.851838 systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System... Jul 15 10:53:55.851859 systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System... Jul 15 10:53:55.851872 systemd[1]: Mounting media.mount - External Media Directory... Jul 15 10:53:55.851886 systemd[1]: proc-xen.mount - /proc/xen was skipped because of an unmet condition check (ConditionVirtualization=xen). Jul 15 10:53:55.851900 systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System... Jul 15 10:53:55.851913 systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System... Jul 15 10:53:55.851926 systemd[1]: Mounting tmp.mount - Temporary Directory /tmp... Jul 15 10:53:55.851941 systemd[1]: var-lib-machines.mount - Virtual Machine and Container Storage (Compatibility) was skipped because of an unmet condition check (ConditionPathExists=/var/lib/machines.raw). Jul 15 10:53:55.851968 systemd[1]: Reached target machines.target - Containers. Jul 15 10:53:55.851986 systemd[1]: Starting flatcar-tmpfiles.service - Create missing system files... Jul 15 10:53:55.852006 systemd[1]: ignition-delete-config.service - Ignition (delete config) was skipped because no trigger condition checks were met. Jul 15 10:53:55.852020 systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes... Jul 15 10:53:55.852033 systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs... Jul 15 10:53:55.852047 systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod... Jul 15 10:53:55.852060 systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm... Jul 15 10:53:55.852073 systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore... Jul 15 10:53:55.852086 systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse... Jul 15 10:53:55.852099 systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop... Jul 15 10:53:55.852126 systemd[1]: setup-nsswitch.service - Create /etc/nsswitch.conf was skipped because of an unmet condition check (ConditionPathExists=!/etc/nsswitch.conf). Jul 15 10:53:55.852140 systemd[1]: systemd-fsck-root.service: Deactivated successfully. Jul 15 10:53:55.852153 systemd[1]: Stopped systemd-fsck-root.service - File System Check on Root Device. Jul 15 10:53:55.852166 systemd[1]: systemd-fsck-usr.service: Deactivated successfully. Jul 15 10:53:55.852179 systemd[1]: Stopped systemd-fsck-usr.service. Jul 15 10:53:55.852193 systemd[1]: systemd-hibernate-clear.service - Clear Stale Hibernate Storage Info was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67). Jul 15 10:53:55.852206 systemd[1]: Starting systemd-journald.service - Journal Service... Jul 15 10:53:55.852220 systemd[1]: Starting systemd-modules-load.service - Load Kernel Modules... Jul 15 10:53:55.852232 kernel: fuse: init (API version 7.39) Jul 15 10:53:55.852252 systemd[1]: Starting systemd-network-generator.service - Generate network units from Kernel command line... Jul 15 10:53:55.852265 kernel: loop: module loaded Jul 15 10:53:55.852278 systemd[1]: Starting systemd-remount-fs.service - Remount Root and Kernel File Systems... Jul 15 10:53:55.852291 systemd[1]: Starting systemd-udev-load-credentials.service - Load udev Rules from Credentials... Jul 15 10:53:55.852305 systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices... Jul 15 10:53:55.852325 systemd[1]: verity-setup.service: Deactivated successfully. Jul 15 10:53:55.852339 systemd[1]: Stopped verity-setup.service. Jul 15 10:53:55.852360 systemd[1]: xenserver-pv-version.service - Set fake PV driver version for XenServer was skipped because of an unmet condition check (ConditionVirtualization=xen). Jul 15 10:53:55.852373 systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System. Jul 15 10:53:55.857272 systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System. Jul 15 10:53:55.857304 systemd[1]: Mounted media.mount - External Media Directory. Jul 15 10:53:55.857318 kernel: ACPI: bus type drm_connector registered Jul 15 10:53:55.857331 systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System. Jul 15 10:53:55.857345 systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System. Jul 15 10:53:55.857358 systemd[1]: Mounted tmp.mount - Temporary Directory /tmp. Jul 15 10:53:55.857371 systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes. Jul 15 10:53:55.857430 systemd[1]: modprobe@configfs.service: Deactivated successfully. Jul 15 10:53:55.857448 systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs. Jul 15 10:53:55.857471 systemd[1]: modprobe@dm_mod.service: Deactivated successfully. Jul 15 10:53:55.857511 systemd-journald[1091]: Collecting audit messages is disabled. Jul 15 10:53:55.857539 systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod. Jul 15 10:53:55.857552 systemd[1]: modprobe@drm.service: Deactivated successfully. Jul 15 10:53:55.857565 systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm. Jul 15 10:53:55.857578 systemd[1]: modprobe@efi_pstore.service: Deactivated successfully. Jul 15 10:53:55.857592 systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore. Jul 15 10:53:55.857605 systemd[1]: modprobe@fuse.service: Deactivated successfully. Jul 15 10:53:55.857626 systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse. Jul 15 10:53:55.857641 systemd-journald[1091]: Journal started Jul 15 10:53:55.857673 systemd-journald[1091]: Runtime Journal (/run/log/journal/360f0ca0f20744b8ad97daf7d2f8a5e0) is 8M, max 78.3M, 70.3M free. Jul 15 10:53:55.439254 systemd[1]: Queued start job for default target multi-user.target. Jul 15 10:53:55.448287 systemd[1]: Unnecessary job was removed for dev-vda6.device - /dev/vda6. Jul 15 10:53:55.448780 systemd[1]: systemd-journald.service: Deactivated successfully. Jul 15 10:53:55.862293 systemd[1]: Started systemd-journald.service - Journal Service. Jul 15 10:53:55.863481 systemd[1]: Finished flatcar-tmpfiles.service - Create missing system files. Jul 15 10:53:55.864934 systemd[1]: modprobe@loop.service: Deactivated successfully. Jul 15 10:53:55.865516 systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop. Jul 15 10:53:55.866636 systemd[1]: Finished systemd-modules-load.service - Load Kernel Modules. Jul 15 10:53:55.867621 systemd[1]: Finished systemd-network-generator.service - Generate network units from Kernel command line. Jul 15 10:53:55.868552 systemd[1]: Finished systemd-remount-fs.service - Remount Root and Kernel File Systems. Jul 15 10:53:55.869546 systemd[1]: Finished systemd-udev-load-credentials.service - Load udev Rules from Credentials. Jul 15 10:53:55.882102 systemd[1]: Reached target network-pre.target - Preparation for Network. Jul 15 10:53:55.892869 systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System... Jul 15 10:53:55.897480 systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System... Jul 15 10:53:55.898203 systemd[1]: remount-root.service - Remount Root File System was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/). Jul 15 10:53:55.898473 systemd[1]: Reached target local-fs.target - Local File Systems. Jul 15 10:53:55.901948 systemd[1]: Listening on systemd-sysext.socket - System Extension Image Management. Jul 15 10:53:55.908342 systemd[1]: Starting dracut-shutdown.service - Restore /run/initramfs on shutdown... Jul 15 10:53:55.910819 systemd[1]: Starting ldconfig.service - Rebuild Dynamic Linker Cache... Jul 15 10:53:55.912010 systemd[1]: systemd-binfmt.service - Set Up Additional Binary Formats was skipped because no trigger condition checks were met. Jul 15 10:53:55.920918 systemd[1]: Starting systemd-hwdb-update.service - Rebuild Hardware Database... Jul 15 10:53:55.925565 systemd[1]: Starting systemd-journal-flush.service - Flush Journal to Persistent Storage... Jul 15 10:53:55.926225 systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore). Jul 15 10:53:55.934521 systemd[1]: Starting systemd-random-seed.service - Load/Save OS Random Seed... Jul 15 10:53:55.935154 systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met. Jul 15 10:53:55.939078 systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables... Jul 15 10:53:55.942630 systemd[1]: Starting systemd-sysext.service - Merge System Extension Images into /usr/ and /opt/... Jul 15 10:53:55.946824 systemd[1]: Starting systemd-tmpfiles-setup-dev-early.service - Create Static Device Nodes in /dev gracefully... Jul 15 10:53:55.949881 systemd[1]: Finished systemd-udev-trigger.service - Coldplug All udev Devices. Jul 15 10:53:55.952897 systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System. Jul 15 10:53:55.954045 systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System. Jul 15 10:53:55.956331 systemd[1]: Finished dracut-shutdown.service - Restore /run/initramfs on shutdown. Jul 15 10:53:55.957507 systemd-journald[1091]: Time spent on flushing to /var/log/journal/360f0ca0f20744b8ad97daf7d2f8a5e0 is 58.335ms for 952 entries. Jul 15 10:53:55.957507 systemd-journald[1091]: System Journal (/var/log/journal/360f0ca0f20744b8ad97daf7d2f8a5e0) is 8M, max 584.8M, 576.8M free. Jul 15 10:53:56.054501 systemd-journald[1091]: Received client request to flush runtime journal. Jul 15 10:53:56.054570 kernel: loop0: detected capacity change from 0 to 138176 Jul 15 10:53:55.983305 systemd[1]: Starting systemd-udev-settle.service - Wait for udev To Complete Device Initialization... Jul 15 10:53:55.993038 systemd[1]: Finished systemd-random-seed.service - Load/Save OS Random Seed. Jul 15 10:53:55.995439 systemd[1]: Reached target first-boot-complete.target - First Boot Complete. Jul 15 10:53:56.003563 systemd[1]: Starting systemd-machine-id-commit.service - Save Transient machine-id to Disk... Jul 15 10:53:56.021968 systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables. Jul 15 10:53:56.047085 udevadm[1147]: systemd-udev-settle.service is deprecated. Please fix lvm2-activation-early.service, lvm2-activation.service not to pull it in. Jul 15 10:53:56.055716 systemd[1]: Finished systemd-journal-flush.service - Flush Journal to Persistent Storage. Jul 15 10:53:56.066426 systemd-tmpfiles[1141]: ACLs are not supported, ignoring. Jul 15 10:53:56.066447 systemd-tmpfiles[1141]: ACLs are not supported, ignoring. Jul 15 10:53:56.072074 systemd[1]: Finished systemd-tmpfiles-setup-dev-early.service - Create Static Device Nodes in /dev gracefully. Jul 15 10:53:56.081598 systemd[1]: Starting systemd-sysusers.service - Create System Users... Jul 15 10:53:56.092269 systemd[1]: Finished systemd-machine-id-commit.service - Save Transient machine-id to Disk. Jul 15 10:53:56.119302 kernel: squashfs: version 4.0 (2009/01/31) Phillip Lougher Jul 15 10:53:56.147800 kernel: loop1: detected capacity change from 0 to 147912 Jul 15 10:53:56.149041 systemd[1]: Finished systemd-sysusers.service - Create System Users. Jul 15 10:53:56.158126 systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev... Jul 15 10:53:56.203583 systemd-tmpfiles[1163]: ACLs are not supported, ignoring. Jul 15 10:53:56.203604 systemd-tmpfiles[1163]: ACLs are not supported, ignoring. Jul 15 10:53:56.213187 systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev. Jul 15 10:53:56.220506 kernel: loop2: detected capacity change from 0 to 8 Jul 15 10:53:56.239430 kernel: loop3: detected capacity change from 0 to 138176 Jul 15 10:53:56.301419 kernel: loop4: detected capacity change from 0 to 147912 Jul 15 10:53:56.375553 kernel: loop5: detected capacity change from 0 to 8 Jul 15 10:53:56.384916 (sd-merge)[1168]: Using extensions 'containerd-flatcar', 'docker-flatcar', 'oem-openstack'. Jul 15 10:53:56.386357 (sd-merge)[1168]: Merged extensions into '/usr'. Jul 15 10:53:56.392192 systemd[1]: Finished systemd-sysext.service - Merge System Extension Images into /usr/ and /opt/. Jul 15 10:53:56.407622 systemd[1]: Starting ensure-sysext.service... Jul 15 10:53:56.409485 systemd[1]: Starting systemd-tmpfiles-setup.service - Create System Files and Directories... Jul 15 10:53:56.452697 systemd[1]: etc-machine\x2did.mount: Deactivated successfully. Jul 15 10:53:56.483266 systemd[1]: Reload requested from client PID 1170 ('systemctl') (unit ensure-sysext.service)... Jul 15 10:53:56.483487 systemd[1]: Reloading... Jul 15 10:53:56.488412 systemd-tmpfiles[1171]: /usr/lib/tmpfiles.d/provision.conf:20: Duplicate line for path "/root", ignoring. Jul 15 10:53:56.488772 systemd-tmpfiles[1171]: /usr/lib/tmpfiles.d/systemd-flatcar.conf:6: Duplicate line for path "/var/log/journal", ignoring. Jul 15 10:53:56.490372 systemd-tmpfiles[1171]: /usr/lib/tmpfiles.d/systemd.conf:29: Duplicate line for path "/var/lib/systemd", ignoring. Jul 15 10:53:56.491485 systemd-tmpfiles[1171]: ACLs are not supported, ignoring. Jul 15 10:53:56.491899 systemd-tmpfiles[1171]: ACLs are not supported, ignoring. Jul 15 10:53:56.499713 systemd-tmpfiles[1171]: Detected autofs mount point /boot during canonicalization of boot. Jul 15 10:53:56.499826 systemd-tmpfiles[1171]: Skipping /boot Jul 15 10:53:56.521350 systemd-tmpfiles[1171]: Detected autofs mount point /boot during canonicalization of boot. Jul 15 10:53:56.521365 systemd-tmpfiles[1171]: Skipping /boot Jul 15 10:53:56.635045 zram_generator::config[1204]: No configuration found. Jul 15 10:53:56.890970 systemd[1]: /usr/lib/systemd/system/docker.socket:6: ListenStream= references a path below legacy directory /var/run/, updating /var/run/docker.sock → /run/docker.sock; please update the unit file accordingly. Jul 15 10:53:56.980505 systemd[1]: Reloading finished in 496 ms. Jul 15 10:53:56.996581 systemd[1]: Finished systemd-tmpfiles-setup.service - Create System Files and Directories. Jul 15 10:53:57.027406 ldconfig[1135]: /sbin/ldconfig: /usr/lib/ld.so.conf is not an ELF file - it has the wrong magic bytes at the start. Jul 15 10:53:57.029577 systemd[1]: Starting audit-rules.service - Load Audit Rules... Jul 15 10:53:57.034569 systemd[1]: Starting clean-ca-certificates.service - Clean up broken links in /etc/ssl/certs... Jul 15 10:53:57.042583 systemd[1]: Starting systemd-journal-catalog-update.service - Rebuild Journal Catalog... Jul 15 10:53:57.050589 systemd[1]: Starting systemd-resolved.service - Network Name Resolution... Jul 15 10:53:57.060371 systemd[1]: Starting systemd-update-utmp.service - Record System Boot/Shutdown in UTMP... Jul 15 10:53:57.061880 systemd[1]: Finished ldconfig.service - Rebuild Dynamic Linker Cache. Jul 15 10:53:57.062923 systemd[1]: Finished systemd-hwdb-update.service - Rebuild Hardware Database. Jul 15 10:53:57.078907 systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files... Jul 15 10:53:57.090304 systemd[1]: Starting systemd-userdbd.service - User Database Manager... Jul 15 10:53:57.099314 systemd[1]: proc-xen.mount - /proc/xen was skipped because of an unmet condition check (ConditionVirtualization=xen). Jul 15 10:53:57.100719 systemd[1]: ignition-delete-config.service - Ignition (delete config) was skipped because no trigger condition checks were met. Jul 15 10:53:57.107524 systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod... Jul 15 10:53:57.114703 systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore... Jul 15 10:53:57.121793 systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop... Jul 15 10:53:57.122456 systemd[1]: systemd-binfmt.service - Set Up Additional Binary Formats was skipped because no trigger condition checks were met. Jul 15 10:53:57.122664 systemd[1]: systemd-hibernate-clear.service - Clear Stale Hibernate Storage Info was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67). Jul 15 10:53:57.122872 systemd[1]: xenserver-pv-version.service - Set fake PV driver version for XenServer was skipped because of an unmet condition check (ConditionVirtualization=xen). Jul 15 10:53:57.126648 systemd[1]: modprobe@dm_mod.service: Deactivated successfully. Jul 15 10:53:57.127339 systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod. Jul 15 10:53:57.130518 systemd[1]: Finished systemd-journal-catalog-update.service - Rebuild Journal Catalog. Jul 15 10:53:57.135964 systemd[1]: Finished systemd-update-utmp.service - Record System Boot/Shutdown in UTMP. Jul 15 10:53:57.148762 systemd-udevd[1280]: Using default interface naming scheme 'v255'. Jul 15 10:53:57.150283 systemd[1]: modprobe@efi_pstore.service: Deactivated successfully. Jul 15 10:53:57.150561 systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore. Jul 15 10:53:57.151750 systemd[1]: modprobe@loop.service: Deactivated successfully. Jul 15 10:53:57.152196 systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop. Jul 15 10:53:57.161526 systemd[1]: proc-xen.mount - /proc/xen was skipped because of an unmet condition check (ConditionVirtualization=xen). Jul 15 10:53:57.161972 systemd[1]: ignition-delete-config.service - Ignition (delete config) was skipped because no trigger condition checks were met. Jul 15 10:53:57.163804 augenrules[1291]: No rules Jul 15 10:53:57.168771 systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod... Jul 15 10:53:57.173706 systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm... Jul 15 10:53:57.177343 systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore... Jul 15 10:53:57.180732 systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop... Jul 15 10:53:57.181707 systemd[1]: systemd-binfmt.service - Set Up Additional Binary Formats was skipped because no trigger condition checks were met. Jul 15 10:53:57.181918 systemd[1]: systemd-hibernate-clear.service - Clear Stale Hibernate Storage Info was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67). Jul 15 10:53:57.195689 systemd[1]: Starting systemd-update-done.service - Update is Completed... Jul 15 10:53:57.200171 systemd[1]: xenserver-pv-version.service - Set fake PV driver version for XenServer was skipped because of an unmet condition check (ConditionVirtualization=xen). Jul 15 10:53:57.202535 systemd[1]: audit-rules.service: Deactivated successfully. Jul 15 10:53:57.203480 systemd[1]: Finished audit-rules.service - Load Audit Rules. Jul 15 10:53:57.205841 systemd[1]: modprobe@dm_mod.service: Deactivated successfully. Jul 15 10:53:57.206019 systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod. Jul 15 10:53:57.213551 systemd[1]: Finished ensure-sysext.service. Jul 15 10:53:57.216762 systemd[1]: Finished systemd-update-done.service - Update is Completed. Jul 15 10:53:57.232016 systemd[1]: Starting systemd-timesyncd.service - Network Time Synchronization... Jul 15 10:53:57.233024 systemd[1]: Started systemd-userdbd.service - User Database Manager. Jul 15 10:53:57.236452 systemd[1]: Finished clean-ca-certificates.service - Clean up broken links in /etc/ssl/certs. Jul 15 10:53:57.237351 systemd[1]: modprobe@drm.service: Deactivated successfully. Jul 15 10:53:57.237579 systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm. Jul 15 10:53:57.238358 systemd[1]: modprobe@efi_pstore.service: Deactivated successfully. Jul 15 10:53:57.239589 systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore. Jul 15 10:53:57.240710 systemd[1]: modprobe@loop.service: Deactivated successfully. Jul 15 10:53:57.240875 systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop. Jul 15 10:53:57.242525 systemd[1]: Started systemd-udevd.service - Rule-based Manager for Device Events and Files. Jul 15 10:53:57.270547 systemd[1]: Starting systemd-networkd.service - Network Configuration... Jul 15 10:53:57.272358 systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore). Jul 15 10:53:57.272457 systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met. Jul 15 10:53:57.272492 systemd[1]: update-ca-certificates.service - Update CA bundle at /etc/ssl/certs/ca-certificates.crt was skipped because of an unmet condition check (ConditionPathIsSymbolicLink=!/etc/ssl/certs/ca-certificates.crt). Jul 15 10:53:57.345532 systemd[1]: Condition check resulted in dev-ttyS0.device - /dev/ttyS0 being skipped. Jul 15 10:53:57.401317 kernel: BTRFS warning: duplicate device /dev/vda3 devid 1 generation 36 scanned by (udev-worker) (1324) Jul 15 10:53:57.402693 systemd-resolved[1265]: Positive Trust Anchors: Jul 15 10:53:57.402719 systemd-resolved[1265]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d Jul 15 10:53:57.402764 systemd-resolved[1265]: Negative trust anchors: home.arpa 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 170.0.0.192.in-addr.arpa 171.0.0.192.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa ipv4only.arpa resolver.arpa corp home internal intranet lan local private test Jul 15 10:53:57.421289 systemd-resolved[1265]: Using system hostname 'ci-4230-2-100-7a68a02958.novalocal'. Jul 15 10:53:57.427569 systemd[1]: Started systemd-resolved.service - Network Name Resolution. Jul 15 10:53:57.428289 systemd[1]: Reached target nss-lookup.target - Host and Network Name Lookups. Jul 15 10:53:57.433620 systemd-networkd[1328]: lo: Link UP Jul 15 10:53:57.433630 systemd-networkd[1328]: lo: Gained carrier Jul 15 10:53:57.435031 systemd-networkd[1328]: Enumeration completed Jul 15 10:53:57.435154 systemd[1]: Started systemd-networkd.service - Network Configuration. Jul 15 10:53:57.436238 systemd[1]: Reached target network.target - Network. Jul 15 10:53:57.445605 systemd[1]: Starting systemd-networkd-persistent-storage.service - Enable Persistent Storage in systemd-networkd... Jul 15 10:53:57.456648 systemd[1]: Starting systemd-networkd-wait-online.service - Wait for Network to be Configured... Jul 15 10:53:57.457807 systemd[1]: Started systemd-timesyncd.service - Network Time Synchronization. Jul 15 10:53:57.461081 systemd[1]: Reached target time-set.target - System Time Set. Jul 15 10:53:57.480843 systemd[1]: Finished systemd-networkd-persistent-storage.service - Enable Persistent Storage in systemd-networkd. Jul 15 10:53:57.502733 systemd-networkd[1328]: eth0: found matching network '/usr/lib/systemd/network/zz-default.network', based on potentially unpredictable interface name. Jul 15 10:53:57.502746 systemd-networkd[1328]: eth0: Configuring with /usr/lib/systemd/network/zz-default.network. Jul 15 10:53:57.505004 systemd-networkd[1328]: eth0: Link UP Jul 15 10:53:57.505013 systemd-networkd[1328]: eth0: Gained carrier Jul 15 10:53:57.505030 systemd-networkd[1328]: eth0: found matching network '/usr/lib/systemd/network/zz-default.network', based on potentially unpredictable interface name. Jul 15 10:53:57.515441 systemd-networkd[1328]: eth0: DHCPv4 address 172.24.4.51/24, gateway 172.24.4.1 acquired from 172.24.4.1 Jul 15 10:53:57.519487 systemd-timesyncd[1312]: Network configuration changed, trying to establish connection. Jul 15 10:53:57.533425 kernel: input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input2 Jul 15 10:53:57.538408 kernel: ACPI: button: Power Button [PWRF] Jul 15 10:53:57.557413 kernel: input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input3 Jul 15 10:53:57.559967 systemd[1]: Found device dev-disk-by\x2dlabel-OEM.device - /dev/disk/by-label/OEM. Jul 15 10:53:57.568588 systemd[1]: Starting systemd-fsck@dev-disk-by\x2dlabel-OEM.service - File System Check on /dev/disk/by-label/OEM... Jul 15 10:53:57.585429 kernel: piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 Jul 15 10:53:57.586116 systemd[1]: Finished systemd-fsck@dev-disk-by\x2dlabel-OEM.service - File System Check on /dev/disk/by-label/OEM. Jul 15 10:53:57.614426 kernel: mousedev: PS/2 mouse device common for all mice Jul 15 10:53:57.634615 systemd[1]: Starting systemd-vconsole-setup.service - Virtual Console Setup... Jul 15 10:53:57.634734 systemd-timesyncd[1312]: Contacted time server 192.155.94.72:123 (0.flatcar.pool.ntp.org). Jul 15 10:53:57.634817 systemd-timesyncd[1312]: Initial clock synchronization to Tue 2025-07-15 10:53:57.949109 UTC. Jul 15 10:53:57.653465 kernel: [drm] pci: virtio-vga detected at 0000:00:02.0 Jul 15 10:53:57.653543 kernel: virtio-pci 0000:00:02.0: vgaarb: deactivate vga console Jul 15 10:53:57.657696 kernel: Console: switching to colour dummy device 80x25 Jul 15 10:53:57.657758 kernel: [drm] features: -virgl +edid -resource_blob -host_visible Jul 15 10:53:57.657774 kernel: [drm] features: -context_init Jul 15 10:53:57.661804 kernel: [drm] number of scanouts: 1 Jul 15 10:53:57.661847 kernel: [drm] number of cap sets: 0 Jul 15 10:53:57.665415 kernel: [drm] Initialized virtio_gpu 0.1.0 0 for 0000:00:02.0 on minor 0 Jul 15 10:53:57.670747 systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. Jul 15 10:53:57.671021 systemd[1]: Stopped systemd-vconsole-setup.service - Virtual Console Setup. Jul 15 10:53:57.673631 systemd[1]: run-credentials-systemd\x2dvconsole\x2dsetup.service.mount: Deactivated successfully. Jul 15 10:53:57.681667 systemd[1]: Starting systemd-vconsole-setup.service - Virtual Console Setup... Jul 15 10:53:57.701464 kernel: fbcon: virtio_gpudrmfb (fb0) is primary device Jul 15 10:53:57.701559 kernel: Console: switching to colour frame buffer device 160x50 Jul 15 10:53:57.714194 kernel: virtio-pci 0000:00:02.0: [drm] fb0: virtio_gpudrmfb frame buffer device Jul 15 10:53:57.715710 systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. Jul 15 10:53:57.716087 systemd[1]: Stopped systemd-vconsole-setup.service - Virtual Console Setup. Jul 15 10:53:57.719595 systemd[1]: run-credentials-systemd\x2dvconsole\x2dsetup.service.mount: Deactivated successfully. Jul 15 10:53:57.726611 systemd[1]: Starting systemd-vconsole-setup.service - Virtual Console Setup... Jul 15 10:53:57.729091 systemd[1]: Finished systemd-udev-settle.service - Wait for udev To Complete Device Initialization. Jul 15 10:53:57.732752 systemd[1]: Starting lvm2-activation-early.service - Activation of LVM2 logical volumes... Jul 15 10:53:57.760513 lvm[1369]: WARNING: Failed to connect to lvmetad. Falling back to device scanning. Jul 15 10:53:57.787731 systemd[1]: Finished lvm2-activation-early.service - Activation of LVM2 logical volumes. Jul 15 10:53:57.788716 systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes. Jul 15 10:53:57.795864 systemd[1]: Starting lvm2-activation.service - Activation of LVM2 logical volumes... Jul 15 10:53:57.800010 lvm[1373]: WARNING: Failed to connect to lvmetad. Falling back to device scanning. Jul 15 10:53:57.811247 systemd[1]: Finished systemd-vconsole-setup.service - Virtual Console Setup. Jul 15 10:53:57.811690 systemd[1]: Reached target sysinit.target - System Initialization. Jul 15 10:53:57.811875 systemd[1]: Started motdgen.path - Watch for update engine configuration changes. Jul 15 10:53:57.812008 systemd[1]: Started user-cloudinit@var-lib-flatcar\x2dinstall-user_data.path - Watch for a cloud-config at /var/lib/flatcar-install/user_data. Jul 15 10:53:57.812278 systemd[1]: Started logrotate.timer - Daily rotation of log files. Jul 15 10:53:57.812829 systemd[1]: Started mdadm.timer - Weekly check for MD array's redundancy information.. Jul 15 10:53:57.814515 systemd[1]: Started systemd-tmpfiles-clean.timer - Daily Cleanup of Temporary Directories. Jul 15 10:53:57.814603 systemd[1]: update-engine-stub.timer - Update Engine Stub Timer was skipped because of an unmet condition check (ConditionPathExists=/usr/.noupdate). Jul 15 10:53:57.814629 systemd[1]: Reached target paths.target - Path Units. Jul 15 10:53:57.814694 systemd[1]: Reached target timers.target - Timer Units. Jul 15 10:53:57.816799 systemd[1]: Listening on dbus.socket - D-Bus System Message Bus Socket. Jul 15 10:53:57.818478 systemd[1]: Starting docker.socket - Docker Socket for the API... Jul 15 10:53:57.822866 systemd[1]: Listening on sshd-unix-local.socket - OpenSSH Server Socket (systemd-ssh-generator, AF_UNIX Local). Jul 15 10:53:57.824622 systemd[1]: Listening on sshd-vsock.socket - OpenSSH Server Socket (systemd-ssh-generator, AF_VSOCK). Jul 15 10:53:57.824805 systemd[1]: Reached target ssh-access.target - SSH Access Available. Jul 15 10:53:57.827213 systemd[1]: Listening on sshd.socket - OpenSSH Server Socket. Jul 15 10:53:57.830248 systemd[1]: Listening on systemd-hostnamed.socket - Hostname Service Socket. Jul 15 10:53:57.833456 systemd[1]: Finished lvm2-activation.service - Activation of LVM2 logical volumes. Jul 15 10:53:57.835796 systemd[1]: Listening on docker.socket - Docker Socket for the API. Jul 15 10:53:57.838374 systemd[1]: Reached target sockets.target - Socket Units. Jul 15 10:53:57.840654 systemd[1]: Reached target basic.target - Basic System. Jul 15 10:53:57.843255 systemd[1]: addon-config@oem.service - Configure Addon /oem was skipped because no trigger condition checks were met. Jul 15 10:53:57.843297 systemd[1]: addon-run@oem.service - Run Addon /oem was skipped because no trigger condition checks were met. Jul 15 10:53:57.854549 systemd[1]: Starting containerd.service - containerd container runtime... Jul 15 10:53:57.859609 systemd[1]: Starting coreos-metadata-sshkeys@core.service - Flatcar Metadata Agent (SSH Keys)... Jul 15 10:53:57.875591 systemd[1]: Starting coreos-metadata.service - Flatcar Metadata Agent... Jul 15 10:53:57.880803 systemd[1]: Starting dbus.service - D-Bus System Message Bus... Jul 15 10:53:57.894952 coreos-metadata[1380]: Jul 15 10:53:57.894 INFO Fetching http://169.254.169.254/latest/meta-data/public-keys: Attempt #1 Jul 15 10:53:57.896484 systemd[1]: Starting enable-oem-cloudinit.service - Enable cloudinit... Jul 15 10:53:57.908705 systemd[1]: Starting extend-filesystems.service - Extend Filesystems... Jul 15 10:53:57.915829 jq[1387]: false Jul 15 10:53:57.914181 systemd[1]: flatcar-setup-environment.service - Modifies /etc/environment for CoreOS was skipped because of an unmet condition check (ConditionPathExists=/oem/bin/flatcar-setup-environment). Jul 15 10:53:57.919889 coreos-metadata[1381]: Jul 15 10:53:57.918 INFO Fetching http://169.254.169.254/openstack/2012-08-10/meta_data.json: Attempt #1 Jul 15 10:53:57.922860 coreos-metadata[1380]: Jul 15 10:53:57.920 INFO Fetch successful Jul 15 10:53:57.922860 coreos-metadata[1380]: Jul 15 10:53:57.920 INFO Fetching http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key: Attempt #1 Jul 15 10:53:57.923277 systemd[1]: Starting motdgen.service - Generate /run/flatcar/motd... Jul 15 10:53:57.929977 systemd[1]: Starting ssh-key-proc-cmdline.service - Install an ssh key from /proc/cmdline... Jul 15 10:53:57.937007 coreos-metadata[1380]: Jul 15 10:53:57.936 INFO Fetch successful Jul 15 10:53:57.938342 coreos-metadata[1381]: Jul 15 10:53:57.938 INFO Fetch successful Jul 15 10:53:57.938475 coreos-metadata[1381]: Jul 15 10:53:57.938 INFO Fetching http://169.254.169.254/latest/meta-data/hostname: Attempt #1 Jul 15 10:53:57.941595 systemd[1]: Starting sshd-keygen.service - Generate sshd host keys... Jul 15 10:53:57.946349 unknown[1380]: wrote ssh authorized keys file for user: core Jul 15 10:53:57.951909 coreos-metadata[1381]: Jul 15 10:53:57.951 INFO Fetch successful Jul 15 10:53:57.951909 coreos-metadata[1381]: Jul 15 10:53:57.951 INFO Fetching http://169.254.169.254/latest/meta-data/instance-id: Attempt #1 Jul 15 10:53:57.952335 dbus-daemon[1384]: [system] SELinux support is enabled Jul 15 10:53:57.956331 systemd[1]: Starting systemd-logind.service - User Login Management... Jul 15 10:53:57.958341 systemd[1]: tcsd.service - TCG Core Services Daemon was skipped because of an unmet condition check (ConditionPathExists=/dev/tpm0). Jul 15 10:53:57.962108 systemd[1]: cgroup compatibility translation between legacy and unified hierarchy settings activated. See cgroup-compat debug messages for details. Jul 15 10:53:57.967549 systemd[1]: Starting update-engine.service - Update Engine... Jul 15 10:53:57.971789 coreos-metadata[1381]: Jul 15 10:53:57.971 INFO Fetch successful Jul 15 10:53:57.971789 coreos-metadata[1381]: Jul 15 10:53:57.971 INFO Fetching http://169.254.169.254/latest/meta-data/instance-type: Attempt #1 Jul 15 10:53:57.977579 systemd[1]: Starting update-ssh-keys-after-ignition.service - Run update-ssh-keys once after Ignition... Jul 15 10:53:57.979260 systemd[1]: Started dbus.service - D-Bus System Message Bus. Jul 15 10:53:57.984462 coreos-metadata[1381]: Jul 15 10:53:57.984 INFO Fetch successful Jul 15 10:53:57.984462 coreos-metadata[1381]: Jul 15 10:53:57.984 INFO Fetching http://169.254.169.254/latest/meta-data/local-ipv4: Attempt #1 Jul 15 10:53:57.991131 extend-filesystems[1388]: Found loop3 Jul 15 10:53:58.006159 extend-filesystems[1388]: Found loop4 Jul 15 10:53:58.006159 extend-filesystems[1388]: Found loop5 Jul 15 10:53:58.006159 extend-filesystems[1388]: Found vda Jul 15 10:53:58.006159 extend-filesystems[1388]: Found vda1 Jul 15 10:53:58.006159 extend-filesystems[1388]: Found vda2 Jul 15 10:53:58.006159 extend-filesystems[1388]: Found vda3 Jul 15 10:53:58.006159 extend-filesystems[1388]: Found usr Jul 15 10:53:58.006159 extend-filesystems[1388]: Found vda4 Jul 15 10:53:58.006159 extend-filesystems[1388]: Found vda6 Jul 15 10:53:58.006159 extend-filesystems[1388]: Found vda7 Jul 15 10:53:58.006159 extend-filesystems[1388]: Found vda9 Jul 15 10:53:58.006159 extend-filesystems[1388]: Checking size of /dev/vda9 Jul 15 10:53:58.081948 coreos-metadata[1381]: Jul 15 10:53:57.998 INFO Fetch successful Jul 15 10:53:58.081948 coreos-metadata[1381]: Jul 15 10:53:57.998 INFO Fetching http://169.254.169.254/latest/meta-data/public-ipv4: Attempt #1 Jul 15 10:53:58.081948 coreos-metadata[1381]: Jul 15 10:53:58.013 INFO Fetch successful Jul 15 10:53:57.997770 systemd[1]: enable-oem-cloudinit.service: Skipped due to 'exec-condition'. Jul 15 10:53:58.082200 extend-filesystems[1388]: Resized partition /dev/vda9 Jul 15 10:53:58.189491 kernel: EXT4-fs (vda9): resizing filesystem from 1617920 to 2014203 blocks Jul 15 10:53:58.189546 kernel: BTRFS warning: duplicate device /dev/vda3 devid 1 generation 36 scanned by (udev-worker) (1324) Jul 15 10:53:58.189568 kernel: EXT4-fs (vda9): resized filesystem to 2014203 Jul 15 10:53:58.189670 jq[1402]: true Jul 15 10:53:57.997984 systemd[1]: Condition check resulted in enable-oem-cloudinit.service - Enable cloudinit being skipped. Jul 15 10:53:58.191672 extend-filesystems[1412]: resize2fs 1.47.1 (20-May-2024) Jul 15 10:53:58.192582 update_engine[1397]: I20250715 10:53:58.042719 1397 main.cc:92] Flatcar Update Engine starting Jul 15 10:53:58.192582 update_engine[1397]: I20250715 10:53:58.047152 1397 update_check_scheduler.cc:74] Next update check in 11m32s Jul 15 10:53:57.998350 systemd[1]: motdgen.service: Deactivated successfully. Jul 15 10:53:57.998587 systemd[1]: Finished motdgen.service - Generate /run/flatcar/motd. Jul 15 10:53:58.034027 systemd[1]: ssh-key-proc-cmdline.service: Deactivated successfully. Jul 15 10:53:58.198334 update-ssh-keys[1408]: Updated "/home/core/.ssh/authorized_keys" Jul 15 10:53:58.034256 systemd[1]: Finished ssh-key-proc-cmdline.service - Install an ssh key from /proc/cmdline. Jul 15 10:53:58.083800 (ntainerd)[1414]: containerd.service: Referenced but unset environment variable evaluates to an empty string: TORCX_IMAGEDIR, TORCX_UNPACKDIR Jul 15 10:53:58.085336 systemd[1]: Finished coreos-metadata-sshkeys@core.service - Flatcar Metadata Agent (SSH Keys). Jul 15 10:53:58.213388 extend-filesystems[1412]: Filesystem at /dev/vda9 is mounted on /; on-line resizing required Jul 15 10:53:58.213388 extend-filesystems[1412]: old_desc_blocks = 1, new_desc_blocks = 1 Jul 15 10:53:58.213388 extend-filesystems[1412]: The filesystem on /dev/vda9 is now 2014203 (4k) blocks long. Jul 15 10:53:58.220519 jq[1413]: true Jul 15 10:53:58.147795 systemd[1]: Started update-engine.service - Update Engine. Jul 15 10:53:58.225690 extend-filesystems[1388]: Resized filesystem in /dev/vda9 Jul 15 10:53:58.159134 systemd[1]: Finished coreos-metadata.service - Flatcar Metadata Agent. Jul 15 10:53:58.169493 systemd[1]: packet-phone-home.service - Report Success to Packet was skipped because no trigger condition checks were met. Jul 15 10:53:58.169621 systemd[1]: system-cloudinit@usr-share-oem-cloud\x2dconfig.yml.service - Load cloud-config from /usr/share/oem/cloud-config.yml was skipped because of an unmet condition check (ConditionFileNotEmpty=/usr/share/oem/cloud-config.yml). Jul 15 10:53:58.169646 systemd[1]: Reached target system-config.target - Load system-provided cloud configs. Jul 15 10:53:58.173099 systemd[1]: user-cloudinit-proc-cmdline.service - Load cloud-config from url defined in /proc/cmdline was skipped because of an unmet condition check (ConditionKernelCommandLine=cloud-config-url). Jul 15 10:53:58.173122 systemd[1]: Reached target user-config.target - Load user-provided cloud configs. Jul 15 10:53:58.186024 systemd[1]: Started locksmithd.service - Cluster reboot manager. Jul 15 10:53:58.206077 systemd[1]: extend-filesystems.service: Deactivated successfully. Jul 15 10:53:58.206375 systemd[1]: Finished extend-filesystems.service - Extend Filesystems. Jul 15 10:53:58.272697 systemd-logind[1395]: New seat seat0. Jul 15 10:53:58.286662 systemd-logind[1395]: Watching system buttons on /dev/input/event1 (Power Button) Jul 15 10:53:58.296242 bash[1446]: Updated "/home/core/.ssh/authorized_keys" Jul 15 10:53:58.286692 systemd-logind[1395]: Watching system buttons on /dev/input/event0 (AT Translated Set 2 keyboard) Jul 15 10:53:58.288861 systemd[1]: Started systemd-logind.service - User Login Management. Jul 15 10:53:58.298719 systemd[1]: Finished update-ssh-keys-after-ignition.service - Run update-ssh-keys once after Ignition. Jul 15 10:53:58.313765 systemd[1]: Starting sshkeys.service... Jul 15 10:53:58.333435 locksmithd[1424]: locksmithd starting currentOperation="UPDATE_STATUS_IDLE" strategy="reboot" Jul 15 10:53:58.363787 systemd[1]: Finished sshkeys.service. Jul 15 10:53:58.512130 containerd[1414]: time="2025-07-15T10:53:58.512044793Z" level=info msg="starting containerd" revision=9b2ad7760328148397346d10c7b2004271249db4 version=v1.7.23 Jul 15 10:53:58.573432 containerd[1414]: time="2025-07-15T10:53:58.573357175Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.aufs\"..." type=io.containerd.snapshotter.v1 Jul 15 10:53:58.577324 containerd[1414]: time="2025-07-15T10:53:58.577273465Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..." error="aufs is not supported (modprobe aufs failed: exit status 1 \"modprobe: FATAL: Module aufs not found in directory /lib/modules/6.6.98-flatcar\\n\"): skip plugin" type=io.containerd.snapshotter.v1 Jul 15 10:53:58.577324 containerd[1414]: time="2025-07-15T10:53:58.577322395Z" level=info msg="loading plugin \"io.containerd.event.v1.exchange\"..." type=io.containerd.event.v1 Jul 15 10:53:58.577406 containerd[1414]: time="2025-07-15T10:53:58.577347073Z" level=info msg="loading plugin \"io.containerd.internal.v1.opt\"..." type=io.containerd.internal.v1 Jul 15 10:53:58.577607 containerd[1414]: time="2025-07-15T10:53:58.577580422Z" level=info msg="loading plugin \"io.containerd.warning.v1.deprecations\"..." type=io.containerd.warning.v1 Jul 15 10:53:58.577640 containerd[1414]: time="2025-07-15T10:53:58.577609328Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.blockfile\"..." type=io.containerd.snapshotter.v1 Jul 15 10:53:58.577711 containerd[1414]: time="2025-07-15T10:53:58.577684216Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.blockfile\"..." error="no scratch file generator: skip plugin" type=io.containerd.snapshotter.v1 Jul 15 10:53:58.577711 containerd[1414]: time="2025-07-15T10:53:58.577707530Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." type=io.containerd.snapshotter.v1 Jul 15 10:53:58.577992 containerd[1414]: time="2025-07-15T10:53:58.577961580Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs (ext4) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1 Jul 15 10:53:58.577992 containerd[1414]: time="2025-07-15T10:53:58.577989653Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.devmapper\"..." type=io.containerd.snapshotter.v1 Jul 15 10:53:58.578060 containerd[1414]: time="2025-07-15T10:53:58.578008124Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.devmapper\"..." error="devmapper not configured: skip plugin" type=io.containerd.snapshotter.v1 Jul 15 10:53:58.578060 containerd[1414]: time="2025-07-15T10:53:58.578022151Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.native\"..." type=io.containerd.snapshotter.v1 Jul 15 10:53:58.578149 containerd[1414]: time="2025-07-15T10:53:58.578122092Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.overlayfs\"..." type=io.containerd.snapshotter.v1 Jul 15 10:53:58.578403 containerd[1414]: time="2025-07-15T10:53:58.578374288Z" level=info msg="loading plugin \"io.containerd.snapshotter.v1.zfs\"..." type=io.containerd.snapshotter.v1 Jul 15 10:53:58.578580 containerd[1414]: time="2025-07-15T10:53:58.578551607Z" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.zfs\"..." error="path /var/lib/containerd/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1 Jul 15 10:53:58.578580 containerd[1414]: time="2025-07-15T10:53:58.578576796Z" level=info msg="loading plugin \"io.containerd.content.v1.content\"..." type=io.containerd.content.v1 Jul 15 10:53:58.578699 containerd[1414]: time="2025-07-15T10:53:58.578674321Z" level=info msg="loading plugin \"io.containerd.metadata.v1.bolt\"..." type=io.containerd.metadata.v1 Jul 15 10:53:58.578769 containerd[1414]: time="2025-07-15T10:53:58.578744639Z" level=info msg="metadata content store policy set" policy=shared Jul 15 10:53:58.587445 containerd[1414]: time="2025-07-15T10:53:58.586619880Z" level=info msg="loading plugin \"io.containerd.gc.v1.scheduler\"..." type=io.containerd.gc.v1 Jul 15 10:53:58.587445 containerd[1414]: time="2025-07-15T10:53:58.586668049Z" level=info msg="loading plugin \"io.containerd.differ.v1.walking\"..." type=io.containerd.differ.v1 Jul 15 10:53:58.587445 containerd[1414]: time="2025-07-15T10:53:58.586687042Z" level=info msg="loading plugin \"io.containerd.lease.v1.manager\"..." type=io.containerd.lease.v1 Jul 15 10:53:58.587445 containerd[1414]: time="2025-07-15T10:53:58.586707586Z" level=info msg="loading plugin \"io.containerd.streaming.v1.manager\"..." type=io.containerd.streaming.v1 Jul 15 10:53:58.587445 containerd[1414]: time="2025-07-15T10:53:58.586726537Z" level=info msg="loading plugin \"io.containerd.runtime.v1.linux\"..." type=io.containerd.runtime.v1 Jul 15 10:53:58.587445 containerd[1414]: time="2025-07-15T10:53:58.586860153Z" level=info msg="loading plugin \"io.containerd.monitor.v1.cgroups\"..." type=io.containerd.monitor.v1 Jul 15 10:53:58.587445 containerd[1414]: time="2025-07-15T10:53:58.587124627Z" level=info msg="loading plugin \"io.containerd.runtime.v2.task\"..." type=io.containerd.runtime.v2 Jul 15 10:53:58.587445 containerd[1414]: time="2025-07-15T10:53:58.587239259Z" level=info msg="loading plugin \"io.containerd.runtime.v2.shim\"..." type=io.containerd.runtime.v2 Jul 15 10:53:58.587445 containerd[1414]: time="2025-07-15T10:53:58.587261585Z" level=info msg="loading plugin \"io.containerd.sandbox.store.v1.local\"..." type=io.containerd.sandbox.store.v1 Jul 15 10:53:58.587445 containerd[1414]: time="2025-07-15T10:53:58.587279891Z" level=info msg="loading plugin \"io.containerd.sandbox.controller.v1.local\"..." type=io.containerd.sandbox.controller.v1 Jul 15 10:53:58.587445 containerd[1414]: time="2025-07-15T10:53:58.587296488Z" level=info msg="loading plugin \"io.containerd.service.v1.containers-service\"..." type=io.containerd.service.v1 Jul 15 10:53:58.587445 containerd[1414]: time="2025-07-15T10:53:58.587312576Z" level=info msg="loading plugin \"io.containerd.service.v1.content-service\"..." type=io.containerd.service.v1 Jul 15 10:53:58.587445 containerd[1414]: time="2025-07-15T10:53:58.587328081Z" level=info msg="loading plugin \"io.containerd.service.v1.diff-service\"..." type=io.containerd.service.v1 Jul 15 10:53:58.587445 containerd[1414]: time="2025-07-15T10:53:58.587344210Z" level=info msg="loading plugin \"io.containerd.service.v1.images-service\"..." type=io.containerd.service.v1 Jul 15 10:53:58.587821 containerd[1414]: time="2025-07-15T10:53:58.587361068Z" level=info msg="loading plugin \"io.containerd.service.v1.introspection-service\"..." type=io.containerd.service.v1 Jul 15 10:53:58.587821 containerd[1414]: time="2025-07-15T10:53:58.587376229Z" level=info msg="loading plugin \"io.containerd.service.v1.namespaces-service\"..." type=io.containerd.service.v1 Jul 15 10:53:58.587821 containerd[1414]: time="2025-07-15T10:53:58.587390973Z" level=info msg="loading plugin \"io.containerd.service.v1.snapshots-service\"..." type=io.containerd.service.v1 Jul 15 10:53:58.587821 containerd[1414]: time="2025-07-15T10:53:58.587406623Z" level=info msg="loading plugin \"io.containerd.service.v1.tasks-service\"..." type=io.containerd.service.v1 Jul 15 10:53:58.587821 containerd[1414]: time="2025-07-15T10:53:58.587459031Z" level=info msg="loading plugin \"io.containerd.grpc.v1.containers\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.587821 containerd[1414]: time="2025-07-15T10:53:58.587477764Z" level=info msg="loading plugin \"io.containerd.grpc.v1.content\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.587821 containerd[1414]: time="2025-07-15T10:53:58.587492997Z" level=info msg="loading plugin \"io.containerd.grpc.v1.diff\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.587821 containerd[1414]: time="2025-07-15T10:53:58.587510563Z" level=info msg="loading plugin \"io.containerd.grpc.v1.events\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.587821 containerd[1414]: time="2025-07-15T10:53:58.587525745Z" level=info msg="loading plugin \"io.containerd.grpc.v1.images\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.587821 containerd[1414]: time="2025-07-15T10:53:58.587541948Z" level=info msg="loading plugin \"io.containerd.grpc.v1.introspection\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.587821 containerd[1414]: time="2025-07-15T10:53:58.587556286Z" level=info msg="loading plugin \"io.containerd.grpc.v1.leases\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.587821 containerd[1414]: time="2025-07-15T10:53:58.587576070Z" level=info msg="loading plugin \"io.containerd.grpc.v1.namespaces\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.587821 containerd[1414]: time="2025-07-15T10:53:58.587593980Z" level=info msg="loading plugin \"io.containerd.grpc.v1.sandbox-controllers\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.587821 containerd[1414]: time="2025-07-15T10:53:58.587613170Z" level=info msg="loading plugin \"io.containerd.grpc.v1.sandboxes\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.588131 containerd[1414]: time="2025-07-15T10:53:58.587627602Z" level=info msg="loading plugin \"io.containerd.grpc.v1.snapshots\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.588131 containerd[1414]: time="2025-07-15T10:53:58.587642160Z" level=info msg="loading plugin \"io.containerd.grpc.v1.streaming\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.588131 containerd[1414]: time="2025-07-15T10:53:58.587656186Z" level=info msg="loading plugin \"io.containerd.grpc.v1.tasks\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.588131 containerd[1414]: time="2025-07-15T10:53:58.587675491Z" level=info msg="loading plugin \"io.containerd.transfer.v1.local\"..." type=io.containerd.transfer.v1 Jul 15 10:53:58.588131 containerd[1414]: time="2025-07-15T10:53:58.587696899Z" level=info msg="loading plugin \"io.containerd.grpc.v1.transfer\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.588131 containerd[1414]: time="2025-07-15T10:53:58.587713049Z" level=info msg="loading plugin \"io.containerd.grpc.v1.version\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.588131 containerd[1414]: time="2025-07-15T10:53:58.587725753Z" level=info msg="loading plugin \"io.containerd.internal.v1.restart\"..." type=io.containerd.internal.v1 Jul 15 10:53:58.588131 containerd[1414]: time="2025-07-15T10:53:58.587767779Z" level=info msg="loading plugin \"io.containerd.tracing.processor.v1.otlp\"..." type=io.containerd.tracing.processor.v1 Jul 15 10:53:58.588131 containerd[1414]: time="2025-07-15T10:53:58.587785866Z" level=info msg="skip loading plugin \"io.containerd.tracing.processor.v1.otlp\"..." error="skip plugin: tracing endpoint not configured" type=io.containerd.tracing.processor.v1 Jul 15 10:53:58.588131 containerd[1414]: time="2025-07-15T10:53:58.587799788Z" level=info msg="loading plugin \"io.containerd.internal.v1.tracing\"..." type=io.containerd.internal.v1 Jul 15 10:53:58.588131 containerd[1414]: time="2025-07-15T10:53:58.587815323Z" level=info msg="skip loading plugin \"io.containerd.internal.v1.tracing\"..." error="skip plugin: tracing endpoint not configured" type=io.containerd.internal.v1 Jul 15 10:53:58.588131 containerd[1414]: time="2025-07-15T10:53:58.587828120Z" level=info msg="loading plugin \"io.containerd.grpc.v1.healthcheck\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.588131 containerd[1414]: time="2025-07-15T10:53:58.587842345Z" level=info msg="loading plugin \"io.containerd.nri.v1.nri\"..." type=io.containerd.nri.v1 Jul 15 10:53:58.588131 containerd[1414]: time="2025-07-15T10:53:58.587854173Z" level=info msg="NRI interface is disabled by configuration." Jul 15 10:53:58.588457 containerd[1414]: time="2025-07-15T10:53:58.587866252Z" level=info msg="loading plugin \"io.containerd.grpc.v1.cri\"..." type=io.containerd.grpc.v1 Jul 15 10:53:58.588495 containerd[1414]: time="2025-07-15T10:53:58.588161713Z" level=info msg="Start cri plugin with config {PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntimeName:runc DefaultRuntime:{Type: Path: Engine: PodAnnotations:[] ContainerAnnotations:[] Root: Options:map[] PrivilegedWithoutHostDevices:false PrivilegedWithoutHostDevicesAllDevicesAllowed:false BaseRuntimeSpec: NetworkPluginConfDir: NetworkPluginMaxConfNum:0 Snapshotter: SandboxMode:} UntrustedWorkloadRuntime:{Type: Path: Engine: PodAnnotations:[] ContainerAnnotations:[] Root: Options:map[] PrivilegedWithoutHostDevices:false PrivilegedWithoutHostDevicesAllDevicesAllowed:false BaseRuntimeSpec: NetworkPluginConfDir: NetworkPluginMaxConfNum:0 Snapshotter: SandboxMode:} Runtimes:map[runc:{Type:io.containerd.runc.v2 Path: Engine: PodAnnotations:[] ContainerAnnotations:[] Root: Options:map[SystemdCgroup:true] PrivilegedWithoutHostDevices:false PrivilegedWithoutHostDevicesAllDevicesAllowed:false BaseRuntimeSpec: NetworkPluginConfDir: NetworkPluginMaxConfNum:0 Snapshotter: SandboxMode:podsandbox}] NoPivot:false DisableSnapshotAnnotations:true DiscardUnpackedLayers:false IgnoreBlockIONotEnabledErrors:false IgnoreRdtNotEnabledErrors:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginMaxConfNum:1 NetworkPluginSetupSerially:false NetworkPluginConfTemplate: IPPreference:} Registry:{ConfigPath: Mirrors:map[] Configs:map[] Auths:map[] Headers:map[]} ImageDecryption:{KeyModel:node} DisableTCPService:true StreamServerAddress:127.0.0.1 StreamServerPort:0 StreamIdleTimeout:4h0m0s EnableSelinux:true SelinuxCategoryRange:1024 SandboxImage:registry.k8s.io/pause:3.8 StatsCollectPeriod:10 SystemdCgroup:false EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384 DisableCgroup:false DisableApparmor:false RestrictOOMScoreAdj:false MaxConcurrentDownloads:3 DisableProcMount:false UnsetSeccompProfile: TolerateMissingHugetlbController:true DisableHugetlbController:true DeviceOwnershipFromSecurityContext:false IgnoreImageDefinedVolumes:false NetNSMountsUnderStateDir:false EnableUnprivilegedPorts:false EnableUnprivilegedICMP:false EnableCDI:false CDISpecDirs:[/etc/cdi /var/run/cdi] ImagePullProgressTimeout:5m0s DrainExecSyncIOTimeout:0s ImagePullWithSyncFs:false IgnoreDeprecationWarnings:[]} ContainerdRootDir:/var/lib/containerd ContainerdEndpoint:/run/containerd/containerd.sock RootDir:/var/lib/containerd/io.containerd.grpc.v1.cri StateDir:/run/containerd/io.containerd.grpc.v1.cri}" Jul 15 10:53:58.588495 containerd[1414]: time="2025-07-15T10:53:58.588221472Z" level=info msg="Connect containerd service" Jul 15 10:53:58.588495 containerd[1414]: time="2025-07-15T10:53:58.588253928Z" level=info msg="using legacy CRI server" Jul 15 10:53:58.588495 containerd[1414]: time="2025-07-15T10:53:58.588262446Z" level=info msg="using experimental NRI integration - disable nri plugin to prevent this" Jul 15 10:53:58.588495 containerd[1414]: time="2025-07-15T10:53:58.588408183Z" level=info msg="Get image filesystem path \"/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs\"" Jul 15 10:53:58.596464 containerd[1414]: time="2025-07-15T10:53:58.589183317Z" level=error msg="failed to load cni during init, please check CRI plugin status before setting up network for pods" error="cni config load failed: no network config found in /etc/cni/net.d: cni plugin not initialized: failed to load cni config" Jul 15 10:53:58.596464 containerd[1414]: time="2025-07-15T10:53:58.589318234Z" level=info msg="Start subscribing containerd event" Jul 15 10:53:58.596464 containerd[1414]: time="2025-07-15T10:53:58.589360854Z" level=info msg="Start recovering state" Jul 15 10:53:58.596464 containerd[1414]: time="2025-07-15T10:53:58.589441937Z" level=info msg="Start event monitor" Jul 15 10:53:58.596464 containerd[1414]: time="2025-07-15T10:53:58.589463294Z" level=info msg="Start snapshots syncer" Jul 15 10:53:58.596464 containerd[1414]: time="2025-07-15T10:53:58.589473322Z" level=info msg="Start cni network conf syncer for default" Jul 15 10:53:58.596464 containerd[1414]: time="2025-07-15T10:53:58.589482589Z" level=info msg="Start streaming server" Jul 15 10:53:58.596464 containerd[1414]: time="2025-07-15T10:53:58.590293824Z" level=info msg=serving... address=/run/containerd/containerd.sock.ttrpc Jul 15 10:53:58.596464 containerd[1414]: time="2025-07-15T10:53:58.590342732Z" level=info msg=serving... address=/run/containerd/containerd.sock Jul 15 10:53:58.596464 containerd[1414]: time="2025-07-15T10:53:58.590409780Z" level=info msg="containerd successfully booted in 0.079720s" Jul 15 10:53:58.590532 systemd[1]: Started containerd.service - containerd container runtime. Jul 15 10:53:58.644801 systemd-networkd[1328]: eth0: Gained IPv6LL Jul 15 10:53:58.649093 systemd[1]: Finished systemd-networkd-wait-online.service - Wait for Network to be Configured. Jul 15 10:53:58.655864 systemd[1]: Reached target network-online.target - Network is Online. Jul 15 10:53:58.679068 systemd[1]: Starting etcd-member.service - etcd (System Application Container)... Jul 15 10:53:58.695853 systemd[1]: Starting nvidia.service - NVIDIA Configure Service... Jul 15 10:53:58.699271 (-wrapper)[1464]: etcd-member.service: Referenced but unset environment variable evaluates to an empty string: ETCD_OPTS Jul 15 10:53:58.725572 systemd[1]: Finished nvidia.service - NVIDIA Configure Service. Jul 15 10:53:58.745182 sshd_keygen[1404]: ssh-keygen: generating new host keys: RSA ECDSA ED25519 Jul 15 10:53:58.783875 systemd[1]: Finished sshd-keygen.service - Generate sshd host keys. Jul 15 10:53:58.796880 systemd[1]: Starting issuegen.service - Generate /run/issue... Jul 15 10:53:58.808305 systemd[1]: issuegen.service: Deactivated successfully. Jul 15 10:53:58.808646 systemd[1]: Finished issuegen.service - Generate /run/issue. Jul 15 10:53:58.821706 systemd[1]: Starting systemd-user-sessions.service - Permit User Sessions... Jul 15 10:53:58.836302 systemd[1]: Finished systemd-user-sessions.service - Permit User Sessions. Jul 15 10:53:58.846802 systemd[1]: Started getty@tty1.service - Getty on tty1. Jul 15 10:53:58.857687 systemd[1]: Started serial-getty@ttyS0.service - Serial Getty on ttyS0. Jul 15 10:53:58.861191 systemd[1]: Reached target getty.target - Login Prompts. Jul 15 10:53:58.884645 systemd[1]: Created slice system-sshd.slice - Slice /system/sshd. Jul 15 10:53:58.892542 systemd[1]: Started sshd@0-172.24.4.51:22-172.24.4.1:51698.service - OpenSSH per-connection server daemon (172.24.4.1:51698). Jul 15 10:53:58.921583 systemd[1]: Starting docker.service - Docker Application Container Engine... Jul 15 10:53:58.926394 (dockerd)[1501]: docker.service: Referenced but unset environment variable evaluates to an empty string: DOCKER_CGROUPS, DOCKER_OPTS, DOCKER_OPT_BIP, DOCKER_OPT_IPMASQ, DOCKER_OPT_MTU Jul 15 10:53:59.586501 dockerd[1501]: time="2025-07-15T10:53:59.586248895Z" level=info msg="Starting up" Jul 15 10:53:59.890155 systemd[1]: var-lib-docker-metacopy\x2dcheck1312359451-merged.mount: Deactivated successfully. Jul 15 10:53:59.932241 dockerd[1501]: time="2025-07-15T10:53:59.932146475Z" level=info msg="Loading containers: start." Jul 15 10:54:00.093832 sshd[1498]: Accepted publickey for core from 172.24.4.1 port 51698 ssh2: RSA SHA256:yBOpfURoKTwgire6O3HnERbyHyOQL97qx11bEhAl8DU Jul 15 10:54:00.101213 sshd-session[1498]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Jul 15 10:54:00.123969 systemd[1]: Created slice user-500.slice - User Slice of UID 500. Jul 15 10:54:00.145838 systemd[1]: Starting user-runtime-dir@500.service - User Runtime Directory /run/user/500... Jul 15 10:54:00.163479 systemd-logind[1395]: New session 1 of user core. Jul 15 10:54:00.172493 systemd[1]: Finished user-runtime-dir@500.service - User Runtime Directory /run/user/500. Jul 15 10:54:00.183739 systemd[1]: Starting user@500.service - User Manager for UID 500... Jul 15 10:54:00.198868 kernel: Initializing XFRM netlink socket Jul 15 10:54:00.210600 (systemd)[1611]: pam_unix(systemd-user:session): session opened for user core(uid=500) by (uid=0) Jul 15 10:54:00.215131 systemd-logind[1395]: New session c1 of user core. Jul 15 10:54:00.308965 systemd-networkd[1328]: docker0: Link UP Jul 15 10:54:00.348305 dockerd[1501]: time="2025-07-15T10:54:00.347601279Z" level=info msg="Loading containers: done." Jul 15 10:54:00.370990 dockerd[1501]: time="2025-07-15T10:54:00.370947016Z" level=warning msg="Not using native diff for overlay2, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled" storage-driver=overlay2 Jul 15 10:54:00.371601 dockerd[1501]: time="2025-07-15T10:54:00.371234749Z" level=info msg="Docker daemon" commit=41ca978a0a5400cc24b274137efa9f25517fcc0b containerd-snapshotter=false storage-driver=overlay2 version=27.3.1 Jul 15 10:54:00.371601 dockerd[1501]: time="2025-07-15T10:54:00.371372093Z" level=info msg="Daemon has completed initialization" Jul 15 10:54:00.416644 systemd[1611]: Queued start job for default target default.target. Jul 15 10:54:00.421762 dockerd[1501]: time="2025-07-15T10:54:00.418472573Z" level=info msg="API listen on /run/docker.sock" Jul 15 10:54:00.421808 etcd-wrapper[1474]: Error response from daemon: No such container: etcd-member Jul 15 10:54:00.422132 systemd[1]: Started docker.service - Docker Application Container Engine. Jul 15 10:54:00.424122 systemd[1611]: Created slice app.slice - User Application Slice. Jul 15 10:54:00.424148 systemd[1611]: Reached target paths.target - Paths. Jul 15 10:54:00.424198 systemd[1611]: Reached target timers.target - Timers. Jul 15 10:54:00.428514 systemd[1611]: Starting dbus.socket - D-Bus User Message Bus Socket... Jul 15 10:54:00.440639 etcd-wrapper[1687]: Error response from daemon: No such container: etcd-member Jul 15 10:54:00.439669 systemd[1611]: Listening on dbus.socket - D-Bus User Message Bus Socket. Jul 15 10:54:00.441625 systemd[1611]: Reached target sockets.target - Sockets. Jul 15 10:54:00.441681 systemd[1611]: Reached target basic.target - Basic System. Jul 15 10:54:00.441721 systemd[1611]: Reached target default.target - Main User Target. Jul 15 10:54:00.441761 systemd[1611]: Startup finished in 217ms. Jul 15 10:54:00.442123 systemd[1]: Started user@500.service - User Manager for UID 500. Jul 15 10:54:00.454704 systemd[1]: Started session-1.scope - Session 1 of User core. Jul 15 10:54:00.483269 etcd-wrapper[1706]: Unable to find image 'quay.io/coreos/etcd:v3.5.16' locally Jul 15 10:54:00.964156 systemd[1]: Started sshd@1-172.24.4.51:22-172.24.4.1:51710.service - OpenSSH per-connection server daemon (172.24.4.1:51710). Jul 15 10:54:01.933518 etcd-wrapper[1706]: v3.5.16: Pulling from coreos/etcd Jul 15 10:54:02.247143 sshd[1716]: Accepted publickey for core from 172.24.4.1 port 51710 ssh2: RSA SHA256:yBOpfURoKTwgire6O3HnERbyHyOQL97qx11bEhAl8DU Jul 15 10:54:02.252497 sshd-session[1716]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Jul 15 10:54:02.287659 systemd-logind[1395]: New session 2 of user core. Jul 15 10:54:02.300920 systemd[1]: Started session-2.scope - Session 2 of User core. Jul 15 10:54:02.303850 etcd-wrapper[1706]: 804c8aba2cc6: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: 2ae710cd8bfe: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: d462aa345367: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: 0f8b424aa0b9: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: d557676654e5: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: c8022d07192e: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: d858cbc252ad: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: 1069fc2daed1: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: b40161cd83fc: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: 5318d93a3a65: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: 307c1adadb60: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: fbb01d9e9dc9: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: fbfea02ac3cf: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: 8c26e4bf18e2: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: 1e59a65f8816: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: ffbd4ca5f0bd: Pulling fs layer Jul 15 10:54:02.303850 etcd-wrapper[1706]: 0f8b424aa0b9: Waiting Jul 15 10:54:02.303850 etcd-wrapper[1706]: d557676654e5: Waiting Jul 15 10:54:02.303850 etcd-wrapper[1706]: c8022d07192e: Waiting Jul 15 10:54:02.303850 etcd-wrapper[1706]: 5318d93a3a65: Waiting Jul 15 10:54:02.303850 etcd-wrapper[1706]: 307c1adadb60: Waiting Jul 15 10:54:02.303850 etcd-wrapper[1706]: d858cbc252ad: Waiting Jul 15 10:54:02.303850 etcd-wrapper[1706]: 1069fc2daed1: Waiting Jul 15 10:54:02.303850 etcd-wrapper[1706]: fbb01d9e9dc9: Waiting Jul 15 10:54:02.303850 etcd-wrapper[1706]: b40161cd83fc: Waiting Jul 15 10:54:02.303850 etcd-wrapper[1706]: fbfea02ac3cf: Waiting Jul 15 10:54:02.303850 etcd-wrapper[1706]: 1e59a65f8816: Waiting Jul 15 10:54:02.303850 etcd-wrapper[1706]: ffbd4ca5f0bd: Waiting Jul 15 10:54:02.303850 etcd-wrapper[1706]: 8c26e4bf18e2: Waiting Jul 15 10:54:02.631261 etcd-wrapper[1706]: 804c8aba2cc6: Download complete Jul 15 10:54:02.695217 etcd-wrapper[1706]: 2ae710cd8bfe: Verifying Checksum Jul 15 10:54:02.695217 etcd-wrapper[1706]: 2ae710cd8bfe: Download complete Jul 15 10:54:02.719472 etcd-wrapper[1706]: 804c8aba2cc6: Pull complete Jul 15 10:54:02.767730 systemd[1]: var-lib-docker-overlay2-c35d23875d157bc1657c0dee6147c3391f73a4c5de7e43c4450ac97413e4b70a-merged.mount: Deactivated successfully. Jul 15 10:54:02.796675 etcd-wrapper[1706]: 2ae710cd8bfe: Pull complete Jul 15 10:54:02.808107 etcd-wrapper[1706]: d462aa345367: Download complete Jul 15 10:54:02.926965 etcd-wrapper[1706]: 0f8b424aa0b9: Verifying Checksum Jul 15 10:54:02.926965 etcd-wrapper[1706]: 0f8b424aa0b9: Download complete Jul 15 10:54:03.014830 sshd[1718]: Connection closed by 172.24.4.1 port 51710 Jul 15 10:54:03.015791 sshd-session[1716]: pam_unix(sshd:session): session closed for user core Jul 15 10:54:03.026753 etcd-wrapper[1706]: d557676654e5: Verifying Checksum Jul 15 10:54:03.026753 etcd-wrapper[1706]: d557676654e5: Download complete Jul 15 10:54:03.027970 systemd[1]: sshd@1-172.24.4.51:22-172.24.4.1:51710.service: Deactivated successfully. Jul 15 10:54:03.029935 systemd[1]: session-2.scope: Deactivated successfully. Jul 15 10:54:03.032023 systemd-logind[1395]: Session 2 logged out. Waiting for processes to exit. Jul 15 10:54:03.039206 systemd[1]: Started sshd@2-172.24.4.51:22-172.24.4.1:51714.service - OpenSSH per-connection server daemon (172.24.4.1:51714). Jul 15 10:54:03.046603 systemd-logind[1395]: Removed session 2. Jul 15 10:54:03.099040 etcd-wrapper[1706]: c8022d07192e: Verifying Checksum Jul 15 10:54:03.099307 etcd-wrapper[1706]: c8022d07192e: Download complete Jul 15 10:54:03.207803 etcd-wrapper[1706]: d858cbc252ad: Verifying Checksum Jul 15 10:54:03.208010 etcd-wrapper[1706]: d858cbc252ad: Download complete Jul 15 10:54:03.304718 etcd-wrapper[1706]: 1069fc2daed1: Verifying Checksum Jul 15 10:54:03.304925 etcd-wrapper[1706]: 1069fc2daed1: Download complete Jul 15 10:54:03.350246 systemd[1]: var-lib-docker-overlay2-2f60553695828758b948fa4fcf0f34964e113c51d3367b5eef3998476fd0ca59-merged.mount: Deactivated successfully. Jul 15 10:54:03.384931 etcd-wrapper[1706]: b40161cd83fc: Download complete Jul 15 10:54:03.425515 etcd-wrapper[1706]: d462aa345367: Pull complete Jul 15 10:54:03.480001 etcd-wrapper[1706]: 0f8b424aa0b9: Pull complete Jul 15 10:54:03.510491 etcd-wrapper[1706]: 5318d93a3a65: Verifying Checksum Jul 15 10:54:03.510491 etcd-wrapper[1706]: 5318d93a3a65: Download complete Jul 15 10:54:03.517600 etcd-wrapper[1706]: d557676654e5: Pull complete Jul 15 10:54:03.552882 etcd-wrapper[1706]: c8022d07192e: Pull complete Jul 15 10:54:03.581866 etcd-wrapper[1706]: 307c1adadb60: Verifying Checksum Jul 15 10:54:03.581866 etcd-wrapper[1706]: 307c1adadb60: Download complete Jul 15 10:54:03.595262 etcd-wrapper[1706]: d858cbc252ad: Pull complete Jul 15 10:54:03.634498 etcd-wrapper[1706]: 1069fc2daed1: Pull complete Jul 15 10:54:03.674020 etcd-wrapper[1706]: b40161cd83fc: Pull complete Jul 15 10:54:03.729452 etcd-wrapper[1706]: 5318d93a3a65: Pull complete Jul 15 10:54:03.748723 systemd[1]: var-lib-docker-overlay2-bf21d8fbe13156eead69f7eb904097e73682afa834ea1ededee726a07bb628bf-merged.mount: Deactivated successfully. Jul 15 10:54:03.775191 systemd[1]: var-lib-docker-overlay2-dfe802131f4569cc688f2e40193e954e00b79ad66aadafa0c19722e9c468151f-merged.mount: Deactivated successfully. Jul 15 10:54:03.819650 etcd-wrapper[1706]: 307c1adadb60: Pull complete Jul 15 10:54:03.929892 login[1495]: pam_unix(login:session): session opened for user core(uid=500) by LOGIN(uid=0) Jul 15 10:54:03.936607 login[1496]: pam_unix(login:session): session opened for user core(uid=500) by LOGIN(uid=0) Jul 15 10:54:03.944498 systemd-logind[1395]: New session 3 of user core. Jul 15 10:54:03.950609 systemd[1]: Started session-3.scope - Session 3 of User core. Jul 15 10:54:03.958165 systemd-logind[1395]: New session 4 of user core. Jul 15 10:54:03.961578 systemd[1]: Started session-4.scope - Session 4 of User core. Jul 15 10:54:04.065761 etcd-wrapper[1706]: fbb01d9e9dc9: Verifying Checksum Jul 15 10:54:04.065761 etcd-wrapper[1706]: fbb01d9e9dc9: Download complete Jul 15 10:54:04.075213 etcd-wrapper[1706]: fbfea02ac3cf: Verifying Checksum Jul 15 10:54:04.075213 etcd-wrapper[1706]: fbfea02ac3cf: Download complete Jul 15 10:54:04.184015 etcd-wrapper[1706]: 8c26e4bf18e2: Verifying Checksum Jul 15 10:54:04.184171 etcd-wrapper[1706]: 8c26e4bf18e2: Download complete Jul 15 10:54:04.350010 etcd-wrapper[1706]: ffbd4ca5f0bd: Verifying Checksum Jul 15 10:54:04.350010 etcd-wrapper[1706]: ffbd4ca5f0bd: Download complete Jul 15 10:54:04.463344 etcd-wrapper[1706]: 1e59a65f8816: Verifying Checksum Jul 15 10:54:04.463344 etcd-wrapper[1706]: 1e59a65f8816: Download complete Jul 15 10:54:04.587742 etcd-wrapper[1706]: fbb01d9e9dc9: Pull complete Jul 15 10:54:04.679620 sshd[1740]: Accepted publickey for core from 172.24.4.1 port 51714 ssh2: RSA SHA256:yBOpfURoKTwgire6O3HnERbyHyOQL97qx11bEhAl8DU Jul 15 10:54:04.683918 sshd-session[1740]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Jul 15 10:54:04.691287 systemd-logind[1395]: New session 5 of user core. Jul 15 10:54:04.697293 systemd[1]: Started session-5.scope - Session 5 of User core. Jul 15 10:54:04.845335 systemd[1]: var-lib-docker-overlay2-8c7178880ff72d45262391ff48c9963c6a149feb57724aff02c88d8aebf754f3-merged.mount: Deactivated successfully. Jul 15 10:54:05.011561 etcd-wrapper[1706]: fbfea02ac3cf: Pull complete Jul 15 10:54:05.282159 systemd[1]: var-lib-docker-overlay2-b91f925a4fa63cbaa0d72f3278861fad8b207ff58c2e466895affc66ef0c29ca-merged.mount: Deactivated successfully. Jul 15 10:54:05.340315 sshd[1818]: Connection closed by 172.24.4.1 port 51714 Jul 15 10:54:05.341357 sshd-session[1740]: pam_unix(sshd:session): session closed for user core Jul 15 10:54:05.346978 systemd[1]: sshd@2-172.24.4.51:22-172.24.4.1:51714.service: Deactivated successfully. Jul 15 10:54:05.350556 systemd[1]: session-5.scope: Deactivated successfully. Jul 15 10:54:05.353993 systemd-logind[1395]: Session 5 logged out. Waiting for processes to exit. Jul 15 10:54:05.356872 systemd-logind[1395]: Removed session 5. Jul 15 10:54:05.403638 etcd-wrapper[1706]: 8c26e4bf18e2: Pull complete Jul 15 10:54:05.469791 etcd-wrapper[1706]: 1e59a65f8816: Pull complete Jul 15 10:54:05.553997 etcd-wrapper[1706]: ffbd4ca5f0bd: Pull complete Jul 15 10:54:05.576153 etcd-wrapper[1706]: Digest: sha256:d967d98a12dc220a1a290794711dba7eba04b8ce465e12b02383d1bfbb33e159 Jul 15 10:54:05.584042 etcd-wrapper[1706]: Status: Downloaded newer image for quay.io/coreos/etcd:v3.5.16 Jul 15 10:54:05.770710 systemd[1]: var-lib-docker-overlay2-a25b2f091ce2beec11484bedaae8dca1a1e3747b605974491fd5d8ed063b9e18-merged.mount: Deactivated successfully. Jul 15 10:54:05.998093 containerd[1414]: time="2025-07-15T10:54:05.997446487Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1 Jul 15 10:54:05.998093 containerd[1414]: time="2025-07-15T10:54:05.997548944Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1 Jul 15 10:54:05.998093 containerd[1414]: time="2025-07-15T10:54:05.997575823Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1 Jul 15 10:54:05.998093 containerd[1414]: time="2025-07-15T10:54:05.997701699Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1 Jul 15 10:54:06.032829 systemd[1]: run-docker-runtime\x2drunc-moby-bf5ae86e3b503c7ebbdf69cbc57270ee2f6cec5b8242b443fe03784f780fb1a6-runc.bOxh9K.mount: Deactivated successfully. Jul 15 10:54:06.047813 systemd[1]: Started docker-bf5ae86e3b503c7ebbdf69cbc57270ee2f6cec5b8242b443fe03784f780fb1a6.scope - libcontainer container bf5ae86e3b503c7ebbdf69cbc57270ee2f6cec5b8242b443fe03784f780fb1a6. Jul 15 10:54:06.167890 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.166233Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_DATA_DIR","variable-value":"/var/lib/etcd"} Jul 15 10:54:06.167890 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.166448Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_NAME","variable-value":"360f0ca0f20744b8ad97daf7d2f8a5e0"} Jul 15 10:54:06.167890 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:54:06.166490Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_IMAGE_URL=quay.io/coreos/etcd"} Jul 15 10:54:06.167890 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:54:06.166508Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_IMAGE_TAG=v3.5.16"} Jul 15 10:54:06.167890 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:54:06.166516Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_USER=etcd"} Jul 15 10:54:06.167890 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:54:06.166525Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_SSL_DIR=/etc/ssl/certs"} Jul 15 10:54:06.167890 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:54:06.166628Z","caller":"embed/config.go:689","msg":"Running http and grpc server on single port. This is not recommended for production."} Jul 15 10:54:06.167890 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.166687Z","caller":"etcdmain/etcd.go:73","msg":"Running: ","args":["/usr/local/bin/etcd","--listen-client-urls=http://0.0.0.0:2379","--advertise-client-urls=http://172.24.4.51:2379"]} Jul 15 10:54:06.167890 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:54:06.166774Z","caller":"embed/config.go:689","msg":"Running http and grpc server on single port. This is not recommended for production."} Jul 15 10:54:06.167890 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.166792Z","caller":"embed/etcd.go:128","msg":"configuring peer listeners","listen-peer-urls":["http://localhost:2380"]} Jul 15 10:54:06.169505 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.167955Z","caller":"embed/etcd.go:136","msg":"configuring client listeners","listen-client-urls":["http://0.0.0.0:2379"]} Jul 15 10:54:06.169505 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.168207Z","caller":"embed/etcd.go:311","msg":"starting an etcd server","etcd-version":"3.5.16","git-sha":"f20bbad","go-version":"go1.22.7","go-os":"linux","go-arch":"amd64","max-cpu-set":2,"max-cpu-available":2,"member-initialized":false,"name":"360f0ca0f20744b8ad97daf7d2f8a5e0","data-dir":"/var/lib/etcd","wal-dir":"","wal-dir-dedicated":"","member-dir":"/var/lib/etcd/member","force-new-cluster":false,"heartbeat-interval":"100ms","election-timeout":"1s","initial-election-tick-advance":true,"snapshot-count":100000,"max-wals":5,"max-snapshots":5,"snapshot-catchup-entries":5000,"initial-advertise-peer-urls":["http://localhost:2380"],"listen-peer-urls":["http://localhost:2380"],"advertise-client-urls":["http://172.24.4.51:2379"],"listen-client-urls":["http://0.0.0.0:2379"],"listen-metrics-urls":[],"cors":["*"],"host-whitelist":["*"],"initial-cluster":"360f0ca0f20744b8ad97daf7d2f8a5e0=http://localhost:2380","initial-cluster-state":"new","initial-cluster-token":"etcd-cluster","quota-backend-bytes":2147483648,"max-request-bytes":1572864,"max-concurrent-streams":4294967295,"pre-vote":true,"initial-corrupt-check":false,"corrupt-check-time-interval":"0s","compact-check-time-enabled":false,"compact-check-time-interval":"1m0s","auto-compaction-mode":"periodic","auto-compaction-retention":"0s","auto-compaction-interval":"0s","discovery-url":"","discovery-proxy":"","downgrade-check-interval":"5s"} Jul 15 10:54:06.181842 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.181488Z","caller":"etcdserver/backend.go:81","msg":"opened backend db","path":"/var/lib/etcd/member/snap/db","took":"12.533411ms"} Jul 15 10:54:06.195054 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.194690Z","caller":"etcdserver/raft.go:505","msg":"starting local member","local-member-id":"8e9e05c52164694d","cluster-id":"cdf818194e3a8c32"} Jul 15 10:54:06.195331 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.195126Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d switched to configuration voters=()"} Jul 15 10:54:06.195649 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.195370Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d became follower at term 0"} Jul 15 10:54:06.195992 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.195717Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"newRaft 8e9e05c52164694d [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]"} Jul 15 10:54:06.196218 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.196026Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d became follower at term 1"} Jul 15 10:54:06.196534 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.196329Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d switched to configuration voters=(10276657743932975437)"} Jul 15 10:54:06.209770 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:54:06.209289Z","caller":"auth/store.go:1241","msg":"simple token is not cryptographically signed"} Jul 15 10:54:06.213100 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.212602Z","caller":"mvcc/kvstore.go:423","msg":"kvstore restored","current-rev":1} Jul 15 10:54:06.215116 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.214691Z","caller":"etcdserver/quota.go:94","msg":"enabled backend quota with default value","quota-name":"v3-applier","quota-size-bytes":2147483648,"quota-size":"2.1 GB"} Jul 15 10:54:06.218335 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.217274Z","caller":"etcdserver/server.go:873","msg":"starting etcd server","local-member-id":"8e9e05c52164694d","local-server-version":"3.5.16","cluster-version":"to_be_decided"} Jul 15 10:54:06.219883 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.218049Z","caller":"v3rpc/health.go:61","msg":"grpc service status changed","service":"","status":"SERVING"} Jul 15 10:54:06.220209 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.218649Z","caller":"etcdserver/server.go:757","msg":"started as single-node; fast-forwarding election ticks","local-member-id":"8e9e05c52164694d","forward-ticks":9,"forward-duration":"900ms","election-ticks":10,"election-timeout":"1s"} Jul 15 10:54:06.220209 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.218886Z","caller":"fileutil/purge.go:50","msg":"started to purge file","dir":"/var/lib/etcd/member/snap","suffix":"snap.db","max":5,"interval":"30s"} Jul 15 10:54:06.220209 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.218932Z","caller":"fileutil/purge.go:50","msg":"started to purge file","dir":"/var/lib/etcd/member/snap","suffix":"snap","max":5,"interval":"30s"} Jul 15 10:54:06.220209 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.218954Z","caller":"fileutil/purge.go:50","msg":"started to purge file","dir":"/var/lib/etcd/member/wal","suffix":"wal","max":5,"interval":"30s"} Jul 15 10:54:06.220209 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.219305Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d switched to configuration voters=(10276657743932975437)"} Jul 15 10:54:06.220209 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.219482Z","caller":"membership/cluster.go:421","msg":"added member","cluster-id":"cdf818194e3a8c32","local-member-id":"8e9e05c52164694d","added-peer-id":"8e9e05c52164694d","added-peer-peer-urls":["http://localhost:2380"]} Jul 15 10:54:06.225933 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.225575Z","caller":"embed/etcd.go:280","msg":"now serving peer/client/metrics","local-member-id":"8e9e05c52164694d","initial-advertise-peer-urls":["http://localhost:2380"],"listen-peer-urls":["http://localhost:2380"],"advertise-client-urls":["http://172.24.4.51:2379"],"listen-client-urls":["http://0.0.0.0:2379"],"listen-metrics-urls":[]} Jul 15 10:54:06.226322 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.225956Z","caller":"embed/etcd.go:600","msg":"serving peer traffic","address":"127.0.0.1:2380"} Jul 15 10:54:06.226322 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.225994Z","caller":"embed/etcd.go:572","msg":"cmux::serve","address":"127.0.0.1:2380"} Jul 15 10:54:06.398780 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.397352Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d is starting a new election at term 1"} Jul 15 10:54:06.398780 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.397452Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d became pre-candidate at term 1"} Jul 15 10:54:06.398780 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.397510Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d received MsgPreVoteResp from 8e9e05c52164694d at term 1"} Jul 15 10:54:06.398780 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.397540Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d became candidate at term 2"} Jul 15 10:54:06.398780 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.397555Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 2"} Jul 15 10:54:06.398780 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.397577Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"8e9e05c52164694d became leader at term 2"} Jul 15 10:54:06.398780 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.397594Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 2"} Jul 15 10:54:06.401914 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.398897Z","caller":"etcdserver/server.go:2651","msg":"setting up initial cluster version using v2 API","cluster-version":"3.5"} Jul 15 10:54:06.401914 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.399003Z","caller":"etcdserver/server.go:2140","msg":"published local member to cluster through raft","local-member-id":"8e9e05c52164694d","local-member-attributes":"{Name:360f0ca0f20744b8ad97daf7d2f8a5e0 ClientURLs:[http://172.24.4.51:2379]}","request-path":"/0/members/8e9e05c52164694d/attributes","cluster-id":"cdf818194e3a8c32","publish-timeout":"7s"} Jul 15 10:54:06.401914 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.399980Z","caller":"embed/serve.go:103","msg":"ready to serve client requests"} Jul 15 10:54:06.406354 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.401914Z","caller":"etcdmain/main.go:44","msg":"notifying init daemon"} Jul 15 10:54:06.406354 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.402906Z","caller":"etcdmain/main.go:50","msg":"successfully notified init daemon"} Jul 15 10:54:06.406354 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.403347Z","caller":"membership/cluster.go:584","msg":"set initial cluster version","cluster-id":"cdf818194e3a8c32","local-member-id":"8e9e05c52164694d","cluster-version":"3.5"} Jul 15 10:54:06.406354 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.403527Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"} Jul 15 10:54:06.406354 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.403573Z","caller":"etcdserver/server.go:2675","msg":"cluster version is updated","cluster-version":"3.5"} Jul 15 10:54:06.406354 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.403868Z","caller":"v3rpc/health.go:61","msg":"grpc service status changed","service":"","status":"SERVING"} Jul 15 10:54:06.405775 systemd[1]: Started etcd-member.service - etcd (System Application Container). Jul 15 10:54:06.409513 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:54:06.408227Z","caller":"embed/serve.go:187","msg":"serving client traffic insecurely; this is strongly discouraged!","traffic":"grpc+http","address":"[::]:2379"} Jul 15 10:54:06.406939 systemd[1]: Reached target multi-user.target - Multi-User System. Jul 15 10:54:06.408742 systemd[1]: Startup finished in 1.239s (kernel) + 11.783s (initrd) + 11.865s (userspace) = 24.888s. Jul 15 10:54:15.474195 systemd[1]: Started sshd@3-172.24.4.51:22-172.24.4.1:35656.service - OpenSSH per-connection server daemon (172.24.4.1:35656). Jul 15 10:54:16.601816 sshd[1886]: Accepted publickey for core from 172.24.4.1 port 35656 ssh2: RSA SHA256:yBOpfURoKTwgire6O3HnERbyHyOQL97qx11bEhAl8DU Jul 15 10:54:16.605488 sshd-session[1886]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Jul 15 10:54:16.622543 systemd-logind[1395]: New session 6 of user core. Jul 15 10:54:16.627262 systemd[1]: Started session-6.scope - Session 6 of User core. Jul 15 10:54:17.329490 sshd[1888]: Connection closed by 172.24.4.1 port 35656 Jul 15 10:54:17.332310 sshd-session[1886]: pam_unix(sshd:session): session closed for user core Jul 15 10:54:17.347862 systemd[1]: sshd@3-172.24.4.51:22-172.24.4.1:35656.service: Deactivated successfully. Jul 15 10:54:17.352145 systemd[1]: session-6.scope: Deactivated successfully. Jul 15 10:54:17.356786 systemd-logind[1395]: Session 6 logged out. Waiting for processes to exit. Jul 15 10:54:17.365048 systemd[1]: Started sshd@4-172.24.4.51:22-172.24.4.1:35670.service - OpenSSH per-connection server daemon (172.24.4.1:35670). Jul 15 10:54:17.368244 systemd-logind[1395]: Removed session 6. Jul 15 10:54:18.487697 sshd[1893]: Accepted publickey for core from 172.24.4.1 port 35670 ssh2: RSA SHA256:yBOpfURoKTwgire6O3HnERbyHyOQL97qx11bEhAl8DU Jul 15 10:54:18.490931 sshd-session[1893]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Jul 15 10:54:18.502828 systemd-logind[1395]: New session 7 of user core. Jul 15 10:54:18.515796 systemd[1]: Started session-7.scope - Session 7 of User core. Jul 15 10:54:19.102432 sshd[1896]: Connection closed by 172.24.4.1 port 35670 Jul 15 10:54:19.103794 sshd-session[1893]: pam_unix(sshd:session): session closed for user core Jul 15 10:54:19.123095 systemd[1]: sshd@4-172.24.4.51:22-172.24.4.1:35670.service: Deactivated successfully. Jul 15 10:54:19.127669 systemd[1]: session-7.scope: Deactivated successfully. Jul 15 10:54:19.129914 systemd-logind[1395]: Session 7 logged out. Waiting for processes to exit. Jul 15 10:54:19.143191 systemd[1]: Started sshd@5-172.24.4.51:22-172.24.4.1:35678.service - OpenSSH per-connection server daemon (172.24.4.1:35678). Jul 15 10:54:19.146570 systemd-logind[1395]: Removed session 7. Jul 15 10:54:20.244599 sshd[1901]: Accepted publickey for core from 172.24.4.1 port 35678 ssh2: RSA SHA256:yBOpfURoKTwgire6O3HnERbyHyOQL97qx11bEhAl8DU Jul 15 10:54:20.248879 sshd-session[1901]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Jul 15 10:54:20.270537 systemd-logind[1395]: New session 8 of user core. Jul 15 10:54:20.298794 systemd[1]: Started session-8.scope - Session 8 of User core. Jul 15 10:54:20.976214 sshd[1904]: Connection closed by 172.24.4.1 port 35678 Jul 15 10:54:20.977814 sshd-session[1901]: pam_unix(sshd:session): session closed for user core Jul 15 10:54:20.998013 systemd[1]: sshd@5-172.24.4.51:22-172.24.4.1:35678.service: Deactivated successfully. Jul 15 10:54:21.004717 systemd[1]: session-8.scope: Deactivated successfully. Jul 15 10:54:21.010827 systemd-logind[1395]: Session 8 logged out. Waiting for processes to exit. Jul 15 10:54:21.020299 systemd[1]: Started sshd@6-172.24.4.51:22-172.24.4.1:35692.service - OpenSSH per-connection server daemon (172.24.4.1:35692). Jul 15 10:54:21.024076 systemd-logind[1395]: Removed session 8. Jul 15 10:54:22.188641 sshd[1909]: Accepted publickey for core from 172.24.4.1 port 35692 ssh2: RSA SHA256:yBOpfURoKTwgire6O3HnERbyHyOQL97qx11bEhAl8DU Jul 15 10:54:22.191023 sshd-session[1909]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Jul 15 10:54:22.203414 systemd-logind[1395]: New session 9 of user core. Jul 15 10:54:22.219849 systemd[1]: Started session-9.scope - Session 9 of User core. Jul 15 10:54:22.662226 sudo[1913]: core : PWD=/home/core ; USER=root ; COMMAND=/usr/sbin/setenforce 1 Jul 15 10:54:22.663999 sudo[1913]: pam_unix(sudo:session): session opened for user root(uid=0) by core(uid=500) Jul 15 10:54:22.688566 sudo[1913]: pam_unix(sudo:session): session closed for user root Jul 15 10:54:22.959652 sshd[1912]: Connection closed by 172.24.4.1 port 35692 Jul 15 10:54:22.960590 sshd-session[1909]: pam_unix(sshd:session): session closed for user core Jul 15 10:54:22.974290 systemd[1]: sshd@6-172.24.4.51:22-172.24.4.1:35692.service: Deactivated successfully. Jul 15 10:54:22.978305 systemd[1]: session-9.scope: Deactivated successfully. Jul 15 10:54:22.980831 systemd-logind[1395]: Session 9 logged out. Waiting for processes to exit. Jul 15 10:54:22.995045 systemd[1]: Started sshd@7-172.24.4.51:22-172.24.4.1:35700.service - OpenSSH per-connection server daemon (172.24.4.1:35700). Jul 15 10:54:23.000207 systemd-logind[1395]: Removed session 9. Jul 15 10:54:23.937352 sshd[1918]: Accepted publickey for core from 172.24.4.1 port 35700 ssh2: RSA SHA256:yBOpfURoKTwgire6O3HnERbyHyOQL97qx11bEhAl8DU Jul 15 10:54:23.941634 sshd-session[1918]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Jul 15 10:54:23.953917 systemd-logind[1395]: New session 10 of user core. Jul 15 10:54:23.966923 systemd[1]: Started session-10.scope - Session 10 of User core. Jul 15 10:54:24.377942 sudo[1923]: core : PWD=/home/core ; USER=root ; COMMAND=/usr/sbin/rm -rf /etc/audit/rules.d/80-selinux.rules /etc/audit/rules.d/99-default.rules Jul 15 10:54:24.378938 sudo[1923]: pam_unix(sudo:session): session opened for user root(uid=0) by core(uid=500) Jul 15 10:54:24.391208 sudo[1923]: pam_unix(sudo:session): session closed for user root Jul 15 10:54:24.410057 sudo[1922]: core : PWD=/home/core ; USER=root ; COMMAND=/usr/sbin/systemctl restart audit-rules Jul 15 10:54:24.410857 sudo[1922]: pam_unix(sudo:session): session opened for user root(uid=0) by core(uid=500) Jul 15 10:54:24.442734 systemd[1]: Starting audit-rules.service - Load Audit Rules... Jul 15 10:54:24.527448 augenrules[1945]: No rules Jul 15 10:54:24.529589 systemd[1]: audit-rules.service: Deactivated successfully. Jul 15 10:54:24.530357 systemd[1]: Finished audit-rules.service - Load Audit Rules. Jul 15 10:54:24.533041 sudo[1922]: pam_unix(sudo:session): session closed for user root Jul 15 10:54:24.707629 sshd[1921]: Connection closed by 172.24.4.1 port 35700 Jul 15 10:54:24.709098 sshd-session[1918]: pam_unix(sshd:session): session closed for user core Jul 15 10:54:24.728979 systemd[1]: sshd@7-172.24.4.51:22-172.24.4.1:35700.service: Deactivated successfully. Jul 15 10:54:24.733943 systemd[1]: session-10.scope: Deactivated successfully. Jul 15 10:54:24.738734 systemd-logind[1395]: Session 10 logged out. Waiting for processes to exit. Jul 15 10:54:24.746993 systemd[1]: Started sshd@8-172.24.4.51:22-172.24.4.1:53214.service - OpenSSH per-connection server daemon (172.24.4.1:53214). Jul 15 10:54:24.750624 systemd-logind[1395]: Removed session 10. Jul 15 10:54:25.688763 sshd[1953]: Accepted publickey for core from 172.24.4.1 port 53214 ssh2: RSA SHA256:yBOpfURoKTwgire6O3HnERbyHyOQL97qx11bEhAl8DU Jul 15 10:54:25.691720 sshd-session[1953]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Jul 15 10:54:25.706256 systemd-logind[1395]: New session 11 of user core. Jul 15 10:54:25.714725 systemd[1]: Started session-11.scope - Session 11 of User core. Jul 15 10:54:26.322455 sshd[1956]: Connection closed by 172.24.4.1 port 53214 Jul 15 10:54:26.328460 sshd-session[1953]: pam_unix(sshd:session): session closed for user core Jul 15 10:54:26.343288 systemd[1]: sshd@8-172.24.4.51:22-172.24.4.1:53214.service: Deactivated successfully. Jul 15 10:54:26.347170 systemd[1]: session-11.scope: Deactivated successfully. Jul 15 10:54:26.349301 systemd-logind[1395]: Session 11 logged out. Waiting for processes to exit. Jul 15 10:54:26.359953 systemd[1]: Started sshd@9-172.24.4.51:22-172.24.4.1:53218.service - OpenSSH per-connection server daemon (172.24.4.1:53218). Jul 15 10:54:26.362033 systemd-logind[1395]: Removed session 11. Jul 15 10:54:27.482232 sshd[1963]: Accepted publickey for core from 172.24.4.1 port 53218 ssh2: RSA SHA256:yBOpfURoKTwgire6O3HnERbyHyOQL97qx11bEhAl8DU Jul 15 10:54:27.486277 sshd-session[1963]: pam_unix(sshd:session): session opened for user core(uid=500) by core(uid=0) Jul 15 10:54:27.510726 systemd-logind[1395]: New session 12 of user core. Jul 15 10:54:27.523754 systemd[1]: Started session-12.scope - Session 12 of User core. Jul 15 10:54:28.348528 sshd[1966]: Connection closed by 172.24.4.1 port 53218 Jul 15 10:54:28.349815 sshd-session[1963]: pam_unix(sshd:session): session closed for user core Jul 15 10:54:28.361261 systemd[1]: sshd@9-172.24.4.51:22-172.24.4.1:53218.service: Deactivated successfully. Jul 15 10:54:28.368633 systemd[1]: session-12.scope: Deactivated successfully. Jul 15 10:54:28.370983 systemd-logind[1395]: Session 12 logged out. Waiting for processes to exit. Jul 15 10:54:28.374028 systemd-logind[1395]: Removed session 12. Jul 15 10:54:43.537519 update_engine[1397]: I20250715 10:54:43.536626 1397 update_attempter.cc:509] Updating boot flags... Jul 15 10:54:43.705919 kernel: BTRFS warning: duplicate device /dev/vda3 devid 1 generation 36 scanned by (udev-worker) (1983) Jul 15 10:54:43.798441 kernel: BTRFS warning: duplicate device /dev/vda3 devid 1 generation 36 scanned by (udev-worker) (1980) Jul 15 10:56:16.151997 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:16.144940Z","caller":"traceutil/trace.go:171","msg":"trace[1920828919] transaction","detail":"{read_only:false; response_revision:243; number_of_response:1; }","duration":"170.36931ms","start":"2025-07-15T10:56:15.974508Z","end":"2025-07-15T10:56:16.144878Z","steps":["trace[1920828919] 'process raft request' (duration: 170.108543ms)"],"step_count":1} Jul 15 10:56:16.400857 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:16.398573Z","caller":"traceutil/trace.go:171","msg":"trace[1628918588] transaction","detail":"{read_only:false; response_revision:244; number_of_response:1; }","duration":"242.596933ms","start":"2025-07-15T10:56:16.155936Z","end":"2025-07-15T10:56:16.398533Z","steps":["trace[1628918588] 'process raft request' (duration: 242.349551ms)"],"step_count":1} Jul 15 10:56:16.588779 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:16.583315Z","caller":"traceutil/trace.go:171","msg":"trace[1635095127] linearizableReadLoop","detail":"{readStateIndex:253; appliedIndex:252; }","duration":"136.942466ms","start":"2025-07-15T10:56:16.446302Z","end":"2025-07-15T10:56:16.583244Z","steps":["trace[1635095127] 'read index received' (duration: 45.171653ms)","trace[1635095127] 'applied index is now lower than readState.Index' (duration: 91.76951ms)"],"step_count":2} Jul 15 10:56:16.588779 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:16.583342Z","caller":"traceutil/trace.go:171","msg":"trace[2039752838] transaction","detail":"{read_only:false; response_revision:246; number_of_response:1; }","duration":"139.14939ms","start":"2025-07-15T10:56:16.444126Z","end":"2025-07-15T10:56:16.583275Z","steps":["trace[2039752838] 'process raft request' (duration: 47.393144ms)","trace[2039752838] 'compare' (duration: 91.579605ms)"],"step_count":2} Jul 15 10:56:16.599250 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:16.584028Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"137.577592ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/namespaces/kube-system\" ","response":"range_response_count:1 size:351"} Jul 15 10:56:16.599250 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:16.588483Z","caller":"traceutil/trace.go:171","msg":"trace[947144259] range","detail":"{range_begin:/registry/namespaces/kube-system; range_end:; response_count:1; response_revision:246; }","duration":"142.17338ms","start":"2025-07-15T10:56:16.446260Z","end":"2025-07-15T10:56:16.588433Z","steps":["trace[947144259] 'agreement among raft nodes before linearized reading' (duration: 137.218461ms)"],"step_count":1} Jul 15 10:56:16.631290 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:16.630301Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"183.717073ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/ci-4230-2-100-6ef5831daf.novalocal\" ","response":"range_response_count:1 size:3817"} Jul 15 10:56:16.631290 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:16.630471Z","caller":"traceutil/trace.go:171","msg":"trace[121879747] range","detail":"{range_begin:/registry/minions/ci-4230-2-100-6ef5831daf.novalocal; range_end:; response_count:1; response_revision:250; }","duration":"183.899505ms","start":"2025-07-15T10:56:16.446545Z","end":"2025-07-15T10:56:16.630445Z","steps":["trace[121879747] 'agreement among raft nodes before linearized reading' (duration: 183.614613ms)"],"step_count":1} Jul 15 10:56:16.632989 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:16.631146Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"176.744504ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/csinodes/ci-4230-2-100-6ef5831daf.novalocal\" ","response":"range_response_count:1 size:772"} Jul 15 10:56:16.632989 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:16.631598Z","caller":"traceutil/trace.go:171","msg":"trace[1891303480] range","detail":"{range_begin:/registry/csinodes/ci-4230-2-100-6ef5831daf.novalocal; range_end:; response_count:1; response_revision:250; }","duration":"177.228729ms","start":"2025-07-15T10:56:16.454346Z","end":"2025-07-15T10:56:16.631575Z","steps":["trace[1891303480] 'agreement among raft nodes before linearized reading' (duration: 176.684432ms)"],"step_count":1} Jul 15 10:56:21.050817 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:21.045080Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"267.407011ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/kube-system/replicaset-controller\" ","response":"range_response_count:1 size:207"} Jul 15 10:56:21.050817 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:21.045506Z","caller":"traceutil/trace.go:171","msg":"trace[915954661] range","detail":"{range_begin:/registry/serviceaccounts/kube-system/replicaset-controller; range_end:; response_count:1; response_revision:342; }","duration":"267.974449ms","start":"2025-07-15T10:56:20.777507Z","end":"2025-07-15T10:56:21.045481Z","steps":["trace[915954661] 'range keys from in-memory index tree' (duration: 267.217707ms)"],"step_count":1} Jul 15 10:56:21.050817 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:21.045790Z","caller":"traceutil/trace.go:171","msg":"trace[1862759734] transaction","detail":"{read_only:false; response_revision:343; number_of_response:1; }","duration":"230.630395ms","start":"2025-07-15T10:56:20.815146Z","end":"2025-07-15T10:56:21.045776Z","steps":["trace[1862759734] 'process raft request' (duration: 196.571808ms)","trace[1862759734] 'compare' (duration: 33.087612ms)"],"step_count":2} Jul 15 10:56:21.050817 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:21.046011Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"182.123388ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/kube-system/kube-proxy\" ","response":"range_response_count:1 size:185"} Jul 15 10:56:21.050817 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:21.046047Z","caller":"traceutil/trace.go:171","msg":"trace[912168406] range","detail":"{range_begin:/registry/serviceaccounts/kube-system/kube-proxy; range_end:; response_count:1; response_revision:343; }","duration":"182.153244ms","start":"2025-07-15T10:56:20.863877Z","end":"2025-07-15T10:56:21.046030Z","steps":["trace[912168406] 'agreement among raft nodes before linearized reading' (duration: 182.100415ms)"],"step_count":1} Jul 15 10:56:21.055409 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:21.045247Z","caller":"traceutil/trace.go:171","msg":"trace[144426264] linearizableReadLoop","detail":"{readStateIndex:353; appliedIndex:352; }","duration":"181.248483ms","start":"2025-07-15T10:56:20.863884Z","end":"2025-07-15T10:56:21.045132Z","steps":["trace[144426264] 'read index received' (duration: 147.7659ms)","trace[144426264] 'applied index is now lower than readState.Index' (duration: 33.481061ms)"],"step_count":2} Jul 15 10:56:21.055409 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:21.046333Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"172.194115ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/installations.operator.tigera.io\" ","response":"range_response_count:0 size:5"} Jul 15 10:56:21.055409 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:21.046394Z","caller":"traceutil/trace.go:171","msg":"trace[1160567454] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/installations.operator.tigera.io; range_end:; response_count:0; response_revision:343; }","duration":"172.257404ms","start":"2025-07-15T10:56:20.874098Z","end":"2025-07-15T10:56:21.046356Z","steps":["trace[1160567454] 'agreement among raft nodes before linearized reading' (duration: 172.191329ms)"],"step_count":1} Jul 15 10:56:33.298315 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:33.293128Z","caller":"traceutil/trace.go:171","msg":"trace[1428257174] transaction","detail":"{read_only:false; response_revision:509; number_of_response:1; }","duration":"109.951116ms","start":"2025-07-15T10:56:33.183105Z","end":"2025-07-15T10:56:33.293056Z","steps":["trace[1428257174] 'process raft request' (duration: 109.785672ms)"],"step_count":1} Jul 15 10:56:33.298315 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:33.293603Z","caller":"traceutil/trace.go:171","msg":"trace[462083095] linearizableReadLoop","detail":"{readStateIndex:525; appliedIndex:525; }","duration":"105.65386ms","start":"2025-07-15T10:56:33.187867Z","end":"2025-07-15T10:56:33.293521Z","steps":["trace[462083095] 'read index received' (duration: 105.642928ms)","trace[462083095] 'applied index is now lower than readState.Index' (duration: 9.258µs)"],"step_count":2} Jul 15 10:56:33.298315 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:33.294109Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"105.998835ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/hostendpoints.crd.projectcalico.org\" ","response":"range_response_count:1 size:5520"} Jul 15 10:56:33.298315 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:33.294242Z","caller":"traceutil/trace.go:171","msg":"trace[1745148005] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/hostendpoints.crd.projectcalico.org; range_end:; response_count:1; response_revision:509; }","duration":"106.368077ms","start":"2025-07-15T10:56:33.187838Z","end":"2025-07-15T10:56:33.294206Z","steps":["trace[1745148005] 'agreement among raft nodes before linearized reading' (duration: 105.89618ms)"],"step_count":1} Jul 15 10:56:53.705294 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:53.700955Z","caller":"traceutil/trace.go:171","msg":"trace[1142423597] transaction","detail":"{read_only:false; response_revision:804; number_of_response:1; }","duration":"126.057515ms","start":"2025-07-15T10:56:53.574807Z","end":"2025-07-15T10:56:53.700865Z","steps":["trace[1142423597] 'process raft request' (duration: 110.033137ms)","trace[1142423597] 'compare' (duration: 15.484811ms)"],"step_count":2} Jul 15 10:56:55.689975 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:55.688850Z","caller":"traceutil/trace.go:171","msg":"trace[1911825906] transaction","detail":"{read_only:false; response_revision:812; number_of_response:1; }","duration":"243.020418ms","start":"2025-07-15T10:56:55.445781Z","end":"2025-07-15T10:56:55.688801Z","steps":["trace[1911825906] 'process raft request' (duration: 232.716696ms)","trace[1911825906] 'compare' (duration: 10.117473ms)"],"step_count":2} Jul 15 10:56:55.703655 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:55.698596Z","caller":"traceutil/trace.go:171","msg":"trace[1998208209] transaction","detail":"{read_only:false; response_revision:813; number_of_response:1; }","duration":"251.491866ms","start":"2025-07-15T10:56:55.447082Z","end":"2025-07-15T10:56:55.698574Z","steps":["trace[1998208209] 'process raft request' (duration: 251.217068ms)"],"step_count":1} Jul 15 10:56:55.703655 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:55.699366Z","caller":"traceutil/trace.go:171","msg":"trace[1816306227] transaction","detail":"{read_only:false; response_revision:815; number_of_response:1; }","duration":"251.500212ms","start":"2025-07-15T10:56:55.447828Z","end":"2025-07-15T10:56:55.699328Z","steps":["trace[1816306227] 'process raft request' (duration: 251.127456ms)"],"step_count":1} Jul 15 10:56:55.703655 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:55.699599Z","caller":"traceutil/trace.go:171","msg":"trace[385372818] transaction","detail":"{read_only:false; response_revision:814; number_of_response:1; }","duration":"252.029058ms","start":"2025-07-15T10:56:55.447552Z","end":"2025-07-15T10:56:55.699581Z","steps":["trace[385372818] 'process raft request' (duration: 250.944603ms)"],"step_count":1} Jul 15 10:56:55.703655 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:55.699733Z","caller":"traceutil/trace.go:171","msg":"trace[383270730] transaction","detail":"{read_only:false; response_revision:816; number_of_response:1; }","duration":"251.37984ms","start":"2025-07-15T10:56:55.448319Z","end":"2025-07-15T10:56:55.699699Z","steps":["trace[383270730] 'process raft request' (duration: 250.882053ms)"],"step_count":1} Jul 15 10:56:55.703655 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:55.700240Z","caller":"traceutil/trace.go:171","msg":"trace[2047733111] transaction","detail":"{read_only:false; response_revision:817; number_of_response:1; }","duration":"250.981764ms","start":"2025-07-15T10:56:55.449240Z","end":"2025-07-15T10:56:55.700221Z","steps":["trace[2047733111] 'process raft request' (duration: 250.310735ms)"],"step_count":1} Jul 15 10:56:55.704440 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:55.700759Z","caller":"traceutil/trace.go:171","msg":"trace[1924687777] transaction","detail":"{read_only:false; response_revision:818; number_of_response:1; }","duration":"251.381353ms","start":"2025-07-15T10:56:55.449349Z","end":"2025-07-15T10:56:55.700731Z","steps":["trace[1924687777] 'process raft request' (duration: 250.768996ms)"],"step_count":1} Jul 15 10:56:55.704440 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:55.701460Z","caller":"traceutil/trace.go:171","msg":"trace[2126513615] transaction","detail":"{read_only:false; response_revision:819; number_of_response:1; }","duration":"233.194043ms","start":"2025-07-15T10:56:55.468242Z","end":"2025-07-15T10:56:55.701436Z","steps":["trace[2126513615] 'process raft request' (duration: 232.122723ms)"],"step_count":1} Jul 15 10:56:55.704440 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:55.702315Z","caller":"traceutil/trace.go:171","msg":"trace[627853820] transaction","detail":"{read_only:false; response_revision:820; number_of_response:1; }","duration":"213.015937ms","start":"2025-07-15T10:56:55.489275Z","end":"2025-07-15T10:56:55.702291Z","steps":["trace[627853820] 'process raft request' (duration: 212.022206ms)"],"step_count":1} Jul 15 10:56:56.221329 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:56.220499Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"160.146ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure:<>>","response":"size:16"} Jul 15 10:56:56.222518 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:56.220772Z","caller":"traceutil/trace.go:171","msg":"trace[2045463930] transaction","detail":"{read_only:false; response_revision:825; number_of_response:1; }","duration":"335.73637ms","start":"2025-07-15T10:56:55.885008Z","end":"2025-07-15T10:56:56.220744Z","steps":["trace[2045463930] 'process raft request' (duration: 172.809291ms)","trace[2045463930] 'compare' (duration: 159.887713ms)"],"step_count":2} Jul 15 10:56:56.222518 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:56.221357Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:56:55.884997Z","time spent":"336.302209ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":757,"response count":0,"response size":40,"request content":"compare: success:> failure:<>"} Jul 15 10:56:56.225585 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:56.223884Z","caller":"traceutil/trace.go:171","msg":"trace[429859397] linearizableReadLoop","detail":"{readStateIndex:847; appliedIndex:845; }","duration":"287.160774ms","start":"2025-07-15T10:56:55.936669Z","end":"2025-07-15T10:56:56.223830Z","steps":["trace[429859397] 'read index received' (duration: 121.15821ms)","trace[429859397] 'applied index is now lower than readState.Index' (duration: 166.001593ms)"],"step_count":2} Jul 15 10:56:56.225585 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:56.224558Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"287.792529ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/calico-system/calico-kube-controllers\" ","response":"range_response_count:1 size:463"} Jul 15 10:56:56.225585 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:56.224652Z","caller":"traceutil/trace.go:171","msg":"trace[463418892] range","detail":"{range_begin:/registry/serviceaccounts/calico-system/calico-kube-controllers; range_end:; response_count:1; response_revision:826; }","duration":"287.991823ms","start":"2025-07-15T10:56:55.936628Z","end":"2025-07-15T10:56:56.224619Z","steps":["trace[463418892] 'agreement among raft nodes before linearized reading' (duration: 287.482183ms)"],"step_count":1} Jul 15 10:56:56.225585 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:56.224813Z","caller":"traceutil/trace.go:171","msg":"trace[896400921] transaction","detail":"{read_only:false; response_revision:826; number_of_response:1; }","duration":"326.457913ms","start":"2025-07-15T10:56:55.898341Z","end":"2025-07-15T10:56:56.224798Z","steps":["trace[896400921] 'process raft request' (duration: 325.319203ms)"],"step_count":1} Jul 15 10:56:56.226945 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:56.226600Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"285.071138ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/calico-system/whisker\" ","response":"range_response_count:1 size:426"} Jul 15 10:56:56.227309 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:56.226679Z","caller":"traceutil/trace.go:171","msg":"trace[934656867] range","detail":"{range_begin:/registry/serviceaccounts/calico-system/whisker; range_end:; response_count:1; response_revision:826; }","duration":"285.167543ms","start":"2025-07-15T10:56:55.941493Z","end":"2025-07-15T10:56:56.226661Z","steps":["trace[934656867] 'agreement among raft nodes before linearized reading' (duration: 284.977016ms)"],"step_count":1} Jul 15 10:56:56.227964 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:56.227563Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:56:55.898316Z","time spent":"326.559639ms","remote":"172.24.4.149:33206","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":5045,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:56:56.228847 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:56.228343Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"273.001503ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:56:56.230038 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:56.228667Z","caller":"traceutil/trace.go:171","msg":"trace[1629217245] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:826; }","duration":"273.330305ms","start":"2025-07-15T10:56:55.955310Z","end":"2025-07-15T10:56:56.228640Z","steps":["trace[1629217245] 'agreement among raft nodes before linearized reading' (duration: 272.964361ms)"],"step_count":1} Jul 15 10:56:56.230038 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:56.229712Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"285.627519ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/kube-system/coredns\" ","response":"range_response_count:1 size:179"} Jul 15 10:56:56.230731 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:56.229777Z","caller":"traceutil/trace.go:171","msg":"trace[680731263] range","detail":"{range_begin:/registry/serviceaccounts/kube-system/coredns; range_end:; response_count:1; response_revision:826; }","duration":"285.702574ms","start":"2025-07-15T10:56:55.944047Z","end":"2025-07-15T10:56:56.229749Z","steps":["trace[680731263] 'agreement among raft nodes before linearized reading' (duration: 285.554489ms)"],"step_count":1} Jul 15 10:56:56.230731 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:56.230208Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"287.242326ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/calico-system/goldmane\" ","response":"range_response_count:1 size:429"} Jul 15 10:56:56.230731 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:56.230263Z","caller":"traceutil/trace.go:171","msg":"trace[753107529] range","detail":"{range_begin:/registry/serviceaccounts/calico-system/goldmane; range_end:; response_count:1; response_revision:826; }","duration":"287.317199ms","start":"2025-07-15T10:56:55.942930Z","end":"2025-07-15T10:56:56.230247Z","steps":["trace[753107529] 'agreement among raft nodes before linearized reading' (duration: 287.007334ms)"],"step_count":1} Jul 15 10:56:56.230731 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:56.230439Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"287.523777ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/calico-apiserver/calico-apiserver\" ","response":"range_response_count:1 size:452"} Jul 15 10:56:56.231751 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:56.230758Z","caller":"traceutil/trace.go:171","msg":"trace[1895779216] range","detail":"{range_begin:/registry/serviceaccounts/calico-apiserver/calico-apiserver; range_end:; response_count:1; response_revision:826; }","duration":"287.859683ms","start":"2025-07-15T10:56:55.942880Z","end":"2025-07-15T10:56:56.230739Z","steps":["trace[1895779216] 'agreement among raft nodes before linearized reading' (duration: 287.439856ms)"],"step_count":1} Jul 15 10:56:56.231751 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:56:56.230936Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"289.369176ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/kube-system/coredns\" ","response":"range_response_count:1 size:179"} Jul 15 10:56:56.231751 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:56.230979Z","caller":"traceutil/trace.go:171","msg":"trace[1455458649] range","detail":"{range_begin:/registry/serviceaccounts/kube-system/coredns; range_end:; response_count:1; response_revision:826; }","duration":"289.435594ms","start":"2025-07-15T10:56:55.941530Z","end":"2025-07-15T10:56:56.230966Z","steps":["trace[1455458649] 'agreement among raft nodes before linearized reading' (duration: 289.365018ms)"],"step_count":1} Jul 15 10:56:56.449362 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:56.448609Z","caller":"traceutil/trace.go:171","msg":"trace[989190577] transaction","detail":"{read_only:false; response_revision:829; number_of_response:1; }","duration":"114.506816ms","start":"2025-07-15T10:56:56.334072Z","end":"2025-07-15T10:56:56.448579Z","steps":["trace[989190577] 'process raft request' (duration: 113.7862ms)"],"step_count":1} Jul 15 10:56:56.620165 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:56.617750Z","caller":"traceutil/trace.go:171","msg":"trace[1896252728] transaction","detail":"{read_only:false; response_revision:830; number_of_response:1; }","duration":"152.353953ms","start":"2025-07-15T10:56:56.465357Z","end":"2025-07-15T10:56:56.617711Z","steps":["trace[1896252728] 'process raft request' (duration: 103.497401ms)","trace[1896252728] 'compare' (duration: 48.680224ms)"],"step_count":2} Jul 15 10:56:56.620165 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:56:56.618868Z","caller":"traceutil/trace.go:171","msg":"trace[406419311] transaction","detail":"{read_only:false; response_revision:831; number_of_response:1; }","duration":"135.341061ms","start":"2025-07-15T10:56:56.483506Z","end":"2025-07-15T10:56:56.618848Z","steps":["trace[406419311] 'process raft request' (duration: 135.193507ms)"],"step_count":1} Jul 15 10:57:00.222711 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:00.219976Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"125.176348ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 10:57:00.230992 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:00.220473Z","caller":"traceutil/trace.go:171","msg":"trace[110729030] transaction","detail":"{read_only:false; response_revision:887; number_of_response:1; }","duration":"140.217608ms","start":"2025-07-15T10:57:00.080195Z","end":"2025-07-15T10:57:00.220413Z","steps":["trace[110729030] 'process raft request' (duration: 14.219306ms)","trace[110729030] 'compare' (duration: 124.94324ms)"],"step_count":2} Jul 15 10:57:07.320084 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:07.317949Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072150,"retry-timeout":"500ms"} Jul 15 10:57:07.420447 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:07.419182Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"100.818021ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 10:57:07.420447 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:07.419491Z","caller":"traceutil/trace.go:171","msg":"trace[158360106] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"101.270414ms","start":"2025-07-15T10:57:07.318090Z","end":"2025-07-15T10:57:07.419361Z","steps":["trace[158360106] 'agreement among raft nodes before linearized reading' (duration: 100.857588ms)"],"step_count":1} Jul 15 10:57:07.449076 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:07.446207Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"2.746639037s","expected-duration":"1s"} Jul 15 10:57:07.833353 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:07.828708Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"380.712968ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 10:57:07.833353 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:07.829086Z","caller":"traceutil/trace.go:171","msg":"trace[1860634169] transaction","detail":"{read_only:false; response_revision:941; number_of_response:1; }","duration":"3.12968154s","start":"2025-07-15T10:57:04.699332Z","end":"2025-07-15T10:57:07.829014Z","steps":["trace[1860634169] 'process raft request' (duration: 2.748180512s)","trace[1860634169] 'compare' (duration: 380.423489ms)"],"step_count":2} Jul 15 10:57:07.833353 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:07.829333Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:04.699312Z","time spent":"3.129872568s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":840,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:57:07.834970 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:07.829574Z","caller":"traceutil/trace.go:171","msg":"trace[1706093042] linearizableReadLoop","detail":"{readStateIndex:965; appliedIndex:963; }","duration":"3.099644362s","start":"2025-07-15T10:57:04.729890Z","end":"2025-07-15T10:57:07.829534Z","steps":["trace[1706093042] 'read index received' (duration: 2.717753522s)","trace[1706093042] 'applied index is now lower than readState.Index' (duration: 381.889808ms)"],"step_count":2} Jul 15 10:57:07.834970 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:07.829688Z","caller":"traceutil/trace.go:171","msg":"trace[28403581] transaction","detail":"{read_only:false; response_revision:943; number_of_response:1; }","duration":"409.380961ms","start":"2025-07-15T10:57:07.420294Z","end":"2025-07-15T10:57:07.829674Z","steps":["trace[28403581] 'process raft request' (duration: 408.963304ms)"],"step_count":1} Jul 15 10:57:07.834970 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:07.829677Z","caller":"traceutil/trace.go:171","msg":"trace[1997925245] transaction","detail":"{read_only:false; response_revision:942; number_of_response:1; }","duration":"511.800742ms","start":"2025-07-15T10:57:07.317858Z","end":"2025-07-15T10:57:07.829659Z","steps":["trace[1997925245] 'process raft request' (duration: 510.998644ms)"],"step_count":1} Jul 15 10:57:07.834970 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:07.829856Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:07.317828Z","time spent":"511.941474ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":490,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:57:07.835520 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:07.829904Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:07.420077Z","time spent":"409.670279ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":625,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:57:07.835520 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:07.830000Z","caller":"traceutil/trace.go:171","msg":"trace[2057598639] transaction","detail":"{read_only:false; response_revision:944; number_of_response:1; }","duration":"409.368406ms","start":"2025-07-15T10:57:07.420617Z","end":"2025-07-15T10:57:07.829985Z","steps":["trace[2057598639] 'process raft request' (duration: 408.810099ms)"],"step_count":1} Jul 15 10:57:07.835520 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:07.830094Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:07.420565Z","time spent":"409.485011ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":490,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:57:07.835520 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:07.831364Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.101395481s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3643"} Jul 15 10:57:07.838359 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:07.831754Z","caller":"traceutil/trace.go:171","msg":"trace[1071687064] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:944; }","duration":"3.101711501s","start":"2025-07-15T10:57:04.729882Z","end":"2025-07-15T10:57:07.831593Z","steps":["trace[1071687064] 'agreement among raft nodes before linearized reading' (duration: 3.100258206s)"],"step_count":1} Jul 15 10:57:07.838359 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:07.832077Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:04.729858Z","time spent":"3.102123064s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":3667,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:57:07.838359 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:07.835827Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"517.57743ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:57:07.838359 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:07.835915Z","caller":"traceutil/trace.go:171","msg":"trace[154441990] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:945; }","duration":"517.690878ms","start":"2025-07-15T10:57:07.318205Z","end":"2025-07-15T10:57:07.835896Z","steps":["trace[154441990] 'agreement among raft nodes before linearized reading' (duration: 517.420095ms)"],"step_count":1} Jul 15 10:57:07.838359 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:07.836019Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:07.318196Z","time spent":"517.781535ms","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:57:07.839092 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:07.836122Z","caller":"traceutil/trace.go:171","msg":"trace[1294708308] transaction","detail":"{read_only:false; response_revision:945; number_of_response:1; }","duration":"369.280756ms","start":"2025-07-15T10:57:07.466712Z","end":"2025-07-15T10:57:07.835992Z","steps":["trace[1294708308] 'process raft request' (duration: 367.863852ms)"],"step_count":1} Jul 15 10:57:07.839092 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:07.836638Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:07.466674Z","time spent":"369.543935ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":516,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:57:08.869963 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:08.774894Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072185,"retry-timeout":"500ms"} Jul 15 10:57:08.968070 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:08.966371Z","caller":"traceutil/trace.go:171","msg":"trace[1799505466] linearizableReadLoop","detail":"{readStateIndex:975; appliedIndex:974; }","duration":"692.443159ms","start":"2025-07-15T10:57:08.273894Z","end":"2025-07-15T10:57:08.966337Z","steps":["trace[1799505466] 'read index received' (duration: 604.819957ms)","trace[1799505466] 'applied index is now lower than readState.Index' (duration: 87.621519ms)"],"step_count":2} Jul 15 10:57:08.968070 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:08.966599Z","caller":"traceutil/trace.go:171","msg":"trace[384272876] transaction","detail":"{read_only:false; response_revision:952; number_of_response:1; }","duration":"703.871939ms","start":"2025-07-15T10:57:08.262708Z","end":"2025-07-15T10:57:08.966580Z","steps":["trace[384272876] 'process raft request' (duration: 615.959217ms)","trace[384272876] 'compare' (duration: 87.527558ms)"],"step_count":2} Jul 15 10:57:08.968623 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:08.967555Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"693.655289ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/deployments/calico-apiserver/calico-apiserver\" ","response":"range_response_count:1 size:4810"} Jul 15 10:57:08.968623 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:08.967640Z","caller":"traceutil/trace.go:171","msg":"trace[701171022] range","detail":"{range_begin:/registry/deployments/calico-apiserver/calico-apiserver; range_end:; response_count:1; response_revision:952; }","duration":"693.763758ms","start":"2025-07-15T10:57:08.273855Z","end":"2025-07-15T10:57:08.967619Z","steps":["trace[701171022] 'agreement among raft nodes before linearized reading' (duration: 693.514857ms)"],"step_count":1} Jul 15 10:57:08.969573 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:08.967728Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:08.273768Z","time spent":"693.915321ms","remote":"172.24.4.149:33462","response type":"/etcdserverpb.KV/Range","request count":0,"request size":57,"response count":1,"response size":4834,"request content":"key:\"/registry/deployments/calico-apiserver/calico-apiserver\" "} Jul 15 10:57:08.970449 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:08.969262Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:08.262670Z","time spent":"704.018823ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1082,"response count":0,"response size":40,"request content":"compare: success:> failure:<>"} Jul 15 10:57:09.542464 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:09.540543Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"145.357777ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 10:57:09.542464 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:09.540710Z","caller":"traceutil/trace.go:171","msg":"trace[868601251] linearizableReadLoop","detail":"{readStateIndex:976; appliedIndex:975; }","duration":"257.957496ms","start":"2025-07-15T10:57:09.282724Z","end":"2025-07-15T10:57:09.540682Z","steps":["trace[868601251] 'read index received' (duration: 112.216668ms)","trace[868601251] 'applied index is now lower than readState.Index' (duration: 145.739153ms)"],"step_count":2} Jul 15 10:57:09.542464 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:09.540779Z","caller":"traceutil/trace.go:171","msg":"trace[531585985] transaction","detail":"{read_only:false; response_revision:953; number_of_response:1; }","duration":"560.116854ms","start":"2025-07-15T10:57:08.980625Z","end":"2025-07-15T10:57:09.540742Z","steps":["trace[531585985] 'process raft request' (duration: 414.385395ms)","trace[531585985] 'compare' (duration: 145.011479ms)"],"step_count":2} Jul 15 10:57:09.543188 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:09.540928Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:08.980600Z","time spent":"560.255061ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":786,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:57:09.543188 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:09.541249Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"258.403786ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/specs/calico-apiserver/calico-api\" ","response":"range_response_count:1 size:856"} Jul 15 10:57:09.543188 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:09.541304Z","caller":"traceutil/trace.go:171","msg":"trace[546493786] range","detail":"{range_begin:/registry/services/specs/calico-apiserver/calico-api; range_end:; response_count:1; response_revision:953; }","duration":"258.612419ms","start":"2025-07-15T10:57:09.282675Z","end":"2025-07-15T10:57:09.541287Z","steps":["trace[546493786] 'agreement among raft nodes before linearized reading' (duration: 258.132974ms)"],"step_count":1} Jul 15 10:57:10.004818 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:10.003850Z","caller":"traceutil/trace.go:171","msg":"trace[1527210825] transaction","detail":"{read_only:false; response_revision:955; number_of_response:1; }","duration":"155.592929ms","start":"2025-07-15T10:57:09.848217Z","end":"2025-07-15T10:57:10.003810Z","steps":["trace[1527210825] 'process raft request' (duration: 126.251059ms)","trace[1527210825] 'compare' (duration: 28.77777ms)"],"step_count":2} Jul 15 10:57:10.006965 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:10.004280Z","caller":"traceutil/trace.go:171","msg":"trace[1725041310] transaction","detail":"{read_only:false; response_revision:956; number_of_response:1; }","duration":"148.191119ms","start":"2025-07-15T10:57:09.856070Z","end":"2025-07-15T10:57:10.004261Z","steps":["trace[1725041310] 'process raft request' (duration: 147.372267ms)"],"step_count":1} Jul 15 10:57:10.019038 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:10.017755Z","caller":"traceutil/trace.go:171","msg":"trace[1795790699] linearizableReadLoop","detail":"{readStateIndex:978; appliedIndex:977; }","duration":"165.848856ms","start":"2025-07-15T10:57:09.851885Z","end":"2025-07-15T10:57:10.017733Z","steps":["trace[1795790699] 'read index received' (duration: 75.822398ms)","trace[1795790699] 'applied index is now lower than readState.Index' (duration: 90.024854ms)"],"step_count":2} Jul 15 10:57:10.019038 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:10.018317Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"166.357558ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rolebindings/calico-apiserver/tigera-extension-apiserver-secrets-access\" ","response":"range_response_count:0 size:5"} Jul 15 10:57:10.019038 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:10.018432Z","caller":"traceutil/trace.go:171","msg":"trace[1076587336] range","detail":"{range_begin:/registry/rolebindings/calico-apiserver/tigera-extension-apiserver-secrets-access; range_end:; response_count:0; response_revision:956; }","duration":"166.619183ms","start":"2025-07-15T10:57:09.851794Z","end":"2025-07-15T10:57:10.018413Z","steps":["trace[1076587336] 'agreement among raft nodes before linearized reading' (duration: 166.401603ms)"],"step_count":1} Jul 15 10:57:11.289442 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:11.288241Z","caller":"traceutil/trace.go:171","msg":"trace[732382250] transaction","detail":"{read_only:false; response_revision:962; number_of_response:1; }","duration":"104.493462ms","start":"2025-07-15T10:57:11.183709Z","end":"2025-07-15T10:57:11.288203Z","steps":["trace[732382250] 'process raft request' (duration: 47.046541ms)","trace[732382250] 'compare' (duration: 56.954761ms)"],"step_count":2} Jul 15 10:57:11.505884 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:11.504497Z","caller":"traceutil/trace.go:171","msg":"trace[493213974] transaction","detail":"{read_only:false; response_revision:965; number_of_response:1; }","duration":"163.342311ms","start":"2025-07-15T10:57:11.341106Z","end":"2025-07-15T10:57:11.504448Z","steps":["trace[493213974] 'process raft request' (duration: 159.538554ms)"],"step_count":1} Jul 15 10:57:11.505884 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:11.504496Z","caller":"traceutil/trace.go:171","msg":"trace[76760516] linearizableReadLoop","detail":"{readStateIndex:989; appliedIndex:988; }","duration":"161.014374ms","start":"2025-07-15T10:57:11.343426Z","end":"2025-07-15T10:57:11.504440Z","steps":["trace[76760516] 'read index received' (duration: 157.161292ms)","trace[76760516] 'applied index is now lower than readState.Index' (duration: 3.85164ms)"],"step_count":2} Jul 15 10:57:11.505884 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:11.504801Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"161.382797ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/calico-tier-getter\" ","response":"range_response_count:1 size:515"} Jul 15 10:57:11.505884 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:11.504923Z","caller":"traceutil/trace.go:171","msg":"trace[322002153] range","detail":"{range_begin:/registry/clusterroles/calico-tier-getter; range_end:; response_count:1; response_revision:965; }","duration":"161.555239ms","start":"2025-07-15T10:57:11.343347Z","end":"2025-07-15T10:57:11.504903Z","steps":["trace[322002153] 'agreement among raft nodes before linearized reading' (duration: 161.217697ms)"],"step_count":1} Jul 15 10:57:14.242975 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:14.241592Z","caller":"traceutil/trace.go:171","msg":"trace[1240258562] transaction","detail":"{read_only:false; response_revision:1004; number_of_response:1; }","duration":"165.783759ms","start":"2025-07-15T10:57:14.075720Z","end":"2025-07-15T10:57:14.241503Z","steps":["trace[1240258562] 'process raft request' (duration: 78.002599ms)","trace[1240258562] 'compare' (duration: 87.435051ms)"],"step_count":2} Jul 15 10:57:14.242975 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:14.241666Z","caller":"traceutil/trace.go:171","msg":"trace[1166835538] transaction","detail":"{read_only:false; response_revision:1005; number_of_response:1; }","duration":"109.963463ms","start":"2025-07-15T10:57:14.131674Z","end":"2025-07-15T10:57:14.241638Z","steps":["trace[1166835538] 'process raft request' (duration: 109.846837ms)"],"step_count":1} Jul 15 10:57:16.537940 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:16.537087Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"180.696313ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 10:57:16.539590 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:16.537377Z","caller":"traceutil/trace.go:171","msg":"trace[646837179] transaction","detail":"{read_only:false; response_revision:1027; number_of_response:1; }","duration":"210.326181ms","start":"2025-07-15T10:57:16.327005Z","end":"2025-07-15T10:57:16.537332Z","steps":["trace[646837179] 'process raft request' (duration: 29.241857ms)","trace[646837179] 'compare' (duration: 180.460057ms)"],"step_count":2} Jul 15 10:57:24.691622 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:19.963811Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072652,"retry-timeout":"500ms"} Jul 15 10:57:24.691622 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:20.464686Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072652,"retry-timeout":"500ms"} Jul 15 10:57:24.691622 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:20.965553Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072652,"retry-timeout":"500ms"} Jul 15 10:57:24.691622 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:21.466322Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072652,"retry-timeout":"500ms"} Jul 15 10:57:24.691622 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:21.959738Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.00106749s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 10:57:24.691622 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:21.960050Z","caller":"traceutil/trace.go:171","msg":"trace[974115931] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.001592807s","start":"2025-07-15T10:57:19.958350Z","end":"2025-07-15T10:57:21.959943Z","steps":["trace[974115931] 'agreement among raft nodes before linearized reading' (duration: 2.001113709s)"],"step_count":1} Jul 15 10:57:24.691622 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:21.960684Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:19.958292Z","time spent":"2.002025595s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:57:24.712011 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:21.967305Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072652,"retry-timeout":"500ms"} Jul 15 10:57:24.712011 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:22.467870Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072652,"retry-timeout":"500ms"} Jul 15 10:57:24.712011 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:22.968748Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072652,"retry-timeout":"500ms"} Jul 15 10:57:24.712011 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:23.469985Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072652,"retry-timeout":"500ms"} Jul 15 10:57:24.712011 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:23.959281Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.00029382s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 10:57:24.712011 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:23.959435Z","caller":"traceutil/trace.go:171","msg":"trace[61209685] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.00072858s","start":"2025-07-15T10:57:21.958642Z","end":"2025-07-15T10:57:23.959371Z","steps":["trace[61209685] 'agreement among raft nodes before linearized reading' (duration: 2.000575865s)"],"step_count":1} Jul 15 10:57:24.712011 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:23.959549Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:21.958601Z","time spent":"2.000892969s","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:57:24.712855 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:23.970962Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072652,"retry-timeout":"500ms"} Jul 15 10:57:24.712855 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.471710Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072652,"retry-timeout":"500ms"} Jul 15 10:57:24.712855 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.703053Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"6.165866439s","expected-duration":"1s"} Jul 15 10:57:24.712855 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.705859Z","caller":"traceutil/trace.go:171","msg":"trace[858908017] transaction","detail":"{read_only:false; response_revision:1053; number_of_response:1; }","duration":"6.168596234s","start":"2025-07-15T10:57:18.537042Z","end":"2025-07-15T10:57:24.705638Z","steps":["trace[858908017] 'process raft request' (duration: 6.166836286s)"],"step_count":1} Jul 15 10:57:24.712855 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.707687Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:18.537016Z","time spent":"6.169154665s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":828,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:57:24.935201 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.932720Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"115.947707ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 10:57:24.935201 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.932982Z","caller":"traceutil/trace.go:171","msg":"trace[1636675968] linearizableReadLoop","detail":"{readStateIndex:1081; appliedIndex:1079; }","duration":"5.469477418s","start":"2025-07-15T10:57:19.463467Z","end":"2025-07-15T10:57:24.932944Z","steps":["trace[1636675968] 'read index received' (duration: 5.24213085s)","trace[1636675968] 'applied index is now lower than readState.Index' (duration: 227.343442ms)"],"step_count":2} Jul 15 10:57:24.935201 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.933107Z","caller":"traceutil/trace.go:171","msg":"trace[958848012] transaction","detail":"{read_only:false; response_revision:1054; number_of_response:1; }","duration":"4.939320262s","start":"2025-07-15T10:57:19.993770Z","end":"2025-07-15T10:57:24.933090Z","steps":["trace[958848012] 'process raft request' (duration: 4.822876284s)","trace[958848012] 'compare' (duration: 115.677323ms)"],"step_count":2} Jul 15 10:57:24.935201 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.933220Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:19.993743Z","time spent":"4.939424073s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":490,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:57:24.935908 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.933305Z","caller":"traceutil/trace.go:171","msg":"trace[744346571] transaction","detail":"{read_only:false; response_revision:1055; number_of_response:1; }","duration":"4.829459556s","start":"2025-07-15T10:57:20.103833Z","end":"2025-07-15T10:57:24.933293Z","steps":["trace[744346571] 'process raft request' (duration: 4.829013933s)"],"step_count":1} Jul 15 10:57:24.935908 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.933809Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.470328074s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/daemonsets/calico-system/calico-node\" ","response":"range_response_count:1 size:11398"} Jul 15 10:57:24.935908 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.933907Z","caller":"traceutil/trace.go:171","msg":"trace[1934399867] range","detail":"{range_begin:/registry/daemonsets/calico-system/calico-node; range_end:; response_count:1; response_revision:1055; }","duration":"5.470440933s","start":"2025-07-15T10:57:19.463423Z","end":"2025-07-15T10:57:24.933864Z","steps":["trace[1934399867] 'agreement among raft nodes before linearized reading' (duration: 5.469891851s)"],"step_count":1} Jul 15 10:57:24.935908 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.933993Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:19.463366Z","time spent":"5.470607376s","remote":"172.24.4.149:33488","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":1,"response size":11422,"request content":"key:\"/registry/daemonsets/calico-system/calico-node\" "} Jul 15 10:57:24.936264 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.934104Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:20.103792Z","time spent":"4.829537699s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:57:25.000670 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.998653Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"289.781735ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/daemonsets/\" range_end:\"/registry/daemonsets0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:57:25.000670 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.998772Z","caller":"traceutil/trace.go:171","msg":"trace[255340746] range","detail":"{range_begin:/registry/daemonsets/; range_end:/registry/daemonsets0; response_count:0; response_revision:1057; }","duration":"289.933869ms","start":"2025-07-15T10:57:24.708815Z","end":"2025-07-15T10:57:24.998749Z","steps":["trace[255340746] 'agreement among raft nodes before linearized reading' (duration: 289.752668ms)"],"step_count":1} Jul 15 10:57:25.000670 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.998913Z","caller":"traceutil/trace.go:171","msg":"trace[1014853760] transaction","detail":"{read_only:false; response_revision:1056; number_of_response:1; }","duration":"291.144222ms","start":"2025-07-15T10:57:24.707754Z","end":"2025-07-15T10:57:24.998898Z","steps":["trace[1014853760] 'process raft request' (duration: 290.081655ms)"],"step_count":1} Jul 15 10:57:25.000670 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.999024Z","caller":"traceutil/trace.go:171","msg":"trace[182616056] transaction","detail":"{read_only:false; response_revision:1057; number_of_response:1; }","duration":"268.169743ms","start":"2025-07-15T10:57:24.730841Z","end":"2025-07-15T10:57:24.999011Z","steps":["trace[182616056] 'process raft request' (duration: 267.208142ms)"],"step_count":1} Jul 15 10:57:25.001285 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.999184Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"217.71937ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/calico-system/whisker-57c76b9cc6-tb7kd.185267835b112f38\" ","response":"range_response_count:1 size:807"} Jul 15 10:57:25.001285 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.999252Z","caller":"traceutil/trace.go:171","msg":"trace[128010565] range","detail":"{range_begin:/registry/events/calico-system/whisker-57c76b9cc6-tb7kd.185267835b112f38; range_end:; response_count:1; response_revision:1057; }","duration":"217.946109ms","start":"2025-07-15T10:57:24.781273Z","end":"2025-07-15T10:57:24.999219Z","steps":["trace[128010565] 'agreement among raft nodes before linearized reading' (duration: 217.806599ms)"],"step_count":1} Jul 15 10:57:25.001285 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.998717Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"289.903801ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:57:25.001285 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.999460Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"223.973669ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/endpoints/calico-system/calico-typha\" ","response":"range_response_count:1 size:688"} Jul 15 10:57:25.001285 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.999503Z","caller":"traceutil/trace.go:171","msg":"trace[830732612] range","detail":"{range_begin:/registry/services/endpoints/calico-system/calico-typha; range_end:; response_count:1; response_revision:1057; }","duration":"224.04713ms","start":"2025-07-15T10:57:24.775444Z","end":"2025-07-15T10:57:24.999491Z","steps":["trace[830732612] 'agreement among raft nodes before linearized reading' (duration: 223.872733ms)"],"step_count":1} Jul 15 10:57:25.001858 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.999495Z","caller":"traceutil/trace.go:171","msg":"trace[379322635] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1057; }","duration":"290.818181ms","start":"2025-07-15T10:57:24.708656Z","end":"2025-07-15T10:57:24.999474Z","steps":["trace[379322635] 'agreement among raft nodes before linearized reading' (duration: 290.019244ms)"],"step_count":1} Jul 15 10:57:25.001858 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.999628Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"238.474132ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowschemas/\" range_end:\"/registry/flowschemas0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:57:25.001858 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.999689Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"444.825834ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 10:57:25.001858 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.999710Z","caller":"traceutil/trace.go:171","msg":"trace[1042119887] range","detail":"{range_begin:/registry/flowschemas/; range_end:/registry/flowschemas0; response_count:0; response_revision:1057; }","duration":"238.528787ms","start":"2025-07-15T10:57:24.761126Z","end":"2025-07-15T10:57:24.999655Z","steps":["trace[1042119887] 'agreement among raft nodes before linearized reading' (duration: 238.46806ms)"],"step_count":1} Jul 15 10:57:25.001858 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.999730Z","caller":"traceutil/trace.go:171","msg":"trace[1817808918] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1057; }","duration":"444.91667ms","start":"2025-07-15T10:57:24.554800Z","end":"2025-07-15T10:57:24.999717Z","steps":["trace[1817808918] 'agreement among raft nodes before linearized reading' (duration: 444.799282ms)"],"step_count":1} Jul 15 10:57:25.003621 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.999769Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:24.554771Z","time spent":"444.984019ms","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 10:57:25.003621 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.999844Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"239.844765ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/endpointslices/\" range_end:\"/registry/endpointslices0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:57:25.003621 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.999856Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.294797316s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/cronjobs/\" range_end:\"/registry/cronjobs0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:57:25.003621 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.999881Z","caller":"traceutil/trace.go:171","msg":"trace[1078084905] range","detail":"{range_begin:/registry/endpointslices/; range_end:/registry/endpointslices0; response_count:0; response_revision:1057; }","duration":"239.884462ms","start":"2025-07-15T10:57:24.759985Z","end":"2025-07-15T10:57:24.999870Z","steps":["trace[1078084905] 'agreement among raft nodes before linearized reading' (duration: 239.828845ms)"],"step_count":1} Jul 15 10:57:25.003621 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:24.999891Z","caller":"traceutil/trace.go:171","msg":"trace[20514071] range","detail":"{range_begin:/registry/cronjobs/; range_end:/registry/cronjobs0; response_count:0; response_revision:1057; }","duration":"3.294841422s","start":"2025-07-15T10:57:21.705039Z","end":"2025-07-15T10:57:24.999880Z","steps":["trace[20514071] 'agreement among raft nodes before linearized reading' (duration: 3.294792076s)"],"step_count":1} Jul 15 10:57:25.004855 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:24.999945Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:21.704471Z","time spent":"3.295444269s","remote":"172.24.4.149:33276","response type":"/etcdserverpb.KV/Range","request count":0,"request size":44,"response count":0,"response size":29,"request content":"key:\"/registry/cronjobs/\" range_end:\"/registry/cronjobs0\" count_only:true "} Jul 15 10:57:25.367518 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:25.366169Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"304.888286ms","expected-duration":"100ms","prefix":"","request":"header: lease_revoke:","response":"size:29"} Jul 15 10:57:25.368959 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:25.366819Z","caller":"traceutil/trace.go:171","msg":"trace[660330364] transaction","detail":"{read_only:false; response_revision:1058; number_of_response:1; }","duration":"362.465063ms","start":"2025-07-15T10:57:25.004336Z","end":"2025-07-15T10:57:25.366801Z","steps":["trace[660330364] 'process raft request' (duration: 362.253032ms)"],"step_count":1} Jul 15 10:57:25.368959 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:25.366867Z","caller":"traceutil/trace.go:171","msg":"trace[549063908] linearizableReadLoop","detail":"{readStateIndex:1084; appliedIndex:1083; }","duration":"363.666929ms","start":"2025-07-15T10:57:25.003131Z","end":"2025-07-15T10:57:25.366798Z","steps":["trace[549063908] 'read index received' (duration: 58.018912ms)","trace[549063908] 'applied index is now lower than readState.Index' (duration: 305.644531ms)"],"step_count":2} Jul 15 10:57:25.368959 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:25.366934Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:25.004295Z","time spent":"362.570315ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":792,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:57:25.368959 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:25.367020Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:25.003324Z","time spent":"363.693191ms","remote":"172.24.4.149:33052","response type":"/etcdserverpb.Lease/LeaseGrant","request count":-1,"request size":-1,"response count":-1,"response size":-1,"request content":""} Jul 15 10:57:25.369397 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:25.367290Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"364.145957ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:57:25.369397 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:25.367341Z","caller":"traceutil/trace.go:171","msg":"trace[400023787] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1058; }","duration":"364.231392ms","start":"2025-07-15T10:57:25.003093Z","end":"2025-07-15T10:57:25.367324Z","steps":["trace[400023787] 'agreement among raft nodes before linearized reading' (duration: 363.895743ms)"],"step_count":1} Jul 15 10:57:25.369397 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:25.367422Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:25.003072Z","time spent":"364.298021ms","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:57:25.692822 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:25.692076Z","caller":"traceutil/trace.go:171","msg":"trace[585059590] transaction","detail":"{read_only:false; response_revision:1061; number_of_response:1; }","duration":"203.061908ms","start":"2025-07-15T10:57:25.488971Z","end":"2025-07-15T10:57:25.692033Z","steps":["trace[585059590] 'process raft request' (duration: 141.646039ms)","trace[585059590] 'compare' (duration: 61.253394ms)"],"step_count":2} Jul 15 10:57:27.104312 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:27.103301Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"1.255582471s","expected-duration":"1s"} Jul 15 10:57:27.104312 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:27.103859Z","caller":"traceutil/trace.go:171","msg":"trace[1599980905] transaction","detail":"{read_only:false; response_revision:1064; number_of_response:1; }","duration":"1.256226469s","start":"2025-07-15T10:57:25.847615Z","end":"2025-07-15T10:57:27.103842Z","steps":["trace[1599980905] 'process raft request' (duration: 1.256122888s)"],"step_count":1} Jul 15 10:57:27.105622 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:27.104306Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:25.847591Z","time spent":"1.25654219s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":795,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:57:28.192757 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:28.191886Z","caller":"traceutil/trace.go:171","msg":"trace[276987785] transaction","detail":"{read_only:false; response_revision:1069; number_of_response:1; }","duration":"120.136432ms","start":"2025-07-15T10:57:28.071714Z","end":"2025-07-15T10:57:28.191851Z","steps":["trace[276987785] 'process raft request' (duration: 119.933009ms)"],"step_count":1} Jul 15 10:57:41.951305 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:41.947573Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"9.133079581s","expected-duration":"1s"} Jul 15 10:57:41.953218 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:41.938155Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:32.814255Z","time spent":"9.12382046s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 10:57:42.449503 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:42.448686Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"493.831731ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure:<>>","response":"size:16"} Jul 15 10:57:42.449503 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:42.448825Z","caller":"traceutil/trace.go:171","msg":"trace[1013416668] linearizableReadLoop","detail":"{readStateIndex:1111; appliedIndex:1110; }","duration":"502.600895ms","start":"2025-07-15T10:57:41.946200Z","end":"2025-07-15T10:57:42.448801Z","steps":["trace[1013416668] 'read index received' (duration: 8.447665ms)","trace[1013416668] 'applied index is now lower than readState.Index' (duration: 494.151837ms)"],"step_count":2} Jul 15 10:57:42.449503 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:42.449139Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"502.836631ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/ippools/\" range_end:\"/registry/crd.projectcalico.org/ippools0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:57:42.449981 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:42.449245Z","caller":"traceutil/trace.go:171","msg":"trace[1343880581] range","detail":"{range_begin:/registry/crd.projectcalico.org/ippools/; range_end:/registry/crd.projectcalico.org/ippools0; response_count:0; response_revision:1082; }","duration":"503.07435ms","start":"2025-07-15T10:57:41.946133Z","end":"2025-07-15T10:57:42.449208Z","steps":["trace[1343880581] 'agreement among raft nodes before linearized reading' (duration: 502.739013ms)"],"step_count":1} Jul 15 10:57:42.449981 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:42.449302Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.946105Z","time spent":"503.17794ms","remote":"172.24.4.149:54896","response type":"/etcdserverpb.KV/Range","request count":0,"request size":86,"response count":1,"response size":31,"request content":"key:\"/registry/crd.projectcalico.org/ippools/\" range_end:\"/registry/crd.projectcalico.org/ippools0\" count_only:true "} Jul 15 10:57:42.972510 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:42.971673Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072727,"retry-timeout":"500ms"} Jul 15 10:57:43.473007 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:43.472315Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072727,"retry-timeout":"500ms"} Jul 15 10:57:43.950738 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:43.949323Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000379143s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 10:57:43.950738 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:43.949713Z","caller":"traceutil/trace.go:171","msg":"trace[1563117451] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.000846478s","start":"2025-07-15T10:57:41.948834Z","end":"2025-07-15T10:57:43.949680Z","steps":["trace[1563117451] 'agreement among raft nodes before linearized reading' (duration: 2.000406476s)"],"step_count":1} Jul 15 10:57:43.950738 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:43.949870Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.948820Z","time spent":"2.00099396s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:57:43.973298 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:43.972752Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072727,"retry-timeout":"500ms"} Jul 15 10:57:44.473787 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:44.473103Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072727,"retry-timeout":"500ms"} Jul 15 10:57:44.973958 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:44.973308Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072727,"retry-timeout":"500ms"} Jul 15 10:57:45.475299 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:45.474598Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072727,"retry-timeout":"500ms"} Jul 15 10:57:45.975343 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:45.974792Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072727,"retry-timeout":"500ms"} Jul 15 10:57:46.475797 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:46.475343Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072727,"retry-timeout":"500ms"} Jul 15 10:57:46.951545 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:46.950460Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.996736326s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/tigera-operator/operator-lock\" ","response":"","error":"context deadline exceeded"} Jul 15 10:57:46.951545 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:46.950460Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.996683131s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-controller-manager\" ","response":"","error":"context deadline exceeded"} Jul 15 10:57:46.951545 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:46.950585Z","caller":"traceutil/trace.go:171","msg":"trace[110109203] range","detail":"{range_begin:/registry/leases/tigera-operator/operator-lock; range_end:; }","duration":"4.996851038s","start":"2025-07-15T10:57:41.953677Z","end":"2025-07-15T10:57:46.950529Z","steps":["trace[110109203] 'agreement among raft nodes before linearized reading' (duration: 4.996731667s)"],"step_count":1} Jul 15 10:57:46.951545 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:46.950648Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.953642Z","time spent":"4.996989467s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":0,"response size":0,"request content":"key:\"/registry/leases/tigera-operator/operator-lock\" "} Jul 15 10:57:46.951545 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:46.950636Z","caller":"traceutil/trace.go:171","msg":"trace[105723029] range","detail":"{range_begin:/registry/leases/kube-system/kube-controller-manager; range_end:; }","duration":"4.996866944s","start":"2025-07-15T10:57:41.953704Z","end":"2025-07-15T10:57:46.950571Z","steps":["trace[105723029] 'agreement among raft nodes before linearized reading' (duration: 4.99671329s)"],"step_count":1} Jul 15 10:57:46.951966 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:46.950708Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.953698Z","time spent":"4.996992518s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":54,"response count":0,"response size":0,"request content":"key:\"/registry/leases/kube-system/kube-controller-manager\" "} Jul 15 10:57:46.951966 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:46.950769Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.000029302s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-scheduler\" ","response":"","error":"context deadline exceeded"} Jul 15 10:57:46.951966 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:46.950826Z","caller":"traceutil/trace.go:171","msg":"trace[587334692] range","detail":"{range_begin:/registry/leases/kube-system/kube-scheduler; range_end:; }","duration":"5.000089666s","start":"2025-07-15T10:57:41.950701Z","end":"2025-07-15T10:57:46.950790Z","steps":["trace[587334692] 'agreement among raft nodes before linearized reading' (duration: 5.000057677s)"],"step_count":1} Jul 15 10:57:46.952263 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:46.951320Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.950659Z","time spent":"5.000237221s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":45,"response count":0,"response size":0,"request content":"key:\"/registry/leases/kube-system/kube-scheduler\" "} Jul 15 10:57:46.976012 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:46.975718Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072727,"retry-timeout":"500ms"} Jul 15 10:57:47.477252 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:47.476780Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072727,"retry-timeout":"500ms"} Jul 15 10:57:47.978061 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:47.977639Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072727,"retry-timeout":"500ms"} Jul 15 10:57:48.479498 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:48.478854Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072727,"retry-timeout":"500ms"} Jul 15 10:57:48.947996 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:48.946979Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.946378Z","time spent":"7.000557462s","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 10:57:48.952708 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:48.952131Z","caller":"etcdserver/server.go:1237","msg":"Failed to check current member's leadership","error":"context deadline exceeded"} Jul 15 10:57:48.953065 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:48.952595Z","caller":"etcdserver/server.go:1177","msg":"Ignore the lease revoking request because current member isn't a leader","local-member-id":10276657743932975437} Jul 15 10:57:48.954663 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:48.954356Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.953576Z","time spent":"7.000773278s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 10:57:48.954948 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:48.954470Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.953438Z","time spent":"7.001020578s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 10:57:48.980515 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:48.980067Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072727,"retry-timeout":"500ms"} Jul 15 10:57:49.451750 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.449863Z","caller":"etcdserver/v3_server.go:932","msg":"timed out waiting for read index response (local node might have slow network)","timeout":"7s"} Jul 15 10:57:49.451750 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.450081Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.124324474s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/horizontalpodautoscalers/\" range_end:\"/registry/horizontalpodautoscalers0\" count_only:true ","response":"","error":"etcdserver: request timed out"} Jul 15 10:57:49.451750 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:49.450154Z","caller":"traceutil/trace.go:171","msg":"trace[566985371] range","detail":"{range_begin:/registry/horizontalpodautoscalers/; range_end:/registry/horizontalpodautoscalers0; }","duration":"7.124455671s","start":"2025-07-15T10:57:42.325657Z","end":"2025-07-15T10:57:49.450113Z","steps":["trace[566985371] 'agreement among raft nodes before linearized reading' (duration: 7.124350367s)"],"step_count":1} Jul 15 10:57:49.451750 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.450215Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:42.325623Z","time spent":"7.12457225s","remote":"172.24.4.149:33250","response type":"/etcdserverpb.KV/Range","request count":0,"request size":76,"response count":0,"response size":0,"request content":"key:\"/registry/horizontalpodautoscalers/\" range_end:\"/registry/horizontalpodautoscalers0\" count_only:true "} Jul 15 10:57:49.451750 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.450302Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.490664599s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/calico-apiserver/calico-apiserver-5cdf69b9c8-rxt2t.1852678727ec63ba\" ","response":"","error":"etcdserver: request timed out"} Jul 15 10:57:49.452191 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:49.450326Z","caller":"traceutil/trace.go:171","msg":"trace[1815259627] range","detail":"{range_begin:/registry/events/calico-apiserver/calico-apiserver-5cdf69b9c8-rxt2t.1852678727ec63ba; range_end:; }","duration":"7.490793997s","start":"2025-07-15T10:57:41.959524Z","end":"2025-07-15T10:57:49.450319Z","steps":["trace[1815259627] 'agreement among raft nodes before linearized reading' (duration: 7.490770234s)"],"step_count":1} Jul 15 10:57:49.452191 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.450357Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.959504Z","time spent":"7.490842951s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":86,"response count":0,"response size":0,"request content":"key:\"/registry/events/calico-apiserver/calico-apiserver-5cdf69b9c8-rxt2t.1852678727ec63ba\" "} Jul 15 10:57:49.452191 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.450436Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.500187699s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"","error":"etcdserver: request timed out"} Jul 15 10:57:49.452191 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:49.450455Z","caller":"traceutil/trace.go:171","msg":"trace[2071061382] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; }","duration":"7.50023532s","start":"2025-07-15T10:57:41.950214Z","end":"2025-07-15T10:57:49.450449Z","steps":["trace[2071061382] 'agreement among raft nodes before linearized reading' (duration: 7.500214452s)"],"step_count":1} Jul 15 10:57:49.452612 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.450677Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.502972693s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/validatingadmissionpolicies/\" range_end:\"/registry/validatingadmissionpolicies0\" count_only:true ","response":"","error":"etcdserver: request timed out"} Jul 15 10:57:49.452612 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:49.450738Z","caller":"traceutil/trace.go:171","msg":"trace[176045697] range","detail":"{range_begin:/registry/validatingadmissionpolicies/; range_end:/registry/validatingadmissionpolicies0; }","duration":"7.503059184s","start":"2025-07-15T10:57:41.947661Z","end":"2025-07-15T10:57:49.450721Z","steps":["trace[176045697] 'agreement among raft nodes before linearized reading' (duration: 7.502971301s)"],"step_count":1} Jul 15 10:57:49.452612 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.450803Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.947584Z","time spent":"7.503176087s","remote":"172.24.4.149:33528","response type":"/etcdserverpb.KV/Range","request count":0,"request size":82,"response count":0,"response size":0,"request content":"key:\"/registry/validatingadmissionpolicies/\" range_end:\"/registry/validatingadmissionpolicies0\" count_only:true "} Jul 15 10:57:49.452612 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.450871Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.501188212s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/validatingwebhookconfigurations/\" range_end:\"/registry/validatingwebhookconfigurations0\" count_only:true ","response":"","error":"etcdserver: request timed out"} Jul 15 10:57:49.452834 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:49.450895Z","caller":"traceutil/trace.go:171","msg":"trace[1561416844] range","detail":"{range_begin:/registry/validatingwebhookconfigurations/; range_end:/registry/validatingwebhookconfigurations0; }","duration":"7.501241554s","start":"2025-07-15T10:57:41.949646Z","end":"2025-07-15T10:57:49.450887Z","steps":["trace[1561416844] 'agreement among raft nodes before linearized reading' (duration: 7.501215746s)"],"step_count":1} Jul 15 10:57:49.452834 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.450898Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.950198Z","time spent":"7.500276378s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":0,"response size":0,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 10:57:49.452834 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.450921Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.949636Z","time spent":"7.501275466s","remote":"172.24.4.149:33504","response type":"/etcdserverpb.KV/Range","request count":0,"request size":90,"response count":0,"response size":0,"request content":"key:\"/registry/validatingwebhookconfigurations/\" range_end:\"/registry/validatingwebhookconfigurations0\" count_only:true "} Jul 15 10:57:49.452834 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.450956Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.503453828s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"","error":"etcdserver: request timed out"} Jul 15 10:57:49.452834 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.450969Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.5014547s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/configmaps/\" range_end:\"/registry/configmaps0\" count_only:true ","response":"","error":"etcdserver: request timed out"} Jul 15 10:57:49.453299 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:49.450976Z","caller":"traceutil/trace.go:171","msg":"trace[310600133] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; }","duration":"7.503494726s","start":"2025-07-15T10:57:41.947473Z","end":"2025-07-15T10:57:49.450968Z","steps":["trace[310600133] 'agreement among raft nodes before linearized reading' (duration: 7.503475239s)"],"step_count":1} Jul 15 10:57:49.453299 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:49.450990Z","caller":"traceutil/trace.go:171","msg":"trace[111550721] range","detail":"{range_begin:/registry/configmaps/; range_end:/registry/configmaps0; }","duration":"7.501512289s","start":"2025-07-15T10:57:41.949471Z","end":"2025-07-15T10:57:49.450983Z","steps":["trace[111550721] 'agreement among raft nodes before linearized reading' (duration: 7.501489528s)"],"step_count":1} Jul 15 10:57:49.453299 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451011Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.947462Z","time spent":"7.503537998s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":0,"response size":0,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:57:49.453299 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451030Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.949460Z","time spent":"7.501560581s","remote":"172.24.4.149:33128","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":0,"response size":0,"request content":"key:\"/registry/configmaps/\" range_end:\"/registry/configmaps0\" count_only:true "} Jul 15 10:57:49.453299 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451062Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.503763822s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowschemas/\" range_end:\"/registry/flowschemas0\" count_only:true ","response":"","error":"etcdserver: request timed out"} Jul 15 10:57:49.453579 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451084Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.501917858s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/validatingadmissionpolicybindings/\" range_end:\"/registry/validatingadmissionpolicybindings0\" count_only:true ","response":"","error":"etcdserver: request timed out"} Jul 15 10:57:49.453579 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:49.451083Z","caller":"traceutil/trace.go:171","msg":"trace[541194865] range","detail":"{range_begin:/registry/flowschemas/; range_end:/registry/flowschemas0; }","duration":"7.503828204s","start":"2025-07-15T10:57:41.947248Z","end":"2025-07-15T10:57:49.451077Z","steps":["trace[541194865] 'agreement among raft nodes before linearized reading' (duration: 7.503795654s)"],"step_count":1} Jul 15 10:57:49.453579 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:49.451107Z","caller":"traceutil/trace.go:171","msg":"trace[2107402129] range","detail":"{range_begin:/registry/validatingadmissionpolicybindings/; range_end:/registry/validatingadmissionpolicybindings0; }","duration":"7.502012059s","start":"2025-07-15T10:57:41.949088Z","end":"2025-07-15T10:57:49.451100Z","steps":["trace[2107402129] 'agreement among raft nodes before linearized reading' (duration: 7.501985259s)"],"step_count":1} Jul 15 10:57:49.453579 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451113Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.947236Z","time spent":"7.503868491s","remote":"172.24.4.149:33438","response type":"/etcdserverpb.KV/Range","request count":0,"request size":50,"response count":0,"response size":0,"request content":"key:\"/registry/flowschemas/\" range_end:\"/registry/flowschemas0\" count_only:true "} Jul 15 10:57:49.453968 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451160Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.949078Z","time spent":"7.502054458s","remote":"172.24.4.149:33530","response type":"/etcdserverpb.KV/Range","request count":0,"request size":94,"response count":0,"response size":0,"request content":"key:\"/registry/validatingadmissionpolicybindings/\" range_end:\"/registry/validatingadmissionpolicybindings0\" count_only:true "} Jul 15 10:57:49.453968 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451187Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.503898762s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"","error":"etcdserver: request timed out"} Jul 15 10:57:49.453968 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:49.451207Z","caller":"traceutil/trace.go:171","msg":"trace[1734705962] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; }","duration":"7.503957614s","start":"2025-07-15T10:57:41.947243Z","end":"2025-07-15T10:57:49.451200Z","steps":["trace[1734705962] 'agreement among raft nodes before linearized reading' (duration: 7.503936154s)"],"step_count":1} Jul 15 10:57:49.453968 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451224Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.947230Z","time spent":"7.503988332s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":0,"response size":0,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:57:49.453968 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451247Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.504330133s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/whiskers/\" range_end:\"/registry/operator.tigera.io/whiskers0\" count_only:true ","response":"","error":"etcdserver: request timed out"} Jul 15 10:57:49.454191 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451242Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.50289934s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/\" range_end:\"/registry/apiextensions.k8s.io/customresourcedefinitions0\" count_only:true ","response":"","error":"etcdserver: request timed out"} Jul 15 10:57:49.454191 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:49.451273Z","caller":"traceutil/trace.go:171","msg":"trace[1940185803] range","detail":"{range_begin:/registry/operator.tigera.io/whiskers/; range_end:/registry/operator.tigera.io/whiskers0; }","duration":"7.504372995s","start":"2025-07-15T10:57:41.946883Z","end":"2025-07-15T10:57:49.451256Z","steps":["trace[1940185803] 'agreement among raft nodes before linearized reading' (duration: 7.504357827s)"],"step_count":1} Jul 15 10:57:49.454191 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:49.451270Z","caller":"traceutil/trace.go:171","msg":"trace[1572693557] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/; range_end:/registry/apiextensions.k8s.io/customresourcedefinitions0; }","duration":"7.502967308s","start":"2025-07-15T10:57:41.948294Z","end":"2025-07-15T10:57:49.451261Z","steps":["trace[1572693557] 'agreement among raft nodes before linearized reading' (duration: 7.502930159s)"],"step_count":1} Jul 15 10:57:49.454191 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451296Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.946844Z","time spent":"7.504439852s","remote":"172.24.4.149:54978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":82,"response count":0,"response size":0,"request content":"key:\"/registry/operator.tigera.io/whiskers/\" range_end:\"/registry/operator.tigera.io/whiskers0\" count_only:true "} Jul 15 10:57:49.454331 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451318Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.948281Z","time spent":"7.503015218s","remote":"172.24.4.149:33078","response type":"/etcdserverpb.KV/Range","request count":0,"request size":120,"response count":0,"response size":0,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/\" range_end:\"/registry/apiextensions.k8s.io/customresourcedefinitions0\" count_only:true "} Jul 15 10:57:49.454331 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451334Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.504661056s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/csidrivers/\" range_end:\"/registry/csidrivers0\" count_only:true ","response":"","error":"etcdserver: request timed out"} Jul 15 10:57:49.454331 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:49.451350Z","caller":"traceutil/trace.go:171","msg":"trace[1797327814] range","detail":"{range_begin:/registry/csidrivers/; range_end:/registry/csidrivers0; }","duration":"7.504724058s","start":"2025-07-15T10:57:41.946620Z","end":"2025-07-15T10:57:49.451344Z","steps":["trace[1797327814] 'agreement among raft nodes before linearized reading' (duration: 7.504706916s)"],"step_count":1} Jul 15 10:57:49.454331 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451371Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.503329851s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/\" range_end:\"/registry/clusterrolebindings0\" count_only:true ","response":"","error":"etcdserver: request timed out"} Jul 15 10:57:49.454533 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451399Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.946607Z","time spent":"7.504759274s","remote":"172.24.4.149:33412","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":0,"response size":0,"request content":"key:\"/registry/csidrivers/\" range_end:\"/registry/csidrivers0\" count_only:true "} Jul 15 10:57:49.454533 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:49.451419Z","caller":"traceutil/trace.go:171","msg":"trace[309546637] range","detail":"{range_begin:/registry/clusterrolebindings/; range_end:/registry/clusterrolebindings0; }","duration":"7.503407859s","start":"2025-07-15T10:57:41.948004Z","end":"2025-07-15T10:57:49.451412Z","steps":["trace[309546637] 'agreement among raft nodes before linearized reading' (duration: 7.503358107s)"],"step_count":1} Jul 15 10:57:49.454533 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.451456Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:41.947893Z","time spent":"7.503553127s","remote":"172.24.4.149:33376","response type":"/etcdserverpb.KV/Range","request count":0,"request size":66,"response count":0,"response size":0,"request content":"key:\"/registry/clusterrolebindings/\" range_end:\"/registry/clusterrolebindings0\" count_only:true "} Jul 15 10:57:49.950105 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:49.949270Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"7.639157503s","expected-duration":"1s"} Jul 15 10:57:50.031264 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.030791Z","caller":"traceutil/trace.go:171","msg":"trace[946865317] linearizableReadLoop","detail":"{readStateIndex:1115; appliedIndex:1111; }","duration":"580.794746ms","start":"2025-07-15T10:57:49.449975Z","end":"2025-07-15T10:57:50.030770Z","steps":["trace[946865317] 'read index received' (duration: 501.07807ms)","trace[946865317] 'applied index is now lower than readState.Index' (duration: 79.715674ms)"],"step_count":2} Jul 15 10:57:50.105640 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.105189Z","caller":"traceutil/trace.go:171","msg":"trace[2021144779] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1086; }","duration":"114.195717ms","start":"2025-07-15T10:57:49.990969Z","end":"2025-07-15T10:57:50.105165Z","steps":["trace[2021144779] 'process raft request' (duration: 113.769106ms)"],"step_count":1} Jul 15 10:57:50.106100 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.105884Z","caller":"traceutil/trace.go:171","msg":"trace[2015690181] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1086; }","duration":"100.306997ms","start":"2025-07-15T10:57:50.005566Z","end":"2025-07-15T10:57:50.105873Z","steps":["trace[2015690181] 'process raft request' (duration: 99.264084ms)"],"step_count":1} Jul 15 10:57:50.141659 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.141167Z","caller":"traceutil/trace.go:171","msg":"trace[1845532206] linearizableReadLoop","detail":"{readStateIndex:1118; appliedIndex:1116; }","duration":"110.271913ms","start":"2025-07-15T10:57:50.030855Z","end":"2025-07-15T10:57:50.141126Z","steps":["trace[1845532206] 'read index received' (duration: 35.039449ms)","trace[1845532206] 'applied index is now lower than readState.Index' (duration: 75.23054ms)"],"step_count":2} Jul 15 10:57:50.142588 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:50.142172Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"163.093609ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/runtimeclasses/\" range_end:\"/registry/runtimeclasses0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:57:50.143126 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.142862Z","caller":"traceutil/trace.go:171","msg":"trace[2139925959] range","detail":"{range_begin:/registry/runtimeclasses/; range_end:/registry/runtimeclasses0; response_count:0; response_revision:1087; }","duration":"163.83889ms","start":"2025-07-15T10:57:49.978997Z","end":"2025-07-15T10:57:50.142836Z","steps":["trace[2139925959] 'agreement among raft nodes before linearized reading' (duration: 162.3604ms)"],"step_count":1} Jul 15 10:57:50.143713 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:50.143453Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"208.317862ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/resourcequotas/\" range_end:\"/registry/resourcequotas0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:57:50.144564 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:50.144265Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"143.575901ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/calico-system/calico-kube-controllers-66dd988f77-4fzng.18526787257665a5\" ","response":"range_response_count:1 size:1091"} Jul 15 10:57:50.144564 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.144462Z","caller":"traceutil/trace.go:171","msg":"trace[1814143083] range","detail":"{range_begin:/registry/resourcequotas/; range_end:/registry/resourcequotas0; response_count:0; response_revision:1087; }","duration":"208.417428ms","start":"2025-07-15T10:57:49.935080Z","end":"2025-07-15T10:57:50.143498Z","steps":["trace[1814143083] 'agreement among raft nodes before linearized reading' (duration: 208.251751ms)"],"step_count":1} Jul 15 10:57:50.146559 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.144785Z","caller":"traceutil/trace.go:171","msg":"trace[1932225810] range","detail":"{range_begin:/registry/events/calico-system/calico-kube-controllers-66dd988f77-4fzng.18526787257665a5; range_end:; response_count:1; response_revision:1087; }","duration":"143.65683ms","start":"2025-07-15T10:57:50.000659Z","end":"2025-07-15T10:57:50.144316Z","steps":["trace[1932225810] 'agreement among raft nodes before linearized reading' (duration: 143.421044ms)"],"step_count":1} Jul 15 10:57:50.147935 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:50.145358Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"210.623134ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/\" range_end:\"/registry/events0\" count_only:true ","response":"range_response_count:0 size:8"} Jul 15 10:57:50.147935 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.145415Z","caller":"traceutil/trace.go:171","msg":"trace[1312959105] range","detail":"{range_begin:/registry/events/; range_end:/registry/events0; response_count:0; response_revision:1087; }","duration":"210.708001ms","start":"2025-07-15T10:57:49.934691Z","end":"2025-07-15T10:57:50.145399Z","steps":["trace[1312959105] 'agreement among raft nodes before linearized reading' (duration: 210.046325ms)"],"step_count":1} Jul 15 10:57:50.147935 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:50.145464Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"211.152525ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/certificatesigningrequests/\" range_end:\"/registry/certificatesigningrequests0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:57:50.147935 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.145491Z","caller":"traceutil/trace.go:171","msg":"trace[2145285422] range","detail":"{range_begin:/registry/certificatesigningrequests/; range_end:/registry/certificatesigningrequests0; response_count:0; response_revision:1087; }","duration":"211.199761ms","start":"2025-07-15T10:57:49.934275Z","end":"2025-07-15T10:57:50.145475Z","steps":["trace[2145285422] 'agreement among raft nodes before linearized reading' (duration: 211.171439ms)"],"step_count":1} Jul 15 10:57:50.147935 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:50.145560Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"211.29627ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/limitranges/\" range_end:\"/registry/limitranges0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:57:50.148704 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.145578Z","caller":"traceutil/trace.go:171","msg":"trace[1119631945] range","detail":"{range_begin:/registry/limitranges/; range_end:/registry/limitranges0; response_count:0; response_revision:1087; }","duration":"211.370829ms","start":"2025-07-15T10:57:49.934201Z","end":"2025-07-15T10:57:50.145572Z","steps":["trace[1119631945] 'agreement among raft nodes before linearized reading' (duration: 211.34407ms)"],"step_count":1} Jul 15 10:57:50.148704 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:50.145658Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"212.409952ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rolebindings/\" range_end:\"/registry/rolebindings0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:57:50.148704 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.145675Z","caller":"traceutil/trace.go:171","msg":"trace[1503354585] range","detail":"{range_begin:/registry/rolebindings/; range_end:/registry/rolebindings0; response_count:0; response_revision:1087; }","duration":"212.484961ms","start":"2025-07-15T10:57:49.933185Z","end":"2025-07-15T10:57:50.145670Z","steps":["trace[1503354585] 'agreement among raft nodes before linearized reading' (duration: 212.444907ms)"],"step_count":1} Jul 15 10:57:50.148704 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:50.145820Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"163.924152ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:0 size:5"} Jul 15 10:57:50.148704 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.145840Z","caller":"traceutil/trace.go:171","msg":"trace[1387924139] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:0; response_revision:1087; }","duration":"163.957363ms","start":"2025-07-15T10:57:49.981876Z","end":"2025-07-15T10:57:50.145834Z","steps":["trace[1387924139] 'agreement among raft nodes before linearized reading' (duration: 163.927478ms)"],"step_count":1} Jul 15 10:57:50.150704 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:50.145955Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"160.881989ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:57:50.150704 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.145977Z","caller":"traceutil/trace.go:171","msg":"trace[110806406] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1087; }","duration":"160.995709ms","start":"2025-07-15T10:57:49.984976Z","end":"2025-07-15T10:57:50.145972Z","steps":["trace[110806406] 'agreement among raft nodes before linearized reading' (duration: 160.965093ms)"],"step_count":1} Jul 15 10:57:50.150704 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:50.145935Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"154.699991ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:57:50.150704 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:50.146122Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"166.026074ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/csistoragecapacities/\" range_end:\"/registry/csistoragecapacities0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:57:50.150704 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.146149Z","caller":"traceutil/trace.go:171","msg":"trace[1440356476] range","detail":"{range_begin:/registry/csistoragecapacities/; range_end:/registry/csistoragecapacities0; response_count:0; response_revision:1087; }","duration":"166.067841ms","start":"2025-07-15T10:57:49.980073Z","end":"2025-07-15T10:57:50.146141Z","steps":["trace[1440356476] 'agreement among raft nodes before linearized reading' (duration: 166.018499ms)"],"step_count":1} Jul 15 10:57:50.151211 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.146120Z","caller":"traceutil/trace.go:171","msg":"trace[1212384004] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1087; }","duration":"154.842075ms","start":"2025-07-15T10:57:49.991184Z","end":"2025-07-15T10:57:50.146026Z","steps":["trace[1212384004] 'agreement among raft nodes before linearized reading' (duration: 154.368147ms)"],"step_count":1} Jul 15 10:57:50.437580 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:50.436666Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"258.988985ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76\" ","response":"range_response_count:1 size:905"} Jul 15 10:57:50.437580 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.436855Z","caller":"traceutil/trace.go:171","msg":"trace[52414486] range","detail":"{range_begin:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76; range_end:; response_count:1; response_revision:1092; }","duration":"259.196048ms","start":"2025-07-15T10:57:50.177572Z","end":"2025-07-15T10:57:50.436768Z","steps":["trace[52414486] 'agreement among raft nodes before linearized reading' (duration: 76.964404ms)","trace[52414486] 'range keys from in-memory index tree' (duration: 181.65213ms)"],"step_count":2} Jul 15 10:57:50.465269 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.464618Z","caller":"traceutil/trace.go:171","msg":"trace[1424370151] transaction","detail":"{read_only:false; response_revision:1093; number_of_response:1; }","duration":"233.505232ms","start":"2025-07-15T10:57:50.231065Z","end":"2025-07-15T10:57:50.464570Z","steps":["trace[1424370151] 'process raft request' (duration: 233.097688ms)"],"step_count":1} Jul 15 10:57:50.466943 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.465649Z","caller":"traceutil/trace.go:171","msg":"trace[1964631789] linearizableReadLoop","detail":"{readStateIndex:1126; appliedIndex:1126; }","duration":"204.143185ms","start":"2025-07-15T10:57:50.261485Z","end":"2025-07-15T10:57:50.465628Z","steps":["trace[1964631789] 'read index received' (duration: 204.13503ms)","trace[1964631789] 'applied index is now lower than readState.Index' (duration: 6.793µs)"],"step_count":2} Jul 15 10:57:50.467117 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:50.465881Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"204.377829ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/endpoints/default/kubernetes\" ","response":"range_response_count:1 size:420"} Jul 15 10:57:50.467117 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.465995Z","caller":"traceutil/trace.go:171","msg":"trace[1753529918] range","detail":"{range_begin:/registry/services/endpoints/default/kubernetes; range_end:; response_count:1; response_revision:1093; }","duration":"204.597257ms","start":"2025-07-15T10:57:50.261377Z","end":"2025-07-15T10:57:50.465975Z","steps":["trace[1753529918] 'agreement among raft nodes before linearized reading' (duration: 204.400181ms)"],"step_count":1} Jul 15 10:57:50.489096 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:50.488258Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"218.524634ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/foo\" ","response":"range_response_count:0 size:5"} Jul 15 10:57:50.489096 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:50.488488Z","caller":"traceutil/trace.go:171","msg":"trace[1268645172] range","detail":"{range_begin:/registry/minions/foo; range_end:; response_count:0; response_revision:1094; }","duration":"218.837624ms","start":"2025-07-15T10:57:50.269616Z","end":"2025-07-15T10:57:50.488453Z","steps":["trace[1268645172] 'agreement among raft nodes before linearized reading' (duration: 218.391387ms)"],"step_count":1} Jul 15 10:57:51.624675 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:51.623851Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072751,"retry-timeout":"500ms"} Jul 15 10:57:53.708541 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:53.620579Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072751,"retry-timeout":"500ms"} Jul 15 10:57:53.708541 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:53.707317Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"3.209357589s","expected-duration":"1s"} Jul 15 10:57:53.910795 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:53.909762Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"201.822337ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure:<>>","response":"size:16"} Jul 15 10:57:53.910795 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:53.909924Z","caller":"traceutil/trace.go:171","msg":"trace[1124012878] transaction","detail":"{read_only:false; response_revision:1095; number_of_response:1; }","duration":"3.412292241s","start":"2025-07-15T10:57:50.497605Z","end":"2025-07-15T10:57:53.909898Z","steps":["trace[1124012878] 'process raft request' (duration: 3.210179686s)","trace[1124012878] 'compare' (duration: 201.464932ms)"],"step_count":2} Jul 15 10:57:53.923876 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:53.910015Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:50.497569Z","time spent":"3.412403507s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":889,"response count":0,"response size":40,"request content":"compare: success:> failure:<>"} Jul 15 10:57:53.962138 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:53.960904Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"339.795562ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 10:57:53.962138 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:53.961137Z","caller":"traceutil/trace.go:171","msg":"trace[46884456] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"340.054944ms","start":"2025-07-15T10:57:53.621036Z","end":"2025-07-15T10:57:53.961091Z","steps":["trace[46884456] 'agreement among raft nodes before linearized reading' (duration: 339.819737ms)"],"step_count":1} Jul 15 10:57:53.962138 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:53.961530Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:53.621032Z","time spent":"340.445299ms","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:57:53.970491 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:53.968337Z","caller":"traceutil/trace.go:171","msg":"trace[1314284327] linearizableReadLoop","detail":"{readStateIndex:1129; appliedIndex:1127; }","duration":"3.422367626s","start":"2025-07-15T10:57:50.545939Z","end":"2025-07-15T10:57:53.968306Z","steps":["trace[1314284327] 'read index received' (duration: 3.162354687s)","trace[1314284327] 'applied index is now lower than readState.Index' (duration: 260.011657ms)"],"step_count":2} Jul 15 10:57:53.970491 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:53.968579Z","caller":"traceutil/trace.go:171","msg":"trace[361212832] transaction","detail":"{read_only:false; response_revision:1096; number_of_response:1; }","duration":"3.444444626s","start":"2025-07-15T10:57:50.524105Z","end":"2025-07-15T10:57:53.968550Z","steps":["trace[361212832] 'process raft request' (duration: 3.44392958s)"],"step_count":1} Jul 15 10:57:53.970491 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:53.968793Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:50.524076Z","time spent":"3.444560221s","remote":"172.24.4.149:33206","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":6403,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:57:53.971088 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:53.969044Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.423099917s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"range_response_count:1 size:814"} Jul 15 10:57:53.971088 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:53.969103Z","caller":"traceutil/trace.go:171","msg":"trace[557714348] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; response_count:1; response_revision:1096; }","duration":"3.423206714s","start":"2025-07-15T10:57:50.545880Z","end":"2025-07-15T10:57:53.969087Z","steps":["trace[557714348] 'agreement among raft nodes before linearized reading' (duration: 3.423024827s)"],"step_count":1} Jul 15 10:57:53.971088 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:53.969155Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:50.545847Z","time spent":"3.423285481s","remote":"172.24.4.149:54994","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":1,"response size":838,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" "} Jul 15 10:57:53.971088 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:53.969350Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.380108344s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/\" range_end:\"/registry/minions0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:57:53.971088 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:53.969484Z","caller":"traceutil/trace.go:171","msg":"trace[926453544] range","detail":"{range_begin:/registry/minions/; range_end:/registry/minions0; response_count:0; response_revision:1096; }","duration":"3.380351605s","start":"2025-07-15T10:57:50.589106Z","end":"2025-07-15T10:57:53.969457Z","steps":["trace[926453544] 'agreement among raft nodes before linearized reading' (duration: 3.38012249s)"],"step_count":1} Jul 15 10:57:53.971758 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:53.969581Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:50.589038Z","time spent":"3.380507636s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Range","request count":0,"request size":42,"response count":1,"response size":31,"request content":"key:\"/registry/minions/\" range_end:\"/registry/minions0\" count_only:true "} Jul 15 10:57:53.971758 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:53.969727Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"348.772951ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/goldmanes/\" range_end:\"/registry/operator.tigera.io/goldmanes0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:57:53.971758 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:53.969770Z","caller":"traceutil/trace.go:171","msg":"trace[211442532] range","detail":"{range_begin:/registry/operator.tigera.io/goldmanes/; range_end:/registry/operator.tigera.io/goldmanes0; response_count:0; response_revision:1096; }","duration":"348.843583ms","start":"2025-07-15T10:57:53.620912Z","end":"2025-07-15T10:57:53.969755Z","steps":["trace[211442532] 'agreement among raft nodes before linearized reading' (duration: 348.760768ms)"],"step_count":1} Jul 15 10:57:53.971758 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:53.969818Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:57:53.620892Z","time spent":"348.905397ms","remote":"172.24.4.149:54972","response type":"/etcdserverpb.KV/Range","request count":0,"request size":84,"response count":1,"response size":31,"request content":"key:\"/registry/operator.tigera.io/goldmanes/\" range_end:\"/registry/operator.tigera.io/goldmanes0\" count_only:true "} Jul 15 10:57:54.126220 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:54.124730Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"114.895558ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 10:57:54.126220 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:54.124873Z","caller":"traceutil/trace.go:171","msg":"trace[89065362] linearizableReadLoop","detail":"{readStateIndex:1130; appliedIndex:1129; }","duration":"125.659733ms","start":"2025-07-15T10:57:53.999192Z","end":"2025-07-15T10:57:54.124852Z","steps":["trace[89065362] 'read index received' (duration: 10.409206ms)","trace[89065362] 'applied index is now lower than readState.Index' (duration: 115.249355ms)"],"step_count":2} Jul 15 10:57:54.126220 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:54.124996Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"125.802058ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:57:54.126220 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:54.125027Z","caller":"traceutil/trace.go:171","msg":"trace[1414542103] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1097; }","duration":"125.877478ms","start":"2025-07-15T10:57:53.999138Z","end":"2025-07-15T10:57:54.125015Z","steps":["trace[1414542103] 'agreement among raft nodes before linearized reading' (duration: 125.818168ms)"],"step_count":1} Jul 15 10:57:54.128318 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:54.125034Z","caller":"traceutil/trace.go:171","msg":"trace[1527275206] transaction","detail":"{read_only:false; response_revision:1097; number_of_response:1; }","duration":"143.227031ms","start":"2025-07-15T10:57:53.981734Z","end":"2025-07-15T10:57:54.124961Z","steps":["trace[1527275206] 'process raft request' (duration: 27.985561ms)","trace[1527275206] 'compare' (duration: 114.70275ms)"],"step_count":2} Jul 15 10:57:54.128318 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:57:54.125260Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"118.544978ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/ci-4230-2-100-6ef5831daf.novalocal\" ","response":"range_response_count:1 size:4695"} Jul 15 10:57:54.128318 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:54.125297Z","caller":"traceutil/trace.go:171","msg":"trace[1411215523] range","detail":"{range_begin:/registry/minions/ci-4230-2-100-6ef5831daf.novalocal; range_end:; response_count:1; response_revision:1097; }","duration":"118.657077ms","start":"2025-07-15T10:57:54.006630Z","end":"2025-07-15T10:57:54.125287Z","steps":["trace[1411215523] 'agreement among raft nodes before linearized reading' (duration: 118.499594ms)"],"step_count":1} Jul 15 10:57:54.596291 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:54.595580Z","caller":"traceutil/trace.go:171","msg":"trace[1618382108] transaction","detail":"{read_only:false; response_revision:1108; number_of_response:1; }","duration":"144.987284ms","start":"2025-07-15T10:57:54.450531Z","end":"2025-07-15T10:57:54.595519Z","steps":["trace[1618382108] 'process raft request' (duration: 72.550353ms)","trace[1618382108] 'compare' (duration: 71.541327ms)"],"step_count":2} Jul 15 10:57:54.618895 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:54.617830Z","caller":"traceutil/trace.go:171","msg":"trace[89945780] transaction","detail":"{read_only:false; response_revision:1109; number_of_response:1; }","duration":"103.999562ms","start":"2025-07-15T10:57:54.513795Z","end":"2025-07-15T10:57:54.617794Z","steps":["trace[89945780] 'process raft request' (duration: 103.805842ms)"],"step_count":1} Jul 15 10:57:54.762873 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:54.761899Z","caller":"traceutil/trace.go:171","msg":"trace[259943267] transaction","detail":"{read_only:false; number_of_response:0; response_revision:1112; }","duration":"112.338324ms","start":"2025-07-15T10:57:54.649535Z","end":"2025-07-15T10:57:54.761873Z","steps":["trace[259943267] 'process raft request' (duration: 112.280246ms)"],"step_count":1} Jul 15 10:57:54.764720 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:57:54.762454Z","caller":"traceutil/trace.go:171","msg":"trace[898613441] transaction","detail":"{read_only:false; response_revision:1112; number_of_response:1; }","duration":"118.857246ms","start":"2025-07-15T10:57:54.643510Z","end":"2025-07-15T10:57:54.762367Z","steps":["trace[898613441] 'process raft request' (duration: 72.346593ms)","trace[898613441] 'compare' (duration: 45.762644ms)"],"step_count":2} Jul 15 10:58:04.549716 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.207125Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885072876,"retry-timeout":"500ms"} Jul 15 10:58:04.709952 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.705984Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.43172049s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/poddisruptionbudgets/\" range_end:\"/registry/poddisruptionbudgets0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:58:04.709952 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.706164Z","caller":"traceutil/trace.go:171","msg":"trace[1833818061] range","detail":"{range_begin:/registry/poddisruptionbudgets/; range_end:/registry/poddisruptionbudgets0; response_count:0; response_revision:1130; }","duration":"4.432000894s","start":"2025-07-15T10:58:00.274109Z","end":"2025-07-15T10:58:04.706110Z","steps":["trace[1833818061] 'count revisions from in-memory index tree' (duration: 4.431551985s)"],"step_count":1} Jul 15 10:58:04.709952 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.706276Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:00.274076Z","time spent":"4.432174828s","remote":"172.24.4.149:33346","response type":"/etcdserverpb.KV/Range","request count":0,"request size":68,"response count":2,"response size":31,"request content":"key:\"/registry/poddisruptionbudgets/\" range_end:\"/registry/poddisruptionbudgets0\" count_only:true "} Jul 15 10:58:04.709952 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.706287Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.05123147s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/persistentvolumes/\" range_end:\"/registry/persistentvolumes0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:58:04.709952 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.706786Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.433488099s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/roles/\" range_end:\"/registry/roles0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:58:04.711118 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.706858Z","caller":"traceutil/trace.go:171","msg":"trace[1218199639] range","detail":"{range_begin:/registry/roles/; range_end:/registry/roles0; response_count:0; response_revision:1130; }","duration":"4.433677474s","start":"2025-07-15T10:58:00.273164Z","end":"2025-07-15T10:58:04.706842Z","steps":["trace[1218199639] 'count revisions from in-memory index tree' (duration: 4.433040724s)"],"step_count":1} Jul 15 10:58:04.711118 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.706918Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:00.273141Z","time spent":"4.433752323s","remote":"172.24.4.149:33348","response type":"/etcdserverpb.KV/Range","request count":0,"request size":38,"response count":12,"response size":31,"request content":"key:\"/registry/roles/\" range_end:\"/registry/roles0\" count_only:true "} Jul 15 10:58:04.711118 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.707181Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.434111854s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 10:58:04.711118 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.707240Z","caller":"traceutil/trace.go:171","msg":"trace[971331662] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1130; }","duration":"4.434198826s","start":"2025-07-15T10:58:00.273028Z","end":"2025-07-15T10:58:04.707227Z","steps":["trace[971331662] 'range keys from in-memory index tree' (duration: 4.433293306s)"],"step_count":1} Jul 15 10:58:04.711118 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.707287Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:00.273009Z","time spent":"4.434260602s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 10:58:04.711723 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.707169Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.052909284s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/\" range_end:\"/registry/serviceaccounts0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:58:04.711723 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.707587Z","caller":"traceutil/trace.go:171","msg":"trace[1923428811] range","detail":"{range_begin:/registry/serviceaccounts/; range_end:/registry/serviceaccounts0; response_count:0; response_revision:1130; }","duration":"1.053432043s","start":"2025-07-15T10:58:03.654134Z","end":"2025-07-15T10:58:04.707566Z","steps":["trace[1923428811] 'count revisions from in-memory index tree' (duration: 1.052127374s)"],"step_count":1} Jul 15 10:58:04.711723 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.708251Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.434837341s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-controller-manager\" ","response":"","error":"rangeKeys: context cancelled: context canceled"} Jul 15 10:58:04.711723 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.708315Z","caller":"traceutil/trace.go:171","msg":"trace[2019720659] range","detail":"{range_begin:/registry/leases/kube-system/kube-controller-manager; range_end:; }","duration":"4.434930936s","start":"2025-07-15T10:58:00.273370Z","end":"2025-07-15T10:58:04.708301Z","steps":["trace[2019720659] 'range keys from in-memory index tree' (duration: 4.434436834s)"],"step_count":1} Jul 15 10:58:04.711723 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.708364Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:00.273364Z","time spent":"4.434984587s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":54,"response count":0,"response size":0,"request content":"key:\"/registry/leases/kube-system/kube-controller-manager\" "} Jul 15 10:58:04.712210 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.708356Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:03.654087Z","time spent":"1.054167016s","remote":"172.24.4.149:33226","response type":"/etcdserverpb.KV/Range","request count":0,"request size":58,"response count":51,"response size":31,"request content":"key:\"/registry/serviceaccounts/\" range_end:\"/registry/serviceaccounts0\" count_only:true "} Jul 15 10:58:04.712210 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.708487Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.435164071s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/ipreservations\" limit:1 ","response":"range_response_count:0 size:5"} Jul 15 10:58:04.712210 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.708526Z","caller":"traceutil/trace.go:171","msg":"trace[165015768] range","detail":"{range_begin:/registry/crd.projectcalico.org/ipreservations; range_end:; response_count:0; response_revision:1130; }","duration":"4.435218143s","start":"2025-07-15T10:58:00.273295Z","end":"2025-07-15T10:58:04.708513Z","steps":["trace[165015768] 'range keys from in-memory index tree' (duration: 4.434654321s)"],"step_count":1} Jul 15 10:58:04.712210 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.708567Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:00.273286Z","time spent":"4.435266494s","remote":"172.24.4.149:39888","response type":"/etcdserverpb.KV/Range","request count":0,"request size":50,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/ipreservations\" limit:1 "} Jul 15 10:58:04.712210 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.708699Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.435439666s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/configmaps/kube-system/extension-apiserver-authentication\" ","response":"range_response_count:1 size:3017"} Jul 15 10:58:04.716768 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.708752Z","caller":"traceutil/trace.go:171","msg":"trace[824925660] range","detail":"{range_begin:/registry/configmaps/kube-system/extension-apiserver-authentication; range_end:; response_count:1; response_revision:1130; }","duration":"4.435503625s","start":"2025-07-15T10:58:00.273235Z","end":"2025-07-15T10:58:04.708738Z","steps":["trace[824925660] 'range keys from in-memory index tree' (duration: 4.434851229s)"],"step_count":1} Jul 15 10:58:04.716768 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.708792Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:00.273229Z","time spent":"4.435550483s","remote":"172.24.4.149:33128","response type":"/etcdserverpb.KV/Range","request count":0,"request size":69,"response count":1,"response size":3041,"request content":"key:\"/registry/configmaps/kube-system/extension-apiserver-authentication\" "} Jul 15 10:58:04.716768 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.709325Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.05314113s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:04.716768 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.709208Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.43548378s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/tiers/\" range_end:\"/registry/crd.projectcalico.org/tiers0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:58:04.716768 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.709779Z","caller":"traceutil/trace.go:171","msg":"trace[1738284862] range","detail":"{range_begin:/registry/crd.projectcalico.org/tiers/; range_end:/registry/crd.projectcalico.org/tiers0; response_count:0; response_revision:1130; }","duration":"4.436030222s","start":"2025-07-15T10:58:00.273683Z","end":"2025-07-15T10:58:04.709713Z","steps":["trace[1738284862] 'count revisions from in-memory index tree' (duration: 4.435313674s)"],"step_count":1} Jul 15 10:58:04.717356 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.709951Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:00.273675Z","time spent":"4.436235586s","remote":"172.24.4.149:54498","response type":"/etcdserverpb.KV/Range","request count":0,"request size":82,"response count":3,"response size":31,"request content":"key:\"/registry/crd.projectcalico.org/tiers/\" range_end:\"/registry/crd.projectcalico.org/tiers0\" count_only:true "} Jul 15 10:58:04.717356 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.706733Z","caller":"traceutil/trace.go:171","msg":"trace[1252110839] range","detail":"{range_begin:/registry/persistentvolumes/; range_end:/registry/persistentvolumes0; response_count:0; response_revision:1130; }","duration":"1.052132885s","start":"2025-07-15T10:58:03.654571Z","end":"2025-07-15T10:58:04.706704Z","steps":["trace[1252110839] 'count revisions from in-memory index tree' (duration: 1.051111777s)"],"step_count":1} Jul 15 10:58:04.717356 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.710096Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:03.654555Z","time spent":"1.055507071s","remote":"172.24.4.149:33186","response type":"/etcdserverpb.KV/Range","request count":0,"request size":62,"response count":0,"response size":29,"request content":"key:\"/registry/persistentvolumes/\" range_end:\"/registry/persistentvolumes0\" count_only:true "} Jul 15 10:58:04.717356 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.710967Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.054228904s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/\" range_end:\"/registry/secrets0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:58:04.717837 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.711099Z","caller":"traceutil/trace.go:171","msg":"trace[1347908428] range","detail":"{range_begin:/registry/secrets/; range_end:/registry/secrets0; response_count:0; response_revision:1130; }","duration":"1.05453201s","start":"2025-07-15T10:58:03.656548Z","end":"2025-07-15T10:58:04.711080Z","steps":["trace[1347908428] 'count revisions from in-memory index tree' (duration: 1.054092188s)"],"step_count":1} Jul 15 10:58:04.717837 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.711155Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:03.656532Z","time spent":"1.05460137s","remote":"172.24.4.149:33122","response type":"/etcdserverpb.KV/Range","request count":0,"request size":42,"response count":15,"response size":31,"request content":"key:\"/registry/secrets/\" range_end:\"/registry/secrets0\" count_only:true "} Jul 15 10:58:04.717837 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.711631Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.056240361s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/globalnetworksets/\" range_end:\"/registry/crd.projectcalico.org/globalnetworksets0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:58:04.717837 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.711744Z","caller":"traceutil/trace.go:171","msg":"trace[2035528526] range","detail":"{range_begin:/registry/crd.projectcalico.org/globalnetworksets/; range_end:/registry/crd.projectcalico.org/globalnetworksets0; response_count:0; response_revision:1130; }","duration":"1.056482695s","start":"2025-07-15T10:58:03.655193Z","end":"2025-07-15T10:58:04.711676Z","steps":["trace[2035528526] 'count revisions from in-memory index tree' (duration: 1.055606297s)"],"step_count":1} Jul 15 10:58:04.718235 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.711805Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:03.655176Z","time spent":"1.056607959s","remote":"172.24.4.149:54536","response type":"/etcdserverpb.KV/Range","request count":0,"request size":106,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/globalnetworksets/\" range_end:\"/registry/crd.projectcalico.org/globalnetworksets0\" count_only:true "} Jul 15 10:58:04.718235 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.712011Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.056239921s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/endpoints/\" range_end:\"/registry/services/endpoints0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:58:04.718235 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.712055Z","caller":"traceutil/trace.go:171","msg":"trace[1167508233] range","detail":"{range_begin:/registry/services/endpoints/; range_end:/registry/services/endpoints0; response_count:0; response_revision:1130; }","duration":"1.05635097s","start":"2025-07-15T10:58:03.655691Z","end":"2025-07-15T10:58:04.712042Z","steps":["trace[1167508233] 'count revisions from in-memory index tree' (duration: 1.05610002s)"],"step_count":1} Jul 15 10:58:04.718235 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.712117Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:03.655682Z","time spent":"1.05639888s","remote":"172.24.4.149:33194","response type":"/etcdserverpb.KV/Range","request count":0,"request size":64,"response count":6,"response size":31,"request content":"key:\"/registry/services/endpoints/\" range_end:\"/registry/services/endpoints0\" count_only:true "} Jul 15 10:58:04.718774 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.709579Z","caller":"traceutil/trace.go:171","msg":"trace[233814157] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1130; }","duration":"1.054025702s","start":"2025-07-15T10:58:03.655532Z","end":"2025-07-15T10:58:04.709558Z","steps":["trace[233814157] 'range keys from in-memory index tree' (duration: 1.052720875s)"],"step_count":1} Jul 15 10:58:04.718774 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.712679Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:03.655519Z","time spent":"1.057086063s","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:58:04.718774 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.712976Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.049309608s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/globalnetworkpolicies/\" range_end:\"/registry/crd.projectcalico.org/globalnetworkpolicies0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:58:04.718774 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.713034Z","caller":"traceutil/trace.go:171","msg":"trace[914652525] range","detail":"{range_begin:/registry/crd.projectcalico.org/globalnetworkpolicies/; range_end:/registry/crd.projectcalico.org/globalnetworkpolicies0; response_count:0; response_revision:1130; }","duration":"1.049452836s","start":"2025-07-15T10:58:03.663562Z","end":"2025-07-15T10:58:04.713015Z","steps":["trace[914652525] 'count revisions from in-memory index tree' (duration: 1.04914016s)"],"step_count":1} Jul 15 10:58:04.719171 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.713084Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:03.663546Z","time spent":"1.049519169s","remote":"172.24.4.149:54512","response type":"/etcdserverpb.KV/Range","request count":0,"request size":114,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/globalnetworkpolicies/\" range_end:\"/registry/crd.projectcalico.org/globalnetworkpolicies0\" count_only:true "} Jul 15 10:58:04.719171 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.708172Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.434683284s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:04.719171 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.713359Z","caller":"traceutil/trace.go:171","msg":"trace[595818561] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1130; }","duration":"4.439885486s","start":"2025-07-15T10:58:00.273452Z","end":"2025-07-15T10:58:04.713337Z","steps":["trace[595818561] 'range keys from in-memory index tree' (duration: 4.43435448s)"],"step_count":1} Jul 15 10:58:04.719171 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.715220Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:00.273446Z","time spent":"4.441572169s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:58:04.719515 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.715303Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"157.014339ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 10:58:04.719515 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.716651Z","caller":"traceutil/trace.go:171","msg":"trace[810447630] transaction","detail":"{read_only:false; response_revision:1131; number_of_response:1; }","duration":"1.042805713s","start":"2025-07-15T10:58:03.673659Z","end":"2025-07-15T10:58:04.716464Z","steps":["trace[810447630] 'process raft request' (duration: 884.250594ms)","trace[810447630] 'compare' (duration: 147.89717ms)"],"step_count":2} Jul 15 10:58:04.719515 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.717680Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:03.673633Z","time spent":"1.04361716s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":625,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:04.720322 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.719369Z","caller":"traceutil/trace.go:171","msg":"trace[425650220] linearizableReadLoop","detail":"{readStateIndex:1169; appliedIndex:1166; }","duration":"1.013008027s","start":"2025-07-15T10:58:03.706339Z","end":"2025-07-15T10:58:04.719347Z","steps":["trace[425650220] 'read index received' (duration: 851.585439ms)","trace[425650220] 'applied index is now lower than readState.Index' (duration: 161.421405ms)"],"step_count":2} Jul 15 10:58:04.720322 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.719912Z","caller":"traceutil/trace.go:171","msg":"trace[1164910989] transaction","detail":"{read_only:false; response_revision:1132; number_of_response:1; }","duration":"1.033635798s","start":"2025-07-15T10:58:03.686256Z","end":"2025-07-15T10:58:04.719892Z","steps":["trace[1164910989] 'process raft request' (duration: 1.03008951s)"],"step_count":1} Jul 15 10:58:04.720965 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.720369Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:03.686229Z","time spent":"1.034063686s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":699,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:04.721834 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.721452Z","caller":"traceutil/trace.go:171","msg":"trace[855890639] transaction","detail":"{read_only:false; response_revision:1133; number_of_response:1; }","duration":"1.030917841s","start":"2025-07-15T10:58:03.690455Z","end":"2025-07-15T10:58:04.721373Z","steps":["trace[855890639] 'process raft request' (duration: 1.027918336s)"],"step_count":1} Jul 15 10:58:04.723555 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.722574Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:03.690432Z","time spent":"1.031800531s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":5389,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:04.723555 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.722859Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.016505523s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"range_response_count:1 size:886"} Jul 15 10:58:04.723555 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.723109Z","caller":"traceutil/trace.go:171","msg":"trace[1678839337] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; response_count:1; response_revision:1133; }","duration":"1.016782629s","start":"2025-07-15T10:58:03.706280Z","end":"2025-07-15T10:58:04.723063Z","steps":["trace[1678839337] 'agreement among raft nodes before linearized reading' (duration: 1.016309266s)"],"step_count":1} Jul 15 10:58:04.724130 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.723608Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:03.706259Z","time spent":"1.017001449s","remote":"172.24.4.149:54994","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":1,"response size":910,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" "} Jul 15 10:58:04.957061 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.952617Z","caller":"traceutil/trace.go:171","msg":"trace[2041755872] transaction","detail":"{read_only:false; number_of_response:0; response_revision:1133; }","duration":"376.453948ms","start":"2025-07-15T10:58:04.576101Z","end":"2025-07-15T10:58:04.952555Z","steps":["trace[2041755872] 'process raft request' (duration: 302.719362ms)","trace[2041755872] 'compare' (duration: 73.419306ms)"],"step_count":2} Jul 15 10:58:04.957061 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.952893Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:04.576042Z","time spent":"376.738309ms","remote":"172.24.4.149:33078","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":29,"request content":"compare: success:> failure:<>"} Jul 15 10:58:04.957061 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.953341Z","caller":"traceutil/trace.go:171","msg":"trace[1004328083] linearizableReadLoop","detail":"{readStateIndex:1171; appliedIndex:1169; }","duration":"233.472434ms","start":"2025-07-15T10:58:04.719828Z","end":"2025-07-15T10:58:04.953301Z","steps":["trace[1004328083] 'read index received' (duration: 159.054521ms)","trace[1004328083] 'applied index is now lower than readState.Index' (duration: 74.41622ms)"],"step_count":2} Jul 15 10:58:04.957061 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.953635Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"402.977779ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/persistentvolumeclaims/\" range_end:\"/registry/persistentvolumeclaims0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:58:04.957874 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.953695Z","caller":"traceutil/trace.go:171","msg":"trace[1686231868] range","detail":"{range_begin:/registry/persistentvolumeclaims/; range_end:/registry/persistentvolumeclaims0; response_count:0; response_revision:1133; }","duration":"403.300812ms","start":"2025-07-15T10:58:04.550371Z","end":"2025-07-15T10:58:04.953672Z","steps":["trace[1686231868] 'agreement among raft nodes before linearized reading' (duration: 403.185947ms)"],"step_count":1} Jul 15 10:58:04.957874 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.953768Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:04.550332Z","time spent":"403.40102ms","remote":"172.24.4.149:33190","response type":"/etcdserverpb.KV/Range","request count":0,"request size":72,"response count":0,"response size":29,"request content":"key:\"/registry/persistentvolumeclaims/\" range_end:\"/registry/persistentvolumeclaims0\" count_only:true "} Jul 15 10:58:04.957874 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.953769Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"230.649869ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76\" ","response":"range_response_count:1 size:905"} Jul 15 10:58:04.957874 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.953839Z","caller":"traceutil/trace.go:171","msg":"trace[268053069] range","detail":"{range_begin:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76; range_end:; response_count:1; response_revision:1133; }","duration":"230.775453ms","start":"2025-07-15T10:58:04.723042Z","end":"2025-07-15T10:58:04.953817Z","steps":["trace[268053069] 'agreement among raft nodes before linearized reading' (duration: 230.521269ms)"],"step_count":1} Jul 15 10:58:04.960163 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.954029Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"167.088037ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/configmaps/kube-system/extension-apiserver-authentication\" ","response":"range_response_count:1 size:3017"} Jul 15 10:58:04.960163 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.954076Z","caller":"traceutil/trace.go:171","msg":"trace[1193208579] range","detail":"{range_begin:/registry/configmaps/kube-system/extension-apiserver-authentication; range_end:; response_count:1; response_revision:1133; }","duration":"167.168537ms","start":"2025-07-15T10:58:04.786893Z","end":"2025-07-15T10:58:04.954061Z","steps":["trace[1193208579] 'agreement among raft nodes before linearized reading' (duration: 167.067378ms)"],"step_count":1} Jul 15 10:58:04.960163 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.954311Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"216.600252ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:04.960163 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.954671Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"215.959985ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/ci-4230-2-100-6ef5831daf.novalocal\" ","response":"range_response_count:1 size:5403"} Jul 15 10:58:04.960163 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.954774Z","caller":"traceutil/trace.go:171","msg":"trace[1643141825] range","detail":"{range_begin:/registry/minions/ci-4230-2-100-6ef5831daf.novalocal; range_end:; response_count:1; response_revision:1133; }","duration":"216.228987ms","start":"2025-07-15T10:58:04.738503Z","end":"2025-07-15T10:58:04.954732Z","steps":["trace[1643141825] 'agreement among raft nodes before linearized reading' (duration: 215.637933ms)"],"step_count":1} Jul 15 10:58:04.960763 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:04.955455Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"201.199789ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/felixconfigurations/default\" ","response":"range_response_count:1 size:1580"} Jul 15 10:58:04.960763 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.955549Z","caller":"traceutil/trace.go:171","msg":"trace[1196103310] range","detail":"{range_begin:/registry/crd.projectcalico.org/felixconfigurations/default; range_end:; response_count:1; response_revision:1133; }","duration":"201.421062ms","start":"2025-07-15T10:58:04.754108Z","end":"2025-07-15T10:58:04.955529Z","steps":["trace[1196103310] 'agreement among raft nodes before linearized reading' (duration: 201.192556ms)"],"step_count":1} Jul 15 10:58:04.960763 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:04.955883Z","caller":"traceutil/trace.go:171","msg":"trace[1944119884] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1133; }","duration":"216.678628ms","start":"2025-07-15T10:58:04.737695Z","end":"2025-07-15T10:58:04.954374Z","steps":["trace[1944119884] 'agreement among raft nodes before linearized reading' (duration: 216.560929ms)"],"step_count":1} Jul 15 10:58:05.093893 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:05.092632Z","caller":"traceutil/trace.go:171","msg":"trace[1089453130] linearizableReadLoop","detail":"{readStateIndex:1174; appliedIndex:1172; }","duration":"101.906891ms","start":"2025-07-15T10:58:04.990651Z","end":"2025-07-15T10:58:05.092558Z","steps":["trace[1089453130] 'read index received' (duration: 77.543963ms)","trace[1089453130] 'applied index is now lower than readState.Index' (duration: 24.360884ms)"],"step_count":2} Jul 15 10:58:05.095747 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:05.092814Z","caller":"traceutil/trace.go:171","msg":"trace[1913879061] transaction","detail":"{read_only:false; number_of_response:0; response_revision:1135; }","duration":"114.66274ms","start":"2025-07-15T10:58:04.978136Z","end":"2025-07-15T10:58:05.092799Z","steps":["trace[1913879061] 'process raft request' (duration: 114.128031ms)"],"step_count":1} Jul 15 10:58:05.095747 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:05.093238Z","caller":"traceutil/trace.go:171","msg":"trace[404273083] transaction","detail":"{read_only:false; response_revision:1135; number_of_response:1; }","duration":"131.105731ms","start":"2025-07-15T10:58:04.961578Z","end":"2025-07-15T10:58:05.092683Z","steps":["trace[404273083] 'process raft request' (duration: 106.769674ms)","trace[404273083] 'compare' (duration: 23.506105ms)"],"step_count":2} Jul 15 10:58:05.095747 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:05.094597Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"101.822444ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/felixconfigurations/node.ci-4230-2-100-6ef5831daf.novalocal\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:05.095747 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:05.094706Z","caller":"traceutil/trace.go:171","msg":"trace[1138029856] range","detail":"{range_begin:/registry/crd.projectcalico.org/felixconfigurations/node.ci-4230-2-100-6ef5831daf.novalocal; range_end:; response_count:0; response_revision:1135; }","duration":"101.992011ms","start":"2025-07-15T10:58:04.992693Z","end":"2025-07-15T10:58:05.094685Z","steps":["trace[1138029856] 'agreement among raft nodes before linearized reading' (duration: 101.476327ms)"],"step_count":1} Jul 15 10:58:05.095747 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:05.094860Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"101.114472ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/endpoints/default/kubernetes\" ","response":"range_response_count:1 size:420"} Jul 15 10:58:05.096595 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:05.094911Z","caller":"traceutil/trace.go:171","msg":"trace[909540881] range","detail":"{range_begin:/registry/services/endpoints/default/kubernetes; range_end:; response_count:1; response_revision:1135; }","duration":"101.211603ms","start":"2025-07-15T10:58:04.993683Z","end":"2025-07-15T10:58:05.094895Z","steps":["trace[909540881] 'agreement among raft nodes before linearized reading' (duration: 101.115673ms)"],"step_count":1} Jul 15 10:58:05.096595 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:05.094895Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"129.712159ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/ippools\" limit:1 ","response":"range_response_count:0 size:5"} Jul 15 10:58:05.096595 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:05.095065Z","caller":"traceutil/trace.go:171","msg":"trace[2116060324] range","detail":"{range_begin:/registry/crd.projectcalico.org/ippools; range_end:; response_count:0; response_revision:1135; }","duration":"130.033248ms","start":"2025-07-15T10:58:04.964961Z","end":"2025-07-15T10:58:05.094994Z","steps":["trace[2116060324] 'agreement among raft nodes before linearized reading' (duration: 128.484843ms)"],"step_count":1} Jul 15 10:58:20.865496 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:20.863960Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073298,"retry-timeout":"500ms"} Jul 15 10:58:21.366052 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:21.365276Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073298,"retry-timeout":"500ms"} Jul 15 10:58:23.616890 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:21.869075Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073298,"retry-timeout":"500ms"} Jul 15 10:58:23.616890 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:22.364747Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000744996s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 10:58:23.616890 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:22.364879Z","caller":"traceutil/trace.go:171","msg":"trace[995108407] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.000982333s","start":"2025-07-15T10:58:20.363859Z","end":"2025-07-15T10:58:22.364841Z","steps":["trace[995108407] 'agreement among raft nodes before linearized reading' (duration: 2.000760394s)"],"step_count":1} Jul 15 10:58:23.616890 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:22.365022Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:20.363848Z","time spent":"2.001152184s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:58:23.616890 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:22.369532Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073298,"retry-timeout":"500ms"} Jul 15 10:58:23.616890 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:22.870554Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073298,"retry-timeout":"500ms"} Jul 15 10:58:23.616890 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:23.387532Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073298,"retry-timeout":"500ms"} Jul 15 10:58:23.721607 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:23.721003Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"3.38830358s","expected-duration":"1s"} Jul 15 10:58:24.924156 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:24.366358Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.999350835s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 10:58:24.924156 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:24.366791Z","caller":"traceutil/trace.go:171","msg":"trace[1423260078] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"1.999819099s","start":"2025-07-15T10:58:22.366937Z","end":"2025-07-15T10:58:24.366756Z","steps":["trace[1423260078] 'agreement among raft nodes before linearized reading' (duration: 1.999371594s)"],"step_count":1} Jul 15 10:58:24.924156 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:24.366914Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:22.366920Z","time spent":"1.999937101s","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:58:25.332053 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.329906Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"1.608648921s","expected-duration":"1s"} Jul 15 10:58:25.342108 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.339921Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.61836188s","expected-duration":"100ms","prefix":"","request":"header: lease_revoke:","response":"size:29"} Jul 15 10:58:25.342108 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:25.341211Z","caller":"traceutil/trace.go:171","msg":"trace[1818870629] transaction","detail":"{read_only:false; response_revision:1172; number_of_response:1; }","duration":"379.943271ms","start":"2025-07-15T10:58:24.961236Z","end":"2025-07-15T10:58:25.341179Z","steps":["trace[1818870629] 'process raft request' (duration: 379.79377ms)"],"step_count":1} Jul 15 10:58:25.343319 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.342902Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:24.961203Z","time spent":"381.561737ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":699,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:25.343319 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.343012Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:20.359198Z","time spent":"4.983810529s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.Lease/LeaseGrant","request count":-1,"request size":-1,"response count":-1,"response size":-1,"request content":""} Jul 15 10:58:25.343743 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:25.343081Z","caller":"traceutil/trace.go:171","msg":"trace[746603390] linearizableReadLoop","detail":"{readStateIndex:1310; appliedIndex:1308; }","duration":"4.979170882s","start":"2025-07-15T10:58:20.363883Z","end":"2025-07-15T10:58:25.343054Z","steps":["trace[746603390] 'read index received' (duration: 3.360686306s)","trace[746603390] 'applied index is now lower than readState.Index' (duration: 1.618483474s)"],"step_count":2} Jul 15 10:58:25.482819 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.479473Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"106.864598ms","expected-duration":"100ms","prefix":"","request":"header: lease_revoke:","response":"size:29"} Jul 15 10:58:25.483668 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.479986Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.448178162s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/ipamblocks/\" range_end:\"/registry/crd.projectcalico.org/ipamblocks0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:58:25.483668 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:25.479756Z","caller":"traceutil/trace.go:171","msg":"trace[583186723] linearizableReadLoop","detail":"{readStateIndex:1312; appliedIndex:1311; }","duration":"136.60933ms","start":"2025-07-15T10:58:25.343113Z","end":"2025-07-15T10:58:25.479722Z","steps":["trace[583186723] 'read index received' (duration: 29.439215ms)","trace[583186723] 'applied index is now lower than readState.Index' (duration: 107.16741ms)"],"step_count":2} Jul 15 10:58:25.483668 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:25.480075Z","caller":"traceutil/trace.go:171","msg":"trace[577499857] range","detail":"{range_begin:/registry/crd.projectcalico.org/ipamblocks/; range_end:/registry/crd.projectcalico.org/ipamblocks0; response_count:0; response_revision:1173; }","duration":"2.448338784s","start":"2025-07-15T10:58:23.031706Z","end":"2025-07-15T10:58:25.480045Z","steps":["trace[577499857] 'agreement among raft nodes before linearized reading' (duration: 2.448157542s)"],"step_count":1} Jul 15 10:58:25.483668 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.480145Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.020205164s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/storageclasses/\" range_end:\"/registry/storageclasses0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:58:25.484263 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:25.480197Z","caller":"traceutil/trace.go:171","msg":"trace[1796823854] range","detail":"{range_begin:/registry/storageclasses/; range_end:/registry/storageclasses0; response_count:0; response_revision:1173; }","duration":"5.020286908s","start":"2025-07-15T10:58:20.459894Z","end":"2025-07-15T10:58:25.480181Z","steps":["trace[1796823854] 'agreement among raft nodes before linearized reading' (duration: 5.020208339s)"],"step_count":1} Jul 15 10:58:25.484263 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.480261Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:20.459876Z","time spent":"5.020357611s","remote":"172.24.4.149:33390","response type":"/etcdserverpb.KV/Range","request count":0,"request size":56,"response count":0,"response size":29,"request content":"key:\"/registry/storageclasses/\" range_end:\"/registry/storageclasses0\" count_only:true "} Jul 15 10:58:25.484263 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.480307Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"546.278448ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/tigera-operator/operator-lock\" ","response":"range_response_count:1 size:505"} Jul 15 10:58:25.484263 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:25.480359Z","caller":"traceutil/trace.go:171","msg":"trace[1003703902] range","detail":"{range_begin:/registry/leases/tigera-operator/operator-lock; range_end:; response_count:1; response_revision:1173; }","duration":"546.362077ms","start":"2025-07-15T10:58:24.933980Z","end":"2025-07-15T10:58:25.480342Z","steps":["trace[1003703902] 'agreement among raft nodes before linearized reading' (duration: 546.198799ms)"],"step_count":1} Jul 15 10:58:25.484748 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.480478Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:24.933952Z","time spent":"546.503444ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":1,"response size":529,"request content":"key:\"/registry/leases/tigera-operator/operator-lock\" "} Jul 15 10:58:25.484748 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.480646Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"925.876953ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/policy.networking.k8s.io/adminnetworkpolicies/\" range_end:\"/registry/policy.networking.k8s.io/adminnetworkpolicies0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:58:25.484748 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:25.480741Z","caller":"traceutil/trace.go:171","msg":"trace[490909111] range","detail":"{range_begin:/registry/policy.networking.k8s.io/adminnetworkpolicies/; range_end:/registry/policy.networking.k8s.io/adminnetworkpolicies0; response_count:0; response_revision:1173; }","duration":"926.019441ms","start":"2025-07-15T10:58:24.554701Z","end":"2025-07-15T10:58:25.480721Z","steps":["trace[490909111] 'agreement among raft nodes before linearized reading' (duration: 925.898093ms)"],"step_count":1} Jul 15 10:58:25.484748 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.480819Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.277805263s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-controller-manager\" ","response":"range_response_count:1 size:531"} Jul 15 10:58:25.485327 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:25.480876Z","caller":"traceutil/trace.go:171","msg":"trace[194997549] range","detail":"{range_begin:/registry/leases/kube-system/kube-controller-manager; range_end:; response_count:1; response_revision:1173; }","duration":"4.277913658s","start":"2025-07-15T10:58:21.202946Z","end":"2025-07-15T10:58:25.480860Z","steps":["trace[194997549] 'agreement among raft nodes before linearized reading' (duration: 4.277742926s)"],"step_count":1} Jul 15 10:58:25.485327 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.480891Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.66162583s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/kubecontrollersconfigurations/\" range_end:\"/registry/crd.projectcalico.org/kubecontrollersconfigurations0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:58:25.485327 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:25.480957Z","caller":"traceutil/trace.go:171","msg":"trace[1869791500] range","detail":"{range_begin:/registry/crd.projectcalico.org/kubecontrollersconfigurations/; range_end:/registry/crd.projectcalico.org/kubecontrollersconfigurations0; response_count:0; response_revision:1173; }","duration":"1.661733291s","start":"2025-07-15T10:58:23.819208Z","end":"2025-07-15T10:58:25.480941Z","steps":["trace[1869791500] 'agreement among raft nodes before linearized reading' (duration: 1.661648972s)"],"step_count":1} Jul 15 10:58:25.485327 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.480932Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:21.202926Z","time spent":"4.277990302s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":54,"response count":1,"response size":555,"request content":"key:\"/registry/leases/kube-system/kube-controller-manager\" "} Jul 15 10:58:25.485819 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.481018Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:23.819183Z","time spent":"1.661811519s","remote":"172.24.4.149:54920","response type":"/etcdserverpb.KV/Range","request count":0,"request size":130,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/kubecontrollersconfigurations/\" range_end:\"/registry/crd.projectcalico.org/kubecontrollersconfigurations0\" count_only:true "} Jul 15 10:58:25.485819 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.480661Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.834600326s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/replicasets/\" range_end:\"/registry/replicasets0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:58:25.485819 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:25.481163Z","caller":"traceutil/trace.go:171","msg":"trace[1321832989] range","detail":"{range_begin:/registry/replicasets/; range_end:/registry/replicasets0; response_count:0; response_revision:1173; }","duration":"3.835128132s","start":"2025-07-15T10:58:21.646016Z","end":"2025-07-15T10:58:25.481144Z","steps":["trace[1321832989] 'agreement among raft nodes before linearized reading' (duration: 3.834585068s)"],"step_count":1} Jul 15 10:58:25.485819 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.481243Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:21.645996Z","time spent":"3.835218252s","remote":"172.24.4.149:33492","response type":"/etcdserverpb.KV/Range","request count":0,"request size":50,"response count":8,"response size":31,"request content":"key:\"/registry/replicasets/\" range_end:\"/registry/replicasets0\" count_only:true "} Jul 15 10:58:25.486587 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.480146Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:23.031687Z","time spent":"2.448430617s","remote":"172.24.4.149:54504","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":1,"response size":31,"request content":"key:\"/registry/crd.projectcalico.org/ipamblocks/\" range_end:\"/registry/crd.projectcalico.org/ipamblocks0\" count_only:true "} Jul 15 10:58:25.486587 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.480825Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:24.554672Z","time spent":"926.131052ms","remote":"172.24.4.149:54616","response type":"/etcdserverpb.KV/Range","request count":0,"request size":118,"response count":0,"response size":29,"request content":"key:\"/registry/policy.networking.k8s.io/adminnetworkpolicies/\" range_end:\"/registry/policy.networking.k8s.io/adminnetworkpolicies0\" count_only:true "} Jul 15 10:58:25.486587 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.481543Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.282951982s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-scheduler\" ","response":"range_response_count:1 size:504"} Jul 15 10:58:25.486587 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:25.481612Z","caller":"traceutil/trace.go:171","msg":"trace[55790825] range","detail":"{range_begin:/registry/leases/kube-system/kube-scheduler; range_end:; response_count:1; response_revision:1173; }","duration":"4.28305115s","start":"2025-07-15T10:58:21.198541Z","end":"2025-07-15T10:58:25.481592Z","steps":["trace[55790825] 'agreement among raft nodes before linearized reading' (duration: 4.282886279s)"],"step_count":1} Jul 15 10:58:25.487074 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:25.481678Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:21.198523Z","time spent":"4.283133135s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":45,"response count":1,"response size":528,"request content":"key:\"/registry/leases/kube-system/kube-scheduler\" "} Jul 15 10:58:27.607623 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:27.605020Z","caller":"traceutil/trace.go:171","msg":"trace[884830327] linearizableReadLoop","detail":"{readStateIndex:1323; appliedIndex:1321; }","duration":"547.276338ms","start":"2025-07-15T10:58:27.057708Z","end":"2025-07-15T10:58:27.604984Z","steps":["trace[884830327] 'read index received' (duration: 22.549778ms)","trace[884830327] 'applied index is now lower than readState.Index' (duration: 524.725518ms)"],"step_count":2} Jul 15 10:58:27.607623 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:27.605277Z","caller":"traceutil/trace.go:171","msg":"trace[938081035] transaction","detail":"{read_only:false; response_revision:1182; number_of_response:1; }","duration":"546.043701ms","start":"2025-07-15T10:58:27.059217Z","end":"2025-07-15T10:58:27.605260Z","steps":["trace[938081035] 'process raft request' (duration: 545.654736ms)"],"step_count":1} Jul 15 10:58:27.607623 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:27.605479Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:27.059193Z","time spent":"546.124213ms","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":5841,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:27.607623 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:27.605667Z","caller":"traceutil/trace.go:171","msg":"trace[1141903029] transaction","detail":"{read_only:false; response_revision:1180; number_of_response:1; }","duration":"552.603329ms","start":"2025-07-15T10:58:27.053019Z","end":"2025-07-15T10:58:27.605622Z","steps":["trace[1141903029] 'process raft request' (duration: 544.341976ms)"],"step_count":1} Jul 15 10:58:27.609036 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:27.605999Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"548.279172ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76\" ","response":"range_response_count:1 size:905"} Jul 15 10:58:27.609036 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:27.606007Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:27.052990Z","time spent":"552.828994ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:27.609036 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:27.606072Z","caller":"traceutil/trace.go:171","msg":"trace[1594714469] range","detail":"{range_begin:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76; range_end:; response_count:1; response_revision:1182; }","duration":"548.38408ms","start":"2025-07-15T10:58:27.057663Z","end":"2025-07-15T10:58:27.606047Z","steps":["trace[1594714469] 'agreement among raft nodes before linearized reading' (duration: 548.120483ms)"],"step_count":1} Jul 15 10:58:27.609036 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:27.606146Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:27.057635Z","time spent":"548.482156ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":97,"response count":1,"response size":929,"request content":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76\" "} Jul 15 10:58:27.609796 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:27.606190Z","caller":"traceutil/trace.go:171","msg":"trace[1174057827] transaction","detail":"{read_only:false; response_revision:1181; number_of_response:1; }","duration":"548.477226ms","start":"2025-07-15T10:58:27.057689Z","end":"2025-07-15T10:58:27.606166Z","steps":["trace[1174057827] 'process raft request' (duration: 546.636089ms)"],"step_count":1} Jul 15 10:58:27.609796 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:27.606296Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"539.633623ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:27.609796 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:27.606301Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:27.057661Z","time spent":"548.587454ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":732,"response count":0,"response size":40,"request content":"compare: success:> failure:<>"} Jul 15 10:58:27.609796 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:27.606345Z","caller":"traceutil/trace.go:171","msg":"trace[1479836040] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1182; }","duration":"539.718625ms","start":"2025-07-15T10:58:27.066612Z","end":"2025-07-15T10:58:27.606330Z","steps":["trace[1479836040] 'agreement among raft nodes before linearized reading' (duration: 539.645757ms)"],"step_count":1} Jul 15 10:58:27.610195 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:27.606427Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:27.066591Z","time spent":"539.779911ms","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:58:27.610195 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:27.606640Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"538.178857ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/endpointslices/default/kubernetes\" ","response":"range_response_count:1 size:474"} Jul 15 10:58:27.610195 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:27.606684Z","caller":"traceutil/trace.go:171","msg":"trace[100631223] range","detail":"{range_begin:/registry/endpointslices/default/kubernetes; range_end:; response_count:1; response_revision:1182; }","duration":"538.315664ms","start":"2025-07-15T10:58:27.068354Z","end":"2025-07-15T10:58:27.606669Z","steps":["trace[100631223] 'agreement among raft nodes before linearized reading' (duration: 538.185299ms)"],"step_count":1} Jul 15 10:58:27.610195 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:27.606738Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:27.068330Z","time spent":"538.395325ms","remote":"172.24.4.149:33310","response type":"/etcdserverpb.KV/Range","request count":0,"request size":45,"response count":1,"response size":498,"request content":"key:\"/registry/endpointslices/default/kubernetes\" "} Jul 15 10:58:29.532595 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.531797Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073336,"retry-timeout":"500ms"} Jul 15 10:58:29.540530 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.538938Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"1.611377483s","expected-duration":"1s"} Jul 15 10:58:29.540530 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.539775Z","caller":"traceutil/trace.go:171","msg":"trace[712724956] transaction","detail":"{read_only:false; response_revision:1184; number_of_response:1; }","duration":"1.612405847s","start":"2025-07-15T10:58:27.927331Z","end":"2025-07-15T10:58:29.539736Z","steps":["trace[712724956] 'process raft request' (duration: 1.612124705s)"],"step_count":1} Jul 15 10:58:29.540530 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.540020Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:27.927289Z","time spent":"1.612611264s","remote":"172.24.4.149:54882","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:29.652657 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.645654Z","caller":"traceutil/trace.go:171","msg":"trace[1471442358] transaction","detail":"{read_only:false; response_revision:1186; number_of_response:1; }","duration":"475.951614ms","start":"2025-07-15T10:58:29.169657Z","end":"2025-07-15T10:58:29.645608Z","steps":["trace[1471442358] 'process raft request' (duration: 475.421422ms)"],"step_count":1} Jul 15 10:58:29.652657 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.645903Z","caller":"traceutil/trace.go:171","msg":"trace[2067583184] linearizableReadLoop","detail":"{readStateIndex:1333; appliedIndex:1326; }","duration":"1.707730582s","start":"2025-07-15T10:58:27.938143Z","end":"2025-07-15T10:58:29.645874Z","steps":["trace[2067583184] 'read index received' (duration: 1.602222685s)","trace[2067583184] 'applied index is now lower than readState.Index' (duration: 105.506334ms)"],"step_count":2} Jul 15 10:58:29.652657 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.645983Z","caller":"traceutil/trace.go:171","msg":"trace[1297801879] transaction","detail":"{read_only:false; response_revision:1185; number_of_response:1; }","duration":"1.657780507s","start":"2025-07-15T10:58:27.988184Z","end":"2025-07-15T10:58:29.645965Z","steps":["trace[1297801879] 'process raft request' (duration: 1.656660621s)"],"step_count":1} Jul 15 10:58:29.652657 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.646088Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:27.988159Z","time spent":"1.657865658s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":4031,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:29.653672 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.646533Z","caller":"traceutil/trace.go:171","msg":"trace[850392467] transaction","detail":"{read_only:false; response_revision:1191; number_of_response:1; }","duration":"451.016187ms","start":"2025-07-15T10:58:29.195493Z","end":"2025-07-15T10:58:29.646509Z","steps":["trace[850392467] 'process raft request' (duration: 450.248917ms)"],"step_count":1} Jul 15 10:58:29.653672 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.646690Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:29.195469Z","time spent":"451.123258ms","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1125,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:29.653672 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.646812Z","caller":"traceutil/trace.go:171","msg":"trace[2013346693] transaction","detail":"{read_only:false; response_revision:1187; number_of_response:1; }","duration":"462.134951ms","start":"2025-07-15T10:58:29.184657Z","end":"2025-07-15T10:58:29.646792Z","steps":["trace[2013346693] 'process raft request' (duration: 460.54706ms)"],"step_count":1} Jul 15 10:58:29.654196 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.646940Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:29.184619Z","time spent":"462.253084ms","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1138,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:29.654196 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.647061Z","caller":"traceutil/trace.go:171","msg":"trace[1303763614] transaction","detail":"{read_only:false; response_revision:1188; number_of_response:1; }","duration":"460.338907ms","start":"2025-07-15T10:58:29.186697Z","end":"2025-07-15T10:58:29.647036Z","steps":["trace[1303763614] 'process raft request' (duration: 458.619007ms)"],"step_count":1} Jul 15 10:58:29.654196 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.647195Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:29.186677Z","time spent":"460.448784ms","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1117,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:29.656090 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.647372Z","caller":"traceutil/trace.go:171","msg":"trace[713558865] transaction","detail":"{read_only:false; response_revision:1189; number_of_response:1; }","duration":"456.868858ms","start":"2025-07-15T10:58:29.190427Z","end":"2025-07-15T10:58:29.647296Z","steps":["trace[713558865] 'process raft request' (duration: 455.115926ms)"],"step_count":1} Jul 15 10:58:29.656090 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.647710Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.709554539s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:4046"} Jul 15 10:58:29.656090 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.647792Z","caller":"traceutil/trace.go:171","msg":"trace[636997652] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1191; }","duration":"1.709677972s","start":"2025-07-15T10:58:27.938091Z","end":"2025-07-15T10:58:29.647769Z","steps":["trace[636997652] 'agreement among raft nodes before linearized reading' (duration: 1.709531856s)"],"step_count":1} Jul 15 10:58:29.656090 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.647799Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:29.190359Z","time spent":"457.205765ms","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1122,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:29.656598 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.647860Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:27.938062Z","time spent":"1.70977718s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":4070,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:58:29.656598 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.645921Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:29.169615Z","time spent":"476.186308ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:29.656598 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.648041Z","caller":"traceutil/trace.go:171","msg":"trace[2084515807] transaction","detail":"{read_only:false; response_revision:1190; number_of_response:1; }","duration":"456.902853ms","start":"2025-07-15T10:58:29.191053Z","end":"2025-07-15T10:58:29.647956Z","steps":["trace[2084515807] 'process raft request' (duration: 454.618035ms)"],"step_count":1} Jul 15 10:58:29.656598 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.648203Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.704405455s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/whisker:csr-creator\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:29.656985 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.648264Z","caller":"traceutil/trace.go:171","msg":"trace[148798362] range","detail":"{range_begin:/registry/clusterrolebindings/whisker:csr-creator; range_end:; response_count:0; response_revision:1191; }","duration":"1.704509582s","start":"2025-07-15T10:58:27.943735Z","end":"2025-07-15T10:58:29.648244Z","steps":["trace[148798362] 'agreement among raft nodes before linearized reading' (duration: 1.704422067s)"],"step_count":1} Jul 15 10:58:29.656985 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.648325Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:27.943685Z","time spent":"1.704622585s","remote":"172.24.4.149:33376","response type":"/etcdserverpb.KV/Range","request count":0,"request size":51,"response count":0,"response size":29,"request content":"key:\"/registry/clusterrolebindings/whisker:csr-creator\" "} Jul 15 10:58:29.656985 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.648525Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"457.524057ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/tigerastatuses/management-cluster-connection\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:29.656985 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.648557Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:29.191037Z","time spent":"457.155551ms","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":774,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:29.657347 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.648594Z","caller":"traceutil/trace.go:171","msg":"trace[1129117860] range","detail":"{range_begin:/registry/operator.tigera.io/tigerastatuses/management-cluster-connection; range_end:; response_count:0; response_revision:1191; }","duration":"457.62669ms","start":"2025-07-15T10:58:29.190947Z","end":"2025-07-15T10:58:29.648574Z","steps":["trace[1129117860] 'agreement among raft nodes before linearized reading' (duration: 457.53681ms)"],"step_count":1} Jul 15 10:58:29.657347 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.648668Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:29.190930Z","time spent":"457.706351ms","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Range","request count":0,"request size":75,"response count":0,"response size":29,"request content":"key:\"/registry/operator.tigera.io/tigerastatuses/management-cluster-connection\" "} Jul 15 10:58:29.657347 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.648778Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"471.519848ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/tigerastatuses/gatewayapi\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:29.657347 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.648838Z","caller":"traceutil/trace.go:171","msg":"trace[903061837] range","detail":"{range_begin:/registry/operator.tigera.io/tigerastatuses/gatewayapi; range_end:; response_count:0; response_revision:1191; }","duration":"471.628583ms","start":"2025-07-15T10:58:29.177190Z","end":"2025-07-15T10:58:29.648818Z","steps":["trace[903061837] 'agreement among raft nodes before linearized reading' (duration: 471.543772ms)"],"step_count":1} Jul 15 10:58:29.657749 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.648905Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:29.177174Z","time spent":"471.708425ms","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Range","request count":0,"request size":56,"response count":0,"response size":29,"request content":"key:\"/registry/operator.tigera.io/tigerastatuses/gatewayapi\" "} Jul 15 10:58:29.657749 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.649008Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"479.816358ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:29.657749 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.649070Z","caller":"traceutil/trace.go:171","msg":"trace[983626747] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1191; }","duration":"479.910356ms","start":"2025-07-15T10:58:29.169141Z","end":"2025-07-15T10:58:29.649051Z","steps":["trace[983626747] 'agreement among raft nodes before linearized reading' (duration: 479.831537ms)"],"step_count":1} Jul 15 10:58:29.657749 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.649142Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:29.169122Z","time spent":"479.997671ms","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:58:29.657749 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.649268Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"487.158155ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:29.658130 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.649325Z","caller":"traceutil/trace.go:171","msg":"trace[777609492] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1191; }","duration":"487.257373ms","start":"2025-07-15T10:58:29.162048Z","end":"2025-07-15T10:58:29.649306Z","steps":["trace[777609492] 'agreement among raft nodes before linearized reading' (duration: 487.182572ms)"],"step_count":1} Jul 15 10:58:29.658130 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.649715Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.68277122s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:4046"} Jul 15 10:58:29.658130 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.649906Z","caller":"traceutil/trace.go:171","msg":"trace[167628425] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1191; }","duration":"1.68293571s","start":"2025-07-15T10:58:27.966890Z","end":"2025-07-15T10:58:29.649826Z","steps":["trace[167628425] 'agreement among raft nodes before linearized reading' (duration: 1.682561624s)"],"step_count":1} Jul 15 10:58:29.658130 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.650070Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:27.966873Z","time spent":"1.683088418s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":4070,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:58:29.658130 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.650615Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:29.162018Z","time spent":"487.35117ms","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:58:29.658601 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.650974Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.692404754s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:4046"} Jul 15 10:58:29.658601 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.651108Z","caller":"traceutil/trace.go:171","msg":"trace[374363366] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1191; }","duration":"1.69256623s","start":"2025-07-15T10:58:27.958519Z","end":"2025-07-15T10:58:29.651086Z","steps":["trace[374363366] 'agreement among raft nodes before linearized reading' (duration: 1.692339602s)"],"step_count":1} Jul 15 10:58:29.658601 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.651304Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:27.958501Z","time spent":"1.692717415s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":4070,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:58:29.658601 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.652190Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"504.778174ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-controller-manager\" ","response":"range_response_count:1 size:531"} Jul 15 10:58:29.658601 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.652363Z","caller":"traceutil/trace.go:171","msg":"trace[1704644692] range","detail":"{range_begin:/registry/leases/kube-system/kube-controller-manager; range_end:; response_count:1; response_revision:1191; }","duration":"505.042364ms","start":"2025-07-15T10:58:29.147282Z","end":"2025-07-15T10:58:29.652324Z","steps":["trace[1704644692] 'agreement among raft nodes before linearized reading' (duration: 504.562797ms)"],"step_count":1} Jul 15 10:58:29.658999 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.652567Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:29.147247Z","time spent":"505.243424ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":54,"response count":1,"response size":555,"request content":"key:\"/registry/leases/kube-system/kube-controller-manager\" "} Jul 15 10:58:29.923760 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.921479Z","caller":"traceutil/trace.go:171","msg":"trace[818957715] linearizableReadLoop","detail":"{readStateIndex:1335; appliedIndex:1334; }","duration":"149.318075ms","start":"2025-07-15T10:58:29.772138Z","end":"2025-07-15T10:58:29.921456Z","steps":["trace[818957715] 'read index received' (duration: 56.178173ms)","trace[818957715] 'applied index is now lower than readState.Index' (duration: 93.1388ms)"],"step_count":2} Jul 15 10:58:29.923760 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.921595Z","caller":"traceutil/trace.go:171","msg":"trace[1115594750] transaction","detail":"{read_only:false; response_revision:1193; number_of_response:1; }","duration":"204.417583ms","start":"2025-07-15T10:58:29.717166Z","end":"2025-07-15T10:58:29.921583Z","steps":["trace[1115594750] 'process raft request' (duration: 111.049841ms)","trace[1115594750] 'compare' (duration: 93.110005ms)"],"step_count":2} Jul 15 10:58:29.923760 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.921808Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"149.658138ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/tigera-operator/tigera-ca-private\" ","response":"range_response_count:1 size:3369"} Jul 15 10:58:29.923760 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.921839Z","caller":"traceutil/trace.go:171","msg":"trace[750516083] range","detail":"{range_begin:/registry/secrets/tigera-operator/tigera-ca-private; range_end:; response_count:1; response_revision:1193; }","duration":"149.748279ms","start":"2025-07-15T10:58:29.772082Z","end":"2025-07-15T10:58:29.921830Z","steps":["trace[750516083] 'agreement among raft nodes before linearized reading' (duration: 149.588526ms)"],"step_count":1} Jul 15 10:58:29.923760 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.921963Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"149.680902ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/tigera-csr-creator\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:29.924148 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.922032Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"149.436961ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:4046"} Jul 15 10:58:29.924148 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.922063Z","caller":"traceutil/trace.go:171","msg":"trace[1507363844] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1193; }","duration":"149.486544ms","start":"2025-07-15T10:58:29.772567Z","end":"2025-07-15T10:58:29.922054Z","steps":["trace[1507363844] 'agreement among raft nodes before linearized reading' (duration: 149.400441ms)"],"step_count":1} Jul 15 10:58:29.924148 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.922092Z","caller":"traceutil/trace.go:171","msg":"trace[1273936821] range","detail":"{range_begin:/registry/clusterroles/tigera-csr-creator; range_end:; response_count:0; response_revision:1193; }","duration":"149.847216ms","start":"2025-07-15T10:58:29.772204Z","end":"2025-07-15T10:58:29.922052Z","steps":["trace[1273936821] 'agreement among raft nodes before linearized reading' (duration: 149.676032ms)"],"step_count":1} Jul 15 10:58:29.924148 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.922201Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"149.566776ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:4046"} Jul 15 10:58:29.924148 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.922240Z","caller":"traceutil/trace.go:171","msg":"trace[1513832421] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1193; }","duration":"149.625446ms","start":"2025-07-15T10:58:29.772607Z","end":"2025-07-15T10:58:29.922232Z","steps":["trace[1513832421] 'agreement among raft nodes before linearized reading' (duration: 149.547137ms)"],"step_count":1} Jul 15 10:58:29.924500 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.922411Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"149.782303ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/calico-system/whisker\" ","response":"range_response_count:1 size:426"} Jul 15 10:58:29.924500 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.922442Z","caller":"traceutil/trace.go:171","msg":"trace[728855890] range","detail":"{range_begin:/registry/serviceaccounts/calico-system/whisker; range_end:; response_count:1; response_revision:1193; }","duration":"149.838039ms","start":"2025-07-15T10:58:29.772597Z","end":"2025-07-15T10:58:29.922435Z","steps":["trace[728855890] 'agreement among raft nodes before linearized reading' (duration: 149.716279ms)"],"step_count":1} Jul 15 10:58:29.924500 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.922589Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"150.012798ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:4046"} Jul 15 10:58:29.924500 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.922665Z","caller":"traceutil/trace.go:171","msg":"trace[75385373] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1193; }","duration":"150.117286ms","start":"2025-07-15T10:58:29.772526Z","end":"2025-07-15T10:58:29.922643Z","steps":["trace[75385373] 'agreement among raft nodes before linearized reading' (duration: 149.708333ms)"],"step_count":1} Jul 15 10:58:29.924500 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.922701Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"150.167141ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/configmaps/calico-system/goldmane-ca-bundle\" ","response":"range_response_count:1 size:1959"} Jul 15 10:58:29.924790 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.922732Z","caller":"traceutil/trace.go:171","msg":"trace[555665161] range","detail":"{range_begin:/registry/configmaps/calico-system/goldmane-ca-bundle; range_end:; response_count:1; response_revision:1193; }","duration":"150.232113ms","start":"2025-07-15T10:58:29.772491Z","end":"2025-07-15T10:58:29.922723Z","steps":["trace[555665161] 'agreement among raft nodes before linearized reading' (duration: 150.13015ms)"],"step_count":1} Jul 15 10:58:29.924790 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:29.922854Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"150.350557ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:4046"} Jul 15 10:58:29.924790 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:29.922890Z","caller":"traceutil/trace.go:171","msg":"trace[450829786] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1193; }","duration":"150.412574ms","start":"2025-07-15T10:58:29.772470Z","end":"2025-07-15T10:58:29.922882Z","steps":["trace[450829786] 'agreement among raft nodes before linearized reading' (duration: 150.338003ms)"],"step_count":1} Jul 15 10:58:31.419757 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:31.417688Z","caller":"traceutil/trace.go:171","msg":"trace[1033750396] linearizableReadLoop","detail":"{readStateIndex:1341; appliedIndex:1340; }","duration":"123.427471ms","start":"2025-07-15T10:58:31.294231Z","end":"2025-07-15T10:58:31.417658Z","steps":["trace[1033750396] 'read index received' (duration: 123.119598ms)","trace[1033750396] 'applied index is now lower than readState.Index' (duration: 306.831µs)"],"step_count":2} Jul 15 10:58:31.419757 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:31.417871Z","caller":"traceutil/trace.go:171","msg":"trace[1421575621] transaction","detail":"{read_only:false; response_revision:1198; number_of_response:1; }","duration":"130.011934ms","start":"2025-07-15T10:58:31.287843Z","end":"2025-07-15T10:58:31.417855Z","steps":["trace[1421575621] 'process raft request' (duration: 129.595607ms)"],"step_count":1} Jul 15 10:58:31.419757 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:31.418171Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"120.256063ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/calico-system/whisker-backend-key-pair\" ","response":"range_response_count:1 size:4802"} Jul 15 10:58:31.419757 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:31.418244Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"123.988522ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/networkpolicies.crd.projectcalico.org\" ","response":"range_response_count:1 size:34255"} Jul 15 10:58:31.420901 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:31.418300Z","caller":"traceutil/trace.go:171","msg":"trace[418132485] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/networkpolicies.crd.projectcalico.org; range_end:; response_count:1; response_revision:1198; }","duration":"124.099031ms","start":"2025-07-15T10:58:31.294185Z","end":"2025-07-15T10:58:31.418284Z","steps":["trace[418132485] 'agreement among raft nodes before linearized reading' (duration: 123.764439ms)"],"step_count":1} Jul 15 10:58:31.420901 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:31.418332Z","caller":"traceutil/trace.go:171","msg":"trace[588256144] range","detail":"{range_begin:/registry/secrets/calico-system/whisker-backend-key-pair; range_end:; response_count:1; response_revision:1198; }","duration":"120.411427ms","start":"2025-07-15T10:58:31.297853Z","end":"2025-07-15T10:58:31.418264Z","steps":["trace[588256144] 'agreement among raft nodes before linearized reading' (duration: 120.19104ms)"],"step_count":1} Jul 15 10:58:31.420901 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:31.418804Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"120.86237ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/tigera-operator/goldmane-key-pair\" ","response":"range_response_count:1 size:4711"} Jul 15 10:58:31.420901 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:31.418861Z","caller":"traceutil/trace.go:171","msg":"trace[844496907] range","detail":"{range_begin:/registry/secrets/tigera-operator/goldmane-key-pair; range_end:; response_count:1; response_revision:1198; }","duration":"120.940928ms","start":"2025-07-15T10:58:31.297905Z","end":"2025-07-15T10:58:31.418846Z","steps":["trace[844496907] 'agreement among raft nodes before linearized reading' (duration: 120.793519ms)"],"step_count":1} Jul 15 10:58:33.030151 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:33.028903Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073720,"retry-timeout":"500ms"} Jul 15 10:58:33.531109 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:33.530545Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073720,"retry-timeout":"500ms"} Jul 15 10:58:34.032304 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:34.031692Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073720,"retry-timeout":"500ms"} Jul 15 10:58:34.532551 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:34.531878Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073720,"retry-timeout":"500ms"} Jul 15 10:58:34.964322 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:34.963025Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000748346s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 10:58:34.964322 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:34.963155Z","caller":"traceutil/trace.go:171","msg":"trace[1500801203] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.001028767s","start":"2025-07-15T10:58:32.962094Z","end":"2025-07-15T10:58:34.963123Z","steps":["trace[1500801203] 'agreement among raft nodes before linearized reading' (duration: 2.0007932s)"],"step_count":1} Jul 15 10:58:34.964322 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:34.963237Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.962054Z","time spent":"2.001159926s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:58:35.033287 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:35.032728Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073720,"retry-timeout":"500ms"} Jul 15 10:58:35.534190 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:35.533618Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073720,"retry-timeout":"500ms"} Jul 15 10:58:36.034729 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:36.034217Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073720,"retry-timeout":"500ms"} Jul 15 10:58:36.534936 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:36.534526Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073720,"retry-timeout":"500ms"} Jul 15 10:58:36.963843 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:36.963448Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000105757s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 10:58:36.963843 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:36.963532Z","caller":"traceutil/trace.go:171","msg":"trace[1390795377] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.000261321s","start":"2025-07-15T10:58:34.963249Z","end":"2025-07-15T10:58:36.963511Z","steps":["trace[1390795377] 'agreement among raft nodes before linearized reading' (duration: 2.00015018s)"],"step_count":1} Jul 15 10:58:36.964549 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:36.963757Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:34.963220Z","time spent":"2.000521935s","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:58:37.035795 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:37.035324Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073720,"retry-timeout":"500ms"} Jul 15 10:58:42.920001 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:37.535792Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073720,"retry-timeout":"500ms"} Jul 15 10:58:42.920001 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:38.036919Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073720,"retry-timeout":"500ms"} Jul 15 10:58:42.920001 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:38.537641Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073720,"retry-timeout":"500ms"} Jul 15 10:58:42.920001 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:38.689004Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:33.690497Z","time spent":"4.998495075s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 10:58:42.920001 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:38.751420Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:33.756645Z","time spent":"4.994763826s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 10:58:42.920001 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:38.972980Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000069524s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 10:58:42.920001 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:38.973089Z","caller":"traceutil/trace.go:171","msg":"trace[915946170] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.000237513s","start":"2025-07-15T10:58:36.972831Z","end":"2025-07-15T10:58:38.973068Z","steps":["trace[915946170] 'agreement among raft nodes before linearized reading' (duration: 2.00009348s)"],"step_count":1} Jul 15 10:58:42.921969 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:38.973147Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:36.972813Z","time spent":"2.000321491s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:58:42.921969 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:38.999086Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:34.002359Z","time spent":"4.996715567s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 10:58:42.921969 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:39.038120Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073720,"retry-timeout":"500ms"} Jul 15 10:58:42.921969 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:39.527249Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.526650Z","time spent":"7.00058307s","remote":"172.24.4.149:33310","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 10:58:42.921969 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:39.528780Z","caller":"etcdserver/v3_server.go:932","msg":"timed out waiting for read index response (local node might have slow network)","timeout":"7s"} Jul 15 10:58:42.921969 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:39.529266Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.001188945s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/apiservers/default\" ","response":"","error":"etcdserver: request timed out"} Jul 15 10:58:42.922521 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:39.529332Z","caller":"traceutil/trace.go:171","msg":"trace[1779335246] range","detail":"{range_begin:/registry/operator.tigera.io/apiservers/default; range_end:; }","duration":"7.001302459s","start":"2025-07-15T10:58:32.528000Z","end":"2025-07-15T10:58:39.529303Z","steps":["trace[1779335246] 'agreement among raft nodes before linearized reading' (duration: 7.001216386s)"],"step_count":1} Jul 15 10:58:42.922521 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:39.529473Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.527976Z","time spent":"7.001399504s","remote":"172.24.4.149:54882","response type":"/etcdserverpb.KV/Range","request count":0,"request size":49,"response count":0,"response size":0,"request content":"key:\"/registry/operator.tigera.io/apiservers/default\" "} Jul 15 10:58:42.922521 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:40.029969Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073730,"retry-timeout":"500ms"} Jul 15 10:58:42.922521 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:40.527638Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:33.526912Z","time spent":"7.00070014s","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 10:58:42.922521 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:40.531101Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073730,"retry-timeout":"500ms"} Jul 15 10:58:42.922521 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:40.665740Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000025743s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 10:58:42.923176 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:40.665858Z","caller":"traceutil/trace.go:171","msg":"trace[309013395] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.000201956s","start":"2025-07-15T10:58:38.665627Z","end":"2025-07-15T10:58:40.665829Z","steps":["trace[309013395] 'agreement among raft nodes before linearized reading' (duration: 2.000044438s)"],"step_count":1} Jul 15 10:58:42.923176 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:40.665936Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:38.665574Z","time spent":"2.000342383s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:58:42.923176 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:40.974552Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000721834s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 10:58:42.923176 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:40.974668Z","caller":"traceutil/trace.go:171","msg":"trace[2007915770] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.000883861s","start":"2025-07-15T10:58:38.973762Z","end":"2025-07-15T10:58:40.974646Z","steps":["trace[2007915770] 'agreement among raft nodes before linearized reading' (duration: 2.000749676s)"],"step_count":1} Jul 15 10:58:42.923176 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:40.974733Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:38.973744Z","time spent":"2.000974052s","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:58:42.923176 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:41.031616Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073730,"retry-timeout":"500ms"} Jul 15 10:58:42.923745 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:41.532287Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073730,"retry-timeout":"500ms"} Jul 15 10:58:42.923745 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:42.033514Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073730,"retry-timeout":"500ms"} Jul 15 10:58:42.923745 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:42.609589Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885073730,"retry-timeout":"500ms"} Jul 15 10:58:42.923745 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:42.610974Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:33.649291Z","time spent":"8.961669001s","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 10:58:42.923745 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:42.695098Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:35.694702Z","time spent":"7.000386129s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 10:58:42.984111 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:42.983137Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.999980128s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 10:58:42.984111 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:42.983258Z","caller":"traceutil/trace.go:171","msg":"trace[1447475060] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.000118521s","start":"2025-07-15T10:58:40.983108Z","end":"2025-07-15T10:58:42.983227Z","steps":["trace[1447475060] 'agreement among raft nodes before linearized reading' (duration: 1.99997574s)"],"step_count":1} Jul 15 10:58:42.984111 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:42.983319Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:40.983044Z","time spent":"2.000261863s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:58:43.031043 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.025741Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"10.498915s","expected-duration":"1s"} Jul 15 10:58:43.647119 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.643929Z","caller":"traceutil/trace.go:171","msg":"trace[1752231910] linearizableReadLoop","detail":"{readStateIndex:1380; appliedIndex:1372; }","duration":"4.114708434s","start":"2025-07-15T10:58:39.529160Z","end":"2025-07-15T10:58:43.643869Z","steps":["trace[1752231910] 'read index received' (duration: 3.497803551s)","trace[1752231910] 'applied index is now lower than readState.Index' (duration: 616.902789ms)"],"step_count":2} Jul 15 10:58:43.647119 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.644172Z","caller":"traceutil/trace.go:171","msg":"trace[1239641908] transaction","detail":"{read_only:false; response_revision:1236; number_of_response:1; }","duration":"6.952581352s","start":"2025-07-15T10:58:36.691541Z","end":"2025-07-15T10:58:43.644123Z","steps":["trace[1239641908] 'process raft request' (duration: 6.951050947s)"],"step_count":1} Jul 15 10:58:43.647119 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.644437Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:36.691513Z","time spent":"6.952722049s","remote":"172.24.4.149:33534","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":5198,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:43.647119 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.644457Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"9.124145958s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/networksets/\" range_end:\"/registry/crd.projectcalico.org/networksets0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:58:43.648203 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.644656Z","caller":"traceutil/trace.go:171","msg":"trace[639944372] range","detail":"{range_begin:/registry/crd.projectcalico.org/networksets/; range_end:/registry/crd.projectcalico.org/networksets0; response_count:0; response_revision:1240; }","duration":"9.12448467s","start":"2025-07-15T10:58:34.520139Z","end":"2025-07-15T10:58:43.644624Z","steps":["trace[639944372] 'agreement among raft nodes before linearized reading' (duration: 9.124160464s)"],"step_count":1} Jul 15 10:58:43.648203 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.644767Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:34.520104Z","time spent":"9.12463694s","remote":"172.24.4.149:54570","response type":"/etcdserverpb.KV/Range","request count":0,"request size":94,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/networksets/\" range_end:\"/registry/crd.projectcalico.org/networksets0\" count_only:true "} Jul 15 10:58:43.648203 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.644811Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"10.990209466s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:4046"} Jul 15 10:58:43.648203 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.644902Z","caller":"traceutil/trace.go:171","msg":"trace[1726550765] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1240; }","duration":"10.990348889s","start":"2025-07-15T10:58:32.654529Z","end":"2025-07-15T10:58:43.644878Z","steps":["trace[1726550765] 'agreement among raft nodes before linearized reading' (duration: 10.990072644s)"],"step_count":1} Jul 15 10:58:43.648825 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.644982Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.654460Z","time spent":"10.990497973s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":4070,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:58:43.648825 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.644976Z","caller":"traceutil/trace.go:171","msg":"trace[584794319] transaction","detail":"{read_only:false; response_revision:1237; number_of_response:1; }","duration":"6.435044939s","start":"2025-07-15T10:58:37.209912Z","end":"2025-07-15T10:58:43.644957Z","steps":["trace[584794319] 'process raft request' (duration: 6.432923119s)"],"step_count":1} Jul 15 10:58:43.648825 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.645174Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:37.209886Z","time spent":"6.435170267s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":624,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:43.648825 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.645277Z","caller":"traceutil/trace.go:171","msg":"trace[461250310] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1237; }","duration":"957.746648ms","start":"2025-07-15T10:58:42.687512Z","end":"2025-07-15T10:58:43.645259Z","steps":["trace[461250310] 'process raft request' (duration: 955.529167ms)"],"step_count":1} Jul 15 10:58:43.649232 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.645372Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:42.687478Z","time spent":"957.829124ms","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":55,"response count":0,"response size":1236,"request content":"compare: success:> failure: >"} Jul 15 10:58:43.649232 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.645602Z","caller":"traceutil/trace.go:171","msg":"trace[62117705] transaction","detail":"{read_only:false; response_revision:1238; number_of_response:1; }","duration":"955.288291ms","start":"2025-07-15T10:58:42.690281Z","end":"2025-07-15T10:58:43.645569Z","steps":["trace[62117705] 'process raft request' (duration: 952.828832ms)"],"step_count":1} Jul 15 10:58:43.649232 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.645681Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:42.690252Z","time spent":"955.38803ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":835,"response count":0,"response size":40,"request content":"compare: success:> failure:<>"} Jul 15 10:58:43.649232 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.645790Z","caller":"traceutil/trace.go:171","msg":"trace[1995145838] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1238; }","duration":"955.144969ms","start":"2025-07-15T10:58:42.690631Z","end":"2025-07-15T10:58:43.645776Z","steps":["trace[1995145838] 'process raft request' (duration: 952.595149ms)"],"step_count":1} Jul 15 10:58:43.655272 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.645867Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:42.690623Z","time spent":"955.199383ms","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":56,"response count":0,"response size":1220,"request content":"compare: success:> failure: >"} Jul 15 10:58:43.655272 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.646049Z","caller":"traceutil/trace.go:171","msg":"trace[302632170] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1238; }","duration":"945.015061ms","start":"2025-07-15T10:58:42.701017Z","end":"2025-07-15T10:58:43.646032Z","steps":["trace[302632170] 'process raft request' (duration: 942.270751ms)"],"step_count":1} Jul 15 10:58:43.655272 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.646134Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:42.700986Z","time spent":"945.101426ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":69,"response count":0,"response size":748,"request content":"compare: success:> failure: >"} Jul 15 10:58:43.655272 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.645372Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.013511917s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rolebindings/kube-system/calico-apiserver-auth-reader\" ","response":"range_response_count:1 size:679"} Jul 15 10:58:43.655796 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.647584Z","caller":"traceutil/trace.go:171","msg":"trace[108703825] range","detail":"{range_begin:/registry/rolebindings/kube-system/calico-apiserver-auth-reader; range_end:; response_count:1; response_revision:1240; }","duration":"11.015791034s","start":"2025-07-15T10:58:32.631763Z","end":"2025-07-15T10:58:43.647554Z","steps":["trace[108703825] 'agreement among raft nodes before linearized reading' (duration: 11.013456291s)"],"step_count":1} Jul 15 10:58:43.655796 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.647677Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.631726Z","time spent":"11.015923905s","remote":"172.24.4.149:33358","response type":"/etcdserverpb.KV/Range","request count":0,"request size":65,"response count":1,"response size":703,"request content":"key:\"/registry/rolebindings/kube-system/calico-apiserver-auth-reader\" "} Jul 15 10:58:43.655796 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.647895Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.043806841s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/calico-tiered-policy-passthrough\" ","response":"range_response_count:1 size:638"} Jul 15 10:58:43.655796 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.647939Z","caller":"traceutil/trace.go:171","msg":"trace[697576266] range","detail":"{range_begin:/registry/clusterroles/calico-tiered-policy-passthrough; range_end:; response_count:1; response_revision:1240; }","duration":"11.043896932s","start":"2025-07-15T10:58:32.604028Z","end":"2025-07-15T10:58:43.647925Z","steps":["trace[697576266] 'agreement among raft nodes before linearized reading' (duration: 11.043731818s)"],"step_count":1} Jul 15 10:58:43.656155 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.647976Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.604001Z","time spent":"11.043963588s","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":57,"response count":1,"response size":662,"request content":"key:\"/registry/clusterroles/calico-tiered-policy-passthrough\" "} Jul 15 10:58:43.656155 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.648151Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.069850076s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/calico-kube-controllers\" ","response":"range_response_count:1 size:1152"} Jul 15 10:58:43.656155 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.648193Z","caller":"traceutil/trace.go:171","msg":"trace[874284785] range","detail":"{range_begin:/registry/clusterroles/calico-kube-controllers; range_end:; response_count:1; response_revision:1240; }","duration":"11.069928213s","start":"2025-07-15T10:58:32.578251Z","end":"2025-07-15T10:58:43.648180Z","steps":["trace[874284785] 'agreement among raft nodes before linearized reading' (duration: 11.069782697s)"],"step_count":1} Jul 15 10:58:43.656155 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.648229Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.578231Z","time spent":"11.069987547s","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":1,"response size":1176,"request content":"key:\"/registry/clusterroles/calico-kube-controllers\" "} Jul 15 10:58:43.656155 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.648425Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.073166071s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/imagesets.operator.tigera.io\" ","response":"range_response_count:1 size:4490"} Jul 15 10:58:43.658259 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.648475Z","caller":"traceutil/trace.go:171","msg":"trace[1499218015] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/imagesets.operator.tigera.io; range_end:; response_count:1; response_revision:1240; }","duration":"11.073251945s","start":"2025-07-15T10:58:32.575206Z","end":"2025-07-15T10:58:43.648458Z","steps":["trace[1499218015] 'agreement among raft nodes before linearized reading' (duration: 11.073085159s)"],"step_count":1} Jul 15 10:58:43.658259 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.648540Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.575070Z","time spent":"11.073438529s","remote":"172.24.4.149:33078","response type":"/etcdserverpb.KV/Range","request count":0,"request size":87,"response count":1,"response size":4514,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/imagesets.operator.tigera.io\" "} Jul 15 10:58:43.658259 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.648701Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.088225846s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/whisker:csr-creator\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:43.658259 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.648761Z","caller":"traceutil/trace.go:171","msg":"trace[461829038] range","detail":"{range_begin:/registry/clusterrolebindings/whisker:csr-creator; range_end:; response_count:0; response_revision:1240; }","duration":"11.088303514s","start":"2025-07-15T10:58:32.560425Z","end":"2025-07-15T10:58:43.648729Z","steps":["trace[461829038] 'agreement among raft nodes before linearized reading' (duration: 11.088245473s)"],"step_count":1} Jul 15 10:58:43.658824 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.648803Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.560364Z","time spent":"11.088426065s","remote":"172.24.4.149:33376","response type":"/etcdserverpb.KV/Range","request count":0,"request size":51,"response count":0,"response size":29,"request content":"key:\"/registry/clusterrolebindings/whisker:csr-creator\" "} Jul 15 10:58:43.658824 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.648964Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.094240888s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/calico-extension-apiserver-auth-access\" ","response":"range_response_count:1 size:690"} Jul 15 10:58:43.658824 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.649007Z","caller":"traceutil/trace.go:171","msg":"trace[827618826] range","detail":"{range_begin:/registry/clusterroles/calico-extension-apiserver-auth-access; range_end:; response_count:1; response_revision:1240; }","duration":"11.094321121s","start":"2025-07-15T10:58:32.554673Z","end":"2025-07-15T10:58:43.648994Z","steps":["trace[827618826] 'agreement among raft nodes before linearized reading' (duration: 11.09421002s)"],"step_count":1} Jul 15 10:58:43.658824 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.649043Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.554643Z","time spent":"11.094388518s","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":63,"response count":1,"response size":714,"request content":"key:\"/registry/clusterroles/calico-extension-apiserver-auth-access\" "} Jul 15 10:58:43.658824 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.649105Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.109249907s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/tigera-webhook-reader\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:43.659239 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.649136Z","caller":"traceutil/trace.go:171","msg":"trace[1465691231] range","detail":"{range_begin:/registry/clusterroles/tigera-webhook-reader; range_end:; response_count:0; response_revision:1240; }","duration":"11.109310511s","start":"2025-07-15T10:58:32.539815Z","end":"2025-07-15T10:58:43.649126Z","steps":["trace[1465691231] 'agreement among raft nodes before linearized reading' (duration: 11.10926765s)"],"step_count":1} Jul 15 10:58:43.659239 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.649174Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.539793Z","time spent":"11.109370444s","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":46,"response count":0,"response size":29,"request content":"key:\"/registry/clusterroles/tigera-webhook-reader\" "} Jul 15 10:58:43.659239 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.649282Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.114148475s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:4046"} Jul 15 10:58:43.659239 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.649323Z","caller":"traceutil/trace.go:171","msg":"trace[1513123516] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1240; }","duration":"11.114217887s","start":"2025-07-15T10:58:32.535088Z","end":"2025-07-15T10:58:43.649306Z","steps":["trace[1513123516] 'agreement among raft nodes before linearized reading' (duration: 11.114120552s)"],"step_count":1} Jul 15 10:58:43.659655 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.649358Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.535067Z","time spent":"11.114279402s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":4070,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:58:43.659655 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.649745Z","caller":"traceutil/trace.go:171","msg":"trace[603767523] transaction","detail":"{read_only:false; response_revision:1239; number_of_response:1; }","duration":"899.210004ms","start":"2025-07-15T10:58:42.750496Z","end":"2025-07-15T10:58:43.649706Z","steps":["trace[603767523] 'process raft request' (duration: 892.840391ms)"],"step_count":1} Jul 15 10:58:43.659655 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.649814Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.120862272s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/calico-node\" ","response":"range_response_count:1 size:1903"} Jul 15 10:58:43.659655 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.649909Z","caller":"traceutil/trace.go:171","msg":"trace[1555416168] range","detail":"{range_begin:/registry/clusterroles/calico-node; range_end:; response_count:1; response_revision:1240; }","duration":"11.120936001s","start":"2025-07-15T10:58:32.528917Z","end":"2025-07-15T10:58:43.649853Z","steps":["trace[1555416168] 'agreement among raft nodes before linearized reading' (duration: 11.120686167s)"],"step_count":1} Jul 15 10:58:43.660100 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.649916Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:42.750467Z","time spent":"899.355299ms","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1117,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:43.660100 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.649958Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.528910Z","time spent":"11.121032024s","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":36,"response count":1,"response size":1927,"request content":"key:\"/registry/clusterroles/calico-node\" "} Jul 15 10:58:43.660100 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.650012Z","caller":"traceutil/trace.go:171","msg":"trace[9360134] transaction","detail":"{read_only:false; response_revision:1240; number_of_response:1; }","duration":"895.816371ms","start":"2025-07-15T10:58:42.754179Z","end":"2025-07-15T10:58:43.649996Z","steps":["trace[9360134] 'process raft request' (duration: 889.474871ms)"],"step_count":1} Jul 15 10:58:43.660544 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.650095Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:42.754136Z","time spent":"895.916762ms","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1138,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:43.660544 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.650188Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.650499231s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-controller-manager\" ","response":"range_response_count:1 size:531"} Jul 15 10:58:43.660544 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.650229Z","caller":"traceutil/trace.go:171","msg":"trace[844056317] range","detail":"{range_begin:/registry/leases/kube-system/kube-controller-manager; range_end:; response_count:1; response_revision:1240; }","duration":"4.650559876s","start":"2025-07-15T10:58:38.999656Z","end":"2025-07-15T10:58:43.650216Z","steps":["trace[844056317] 'agreement among raft nodes before linearized reading' (duration: 4.650486356s)"],"step_count":1} Jul 15 10:58:43.660544 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.650336Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.121269494s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/calico-tier-getter\" ","response":"range_response_count:1 size:515"} Jul 15 10:58:43.660897 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.651595Z","caller":"traceutil/trace.go:171","msg":"trace[1713686881] range","detail":"{range_begin:/registry/clusterroles/calico-tier-getter; range_end:; response_count:1; response_revision:1240; }","duration":"11.122700742s","start":"2025-07-15T10:58:32.528874Z","end":"2025-07-15T10:58:43.651575Z","steps":["trace[1713686881] 'agreement among raft nodes before linearized reading' (duration: 11.121188561s)"],"step_count":1} Jul 15 10:58:43.660897 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.651763Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.528864Z","time spent":"11.122835426s","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":43,"response count":1,"response size":539,"request content":"key:\"/registry/clusterroles/calico-tier-getter\" "} Jul 15 10:58:43.660897 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.650348Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:38.999640Z","time spent":"4.650687309s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":54,"response count":1,"response size":555,"request content":"key:\"/registry/leases/kube-system/kube-controller-manager\" "} Jul 15 10:58:43.660897 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.650723Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.897504277s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-scheduler\" ","response":"range_response_count:1 size:504"} Jul 15 10:58:43.660897 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.652038Z","caller":"traceutil/trace.go:171","msg":"trace[191874966] range","detail":"{range_begin:/registry/leases/kube-system/kube-scheduler; range_end:; response_count:1; response_revision:1240; }","duration":"4.898911911s","start":"2025-07-15T10:58:38.753098Z","end":"2025-07-15T10:58:43.652009Z","steps":["trace[191874966] 'agreement among raft nodes before linearized reading' (duration: 4.897274853s)"],"step_count":1} Jul 15 10:58:43.661241 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.652267Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:38.753068Z","time spent":"4.899110007s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":45,"response count":1,"response size":528,"request content":"key:\"/registry/leases/kube-system/kube-scheduler\" "} Jul 15 10:58:43.661241 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.650818Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.959702329s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/tigera-operator/operator-lock\" ","response":"range_response_count:1 size:505"} Jul 15 10:58:43.661241 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.652587Z","caller":"traceutil/trace.go:171","msg":"trace[1305309946] range","detail":"{range_begin:/registry/leases/tigera-operator/operator-lock; range_end:; response_count:1; response_revision:1240; }","duration":"4.961444206s","start":"2025-07-15T10:58:38.691046Z","end":"2025-07-15T10:58:43.652490Z","steps":["trace[1305309946] 'agreement among raft nodes before linearized reading' (duration: 4.959712539s)"],"step_count":1} Jul 15 10:58:43.661241 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.652694Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:38.691015Z","time spent":"4.961657129s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":1,"response size":529,"request content":"key:\"/registry/leases/tigera-operator/operator-lock\" "} Jul 15 10:58:43.661241 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.650874Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.985562151s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/networkpolicies/\" range_end:\"/registry/networkpolicies0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:58:43.661628 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.653208Z","caller":"traceutil/trace.go:171","msg":"trace[2141632950] range","detail":"{range_begin:/registry/networkpolicies/; range_end:/registry/networkpolicies0; response_count:0; response_revision:1240; }","duration":"4.987875254s","start":"2025-07-15T10:58:38.665267Z","end":"2025-07-15T10:58:43.653142Z","steps":["trace[2141632950] 'agreement among raft nodes before linearized reading' (duration: 4.985575558s)"],"step_count":1} Jul 15 10:58:43.661628 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.653619Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:38.665245Z","time spent":"4.988116481s","remote":"172.24.4.149:33316","response type":"/etcdserverpb.KV/Range","request count":0,"request size":58,"response count":3,"response size":31,"request content":"key:\"/registry/networkpolicies/\" range_end:\"/registry/networkpolicies0\" count_only:true "} Jul 15 10:58:43.661628 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.653721Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"6.681358214s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76\" ","response":"range_response_count:1 size:905"} Jul 15 10:58:43.661628 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.656461Z","caller":"traceutil/trace.go:171","msg":"trace[1651349689] range","detail":"{range_begin:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76; range_end:; response_count:1; response_revision:1240; }","duration":"6.686411699s","start":"2025-07-15T10:58:36.970014Z","end":"2025-07-15T10:58:43.656425Z","steps":["trace[1651349689] 'agreement among raft nodes before linearized reading' (duration: 6.681049979s)"],"step_count":1} Jul 15 10:58:43.662469 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.656579Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:36.969935Z","time spent":"6.686609484s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":97,"response count":1,"response size":929,"request content":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76\" "} Jul 15 10:58:43.662469 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.650919Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.184494603s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/gatewayapis/\" range_end:\"/registry/operator.tigera.io/gatewayapis0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:58:43.662469 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.656718Z","caller":"traceutil/trace.go:171","msg":"trace[1407627437] range","detail":"{range_begin:/registry/operator.tigera.io/gatewayapis/; range_end:/registry/operator.tigera.io/gatewayapis0; response_count:0; response_revision:1240; }","duration":"5.190342677s","start":"2025-07-15T10:58:38.466358Z","end":"2025-07-15T10:58:43.656701Z","steps":["trace[1407627437] 'agreement among raft nodes before linearized reading' (duration: 5.184535321s)"],"step_count":1} Jul 15 10:58:43.662469 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.656799Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:38.466337Z","time spent":"5.190416326s","remote":"172.24.4.149:54960","response type":"/etcdserverpb.KV/Range","request count":0,"request size":88,"response count":0,"response size":29,"request content":"key:\"/registry/operator.tigera.io/gatewayapis/\" range_end:\"/registry/operator.tigera.io/gatewayapis0\" count_only:true "} Jul 15 10:58:43.662852 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.651035Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"6.017975901s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/\" range_end:\"/registry/crd.projectcalico.org/clusterinformations0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:58:43.662852 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:43.656887Z","caller":"traceutil/trace.go:171","msg":"trace[1147663330] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/; range_end:/registry/crd.projectcalico.org/clusterinformations0; response_count:0; response_revision:1240; }","duration":"6.023863038s","start":"2025-07-15T10:58:37.633009Z","end":"2025-07-15T10:58:43.656872Z","steps":["trace[1147663330] 'agreement among raft nodes before linearized reading' (duration: 6.01793327s)"],"step_count":1} Jul 15 10:58:43.662852 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:43.657611Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:37.632987Z","time spent":"6.024585088s","remote":"172.24.4.149:54994","response type":"/etcdserverpb.KV/Range","request count":0,"request size":110,"response count":1,"response size":31,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/\" range_end:\"/registry/crd.projectcalico.org/clusterinformations0\" count_only:true "} Jul 15 10:58:44.004819 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.000606Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"289.633858ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-controller-manager\" ","response":"","error":"context canceled"} Jul 15 10:58:44.004819 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.000725Z","caller":"traceutil/trace.go:171","msg":"trace[447972394] range","detail":"{range_begin:/registry/leases/kube-system/kube-controller-manager; range_end:; }","duration":"289.818599ms","start":"2025-07-15T10:58:43.710878Z","end":"2025-07-15T10:58:44.000697Z","steps":["trace[447972394] 'agreement among raft nodes before linearized reading' (duration: 289.666861ms)"],"step_count":1} Jul 15 10:58:44.030646 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.029171Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"306.780376ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/apiservers/default\" ","response":"","error":"context canceled"} Jul 15 10:58:44.030646 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.029320Z","caller":"traceutil/trace.go:171","msg":"trace[1524863612] range","detail":"{range_begin:/registry/operator.tigera.io/apiservers/default; range_end:; }","duration":"306.983272ms","start":"2025-07-15T10:58:43.722300Z","end":"2025-07-15T10:58:44.029284Z","steps":["trace[1524863612] 'agreement among raft nodes before linearized reading' (duration: 306.79792ms)"],"step_count":1} Jul 15 10:58:44.030646 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.029324Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"308.507847ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"","error":"context canceled"} Jul 15 10:58:44.030646 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.029489Z","caller":"traceutil/trace.go:171","msg":"trace[1097596363] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; }","duration":"308.769834ms","start":"2025-07-15T10:58:43.720690Z","end":"2025-07-15T10:58:44.029460Z","steps":["trace[1097596363] 'agreement among raft nodes before linearized reading' (duration: 308.550867ms)"],"step_count":1} Jul 15 10:58:44.030646 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.029486Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:43.722283Z","time spent":"307.169135ms","remote":"172.24.4.149:54882","response type":"/etcdserverpb.KV/Range","request count":0,"request size":49,"response count":0,"response size":0,"request content":"key:\"/registry/operator.tigera.io/apiservers/default\" "} Jul 15 10:58:44.033954 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.029591Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"309.91116ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/apiservers/default\" ","response":"","error":"context canceled"} Jul 15 10:58:44.033954 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.029569Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:43.720658Z","time spent":"308.88833ms","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":0,"response size":0,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:58:44.033954 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.029648Z","caller":"traceutil/trace.go:171","msg":"trace[2138069539] range","detail":"{range_begin:/registry/operator.tigera.io/apiservers/default; range_end:; }","duration":"310.008717ms","start":"2025-07-15T10:58:43.719619Z","end":"2025-07-15T10:58:44.029628Z","steps":["trace[2138069539] 'agreement among raft nodes before linearized reading' (duration: 309.941479ms)"],"step_count":1} Jul 15 10:58:44.033954 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.029664Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"305.579216ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"","error":"context canceled"} Jul 15 10:58:44.033954 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.029700Z","caller":"traceutil/trace.go:171","msg":"trace[1130547311] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; }","duration":"305.638509ms","start":"2025-07-15T10:58:43.724049Z","end":"2025-07-15T10:58:44.029687Z","steps":["trace[1130547311] 'agreement among raft nodes before linearized reading' (duration: 305.60216ms)"],"step_count":1} Jul 15 10:58:44.034474 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.029712Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:43.719599Z","time spent":"310.092475ms","remote":"172.24.4.149:54882","response type":"/etcdserverpb.KV/Range","request count":0,"request size":49,"response count":0,"response size":0,"request content":"key:\"/registry/operator.tigera.io/apiservers/default\" "} Jul 15 10:58:44.034474 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.029742Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:43.724031Z","time spent":"305.698093ms","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":0,"response size":0,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:58:44.034474 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.029793Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"308.745828ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/apiservers/default\" ","response":"","error":"context canceled"} Jul 15 10:58:44.034474 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.029830Z","caller":"traceutil/trace.go:171","msg":"trace[939539226] range","detail":"{range_begin:/registry/operator.tigera.io/apiservers/default; range_end:; }","duration":"308.807616ms","start":"2025-07-15T10:58:43.721011Z","end":"2025-07-15T10:58:44.029818Z","steps":["trace[939539226] 'agreement among raft nodes before linearized reading' (duration: 308.771226ms)"],"step_count":1} Jul 15 10:58:44.034474 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.029850Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"307.37735ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/apiservers/default\" ","response":"","error":"context canceled"} Jul 15 10:58:44.034909 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.029914Z","caller":"traceutil/trace.go:171","msg":"trace[13362033] range","detail":"{range_begin:/registry/operator.tigera.io/apiservers/default; range_end:; }","duration":"307.525923ms","start":"2025-07-15T10:58:43.722369Z","end":"2025-07-15T10:58:44.029894Z","steps":["trace[13362033] 'agreement among raft nodes before linearized reading' (duration: 307.458093ms)"],"step_count":1} Jul 15 10:58:44.034909 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.029867Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:43.720993Z","time spent":"308.862009ms","remote":"172.24.4.149:54882","response type":"/etcdserverpb.KV/Range","request count":0,"request size":49,"response count":0,"response size":0,"request content":"key:\"/registry/operator.tigera.io/apiservers/default\" "} Jul 15 10:58:44.034909 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.030100Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:43.722344Z","time spent":"307.700555ms","remote":"172.24.4.149:54882","response type":"/etcdserverpb.KV/Range","request count":0,"request size":49,"response count":0,"response size":0,"request content":"key:\"/registry/operator.tigera.io/apiservers/default\" "} Jul 15 10:58:44.055362 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.053710Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.524829788s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/calico-typha\" ","response":"","error":"rangeKeys: context cancelled: context canceled"} Jul 15 10:58:44.056787 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.055602Z","caller":"traceutil/trace.go:171","msg":"trace[2012964244] range","detail":"{range_begin:/registry/clusterroles/calico-typha; range_end:; }","duration":"11.526772236s","start":"2025-07-15T10:58:32.528770Z","end":"2025-07-15T10:58:44.055543Z","steps":["trace[2012964244] 'agreement among raft nodes before linearized reading' (duration: 11.12552869s)","trace[2012964244] 'range keys from in-memory index tree' (duration: 399.356293ms)"],"step_count":2} Jul 15 10:58:44.056787 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.055726Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.528763Z","time spent":"11.526926902s","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":0,"response size":0,"request content":"key:\"/registry/clusterroles/calico-typha\" "} Jul 15 10:58:44.056787 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.054376Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.363865772s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/namespaces/calico-apiserver\" ","response":"","error":"rangeKeys: context cancelled: context canceled"} Jul 15 10:58:44.056787 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.055787Z","caller":"traceutil/trace.go:171","msg":"trace[1184798903] range","detail":"{range_begin:/registry/namespaces/calico-apiserver; range_end:; }","duration":"1.365305105s","start":"2025-07-15T10:58:42.690464Z","end":"2025-07-15T10:58:44.055769Z","steps":["trace[1184798903] 'agreement among raft nodes before linearized reading' (duration: 963.997365ms)","trace[1184798903] 'range keys from in-memory index tree' (duration: 399.717369ms)"],"step_count":2} Jul 15 10:58:44.057467 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.054484Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.525719448s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/calico-cni-plugin\" ","response":"","error":"rangeKeys: context cancelled: context canceled"} Jul 15 10:58:44.057467 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.056032Z","caller":"traceutil/trace.go:171","msg":"trace[1255916316] range","detail":"{range_begin:/registry/clusterroles/calico-cni-plugin; range_end:; }","duration":"11.527270362s","start":"2025-07-15T10:58:32.528734Z","end":"2025-07-15T10:58:44.056005Z","steps":["trace[1255916316] 'agreement among raft nodes before linearized reading' (duration: 11.125581339s)","trace[1255916316] 'range keys from in-memory index tree' (duration: 400.146714ms)"],"step_count":2} Jul 15 10:58:44.059032 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.056270Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.528727Z","time spent":"11.527505911s","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":42,"response count":0,"response size":0,"request content":"key:\"/registry/clusterroles/calico-cni-plugin\" "} Jul 15 10:58:44.059032 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.056493Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:42.690455Z","time spent":"1.365907269s","remote":"172.24.4.149:33144","response type":"/etcdserverpb.KV/Range","request count":0,"request size":39,"response count":0,"response size":0,"request content":"key:\"/registry/namespaces/calico-apiserver\" "} Jul 15 10:58:44.059032 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.054521Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.525795351s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/calico-crds\" ","response":"","error":"rangeKeys: context cancelled: context canceled"} Jul 15 10:58:44.059032 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.054560Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.525885081s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/calico-webhook-reader\" ","response":"","error":"rangeKeys: context cancelled: context canceled"} Jul 15 10:58:44.059032 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.056983Z","caller":"traceutil/trace.go:171","msg":"trace[1632614003] range","detail":"{range_begin:/registry/clusterroles/calico-webhook-reader; range_end:; }","duration":"11.528320908s","start":"2025-07-15T10:58:32.528642Z","end":"2025-07-15T10:58:44.056963Z","steps":["trace[1632614003] 'agreement among raft nodes before linearized reading' (duration: 11.12568758s)","trace[1632614003] 'range keys from in-memory index tree' (duration: 400.216227ms)"],"step_count":2} Jul 15 10:58:44.062719 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.057078Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.528626Z","time spent":"11.528429473s","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":46,"response count":0,"response size":0,"request content":"key:\"/registry/clusterroles/calico-webhook-reader\" "} Jul 15 10:58:44.062719 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.054595Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.526282093s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/goldmanes/default\" ","response":"","error":"rangeKeys: context cancelled: context canceled"} Jul 15 10:58:44.062719 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.057126Z","caller":"traceutil/trace.go:171","msg":"trace[1646680289] range","detail":"{range_begin:/registry/operator.tigera.io/goldmanes/default; range_end:; }","duration":"11.528829701s","start":"2025-07-15T10:58:32.528282Z","end":"2025-07-15T10:58:44.057112Z","steps":["trace[1646680289] 'agreement among raft nodes before linearized reading' (duration: 11.126052801s)","trace[1646680289] 'range keys from in-memory index tree' (duration: 400.246225ms)"],"step_count":2} Jul 15 10:58:44.062719 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.057163Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.528268Z","time spent":"11.528882572s","remote":"172.24.4.149:54972","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":0,"response size":0,"request content":"key:\"/registry/operator.tigera.io/goldmanes/default\" "} Jul 15 10:58:44.062719 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.054652Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.470648402s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/daemonsets/\" range_end:\"/registry/daemonsets0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:58:44.063852 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.057232Z","caller":"traceutil/trace.go:171","msg":"trace[118232174] range","detail":"{range_begin:/registry/daemonsets/; range_end:/registry/daemonsets0; response_count:0; response_revision:1240; }","duration":"4.473249002s","start":"2025-07-15T10:58:39.583949Z","end":"2025-07-15T10:58:44.057198Z","steps":["trace[118232174] 'agreement among raft nodes before linearized reading' (duration: 4.070411587s)","trace[118232174] 'count revisions from in-memory index tree' (duration: 400.195077ms)"],"step_count":2} Jul 15 10:58:44.063852 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.057288Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:39.583926Z","time spent":"4.473341867s","remote":"172.24.4.149:33488","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":3,"response size":31,"request content":"key:\"/registry/daemonsets/\" range_end:\"/registry/daemonsets0\" count_only:true "} Jul 15 10:58:44.063852 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.054742Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.069631794s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:44.063852 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.057437Z","caller":"traceutil/trace.go:171","msg":"trace[390295766] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1240; }","duration":"1.072395664s","start":"2025-07-15T10:58:42.985025Z","end":"2025-07-15T10:58:44.057421Z","steps":["trace[390295766] 'agreement among raft nodes before linearized reading' (duration: 669.430576ms)","trace[390295766] 'range keys from in-memory index tree' (duration: 400.267134ms)"],"step_count":2} Jul 15 10:58:44.064498 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.057487Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:42.984992Z","time spent":"1.072477679s","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:58:44.064498 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.054800Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"8.564474246s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/ipreservations/\" range_end:\"/registry/crd.projectcalico.org/ipreservations0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:58:44.064498 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.057568Z","caller":"traceutil/trace.go:171","msg":"trace[40340070] range","detail":"{range_begin:/registry/crd.projectcalico.org/ipreservations/; range_end:/registry/crd.projectcalico.org/ipreservations0; response_count:0; response_revision:1240; }","duration":"8.567275374s","start":"2025-07-15T10:58:35.490275Z","end":"2025-07-15T10:58:44.057551Z","steps":["trace[40340070] 'agreement among raft nodes before linearized reading' (duration: 8.164494566s)","trace[40340070] 'count revisions from in-memory index tree' (duration: 400.008541ms)"],"step_count":2} Jul 15 10:58:44.064498 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.057615Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:35.490252Z","time spent":"8.567343243s","remote":"172.24.4.149:39888","response type":"/etcdserverpb.KV/Range","request count":0,"request size":100,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/ipreservations/\" range_end:\"/registry/crd.projectcalico.org/ipreservations0\" count_only:true "} Jul 15 10:58:44.065056 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.054842Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.379242986s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/managementclusterconnections/\" range_end:\"/registry/operator.tigera.io/managementclusterconnections0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:58:44.065056 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.057687Z","caller":"traceutil/trace.go:171","msg":"trace[1221912630] range","detail":"{range_begin:/registry/operator.tigera.io/managementclusterconnections/; range_end:/registry/operator.tigera.io/managementclusterconnections0; response_count:0; response_revision:1240; }","duration":"3.382135389s","start":"2025-07-15T10:58:40.675536Z","end":"2025-07-15T10:58:44.057671Z","steps":["trace[1221912630] 'agreement among raft nodes before linearized reading' (duration: 2.978930556s)","trace[1221912630] 'count revisions from in-memory index tree' (duration: 400.35457ms)"],"step_count":2} Jul 15 10:58:44.065056 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.057774Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:40.675507Z","time spent":"3.382240547s","remote":"172.24.4.149:54944","response type":"/etcdserverpb.KV/Range","request count":0,"request size":122,"response count":0,"response size":29,"request content":"key:\"/registry/operator.tigera.io/managementclusterconnections/\" range_end:\"/registry/operator.tigera.io/managementclusterconnections0\" count_only:true "} Jul 15 10:58:44.065056 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.057953Z","caller":"traceutil/trace.go:171","msg":"trace[885956821] range","detail":"{range_begin:/registry/clusterroles/calico-crds; range_end:; }","duration":"11.52819031s","start":"2025-07-15T10:58:32.528699Z","end":"2025-07-15T10:58:44.056889Z","steps":["trace[885956821] 'agreement among raft nodes before linearized reading' (duration: 11.125626264s)","trace[885956821] 'range keys from in-memory index tree' (duration: 400.181301ms)"],"step_count":2} Jul 15 10:58:44.065643 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.054891Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.44457709s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/prioritylevelconfigurations/\" range_end:\"/registry/prioritylevelconfigurations0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:58:44.065643 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.055062Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.077748379s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 10:58:44.065643 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.058930Z","caller":"traceutil/trace.go:171","msg":"trace[666360669] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1240; }","duration":"4.081643486s","start":"2025-07-15T10:58:39.977261Z","end":"2025-07-15T10:58:44.058904Z","steps":["trace[666360669] 'agreement among raft nodes before linearized reading' (duration: 3.678730407s)","trace[666360669] 'range keys from in-memory index tree' (duration: 398.923853ms)"],"step_count":2} Jul 15 10:58:44.065643 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.059012Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:39.977228Z","time spent":"4.081759195s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 10:58:44.065643 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.055113Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"9.008272927s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/imagesets/\" range_end:\"/registry/operator.tigera.io/imagesets0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:58:44.067110 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.059119Z","caller":"traceutil/trace.go:171","msg":"trace[189986792] range","detail":"{range_begin:/registry/operator.tigera.io/imagesets/; range_end:/registry/operator.tigera.io/imagesets0; response_count:0; response_revision:1240; }","duration":"9.012314773s","start":"2025-07-15T10:58:35.046786Z","end":"2025-07-15T10:58:44.059101Z","steps":["trace[189986792] 'agreement among raft nodes before linearized reading' (duration: 8.607990515s)","trace[189986792] 'count revisions from in-memory index tree' (duration: 400.319933ms)"],"step_count":2} Jul 15 10:58:44.067110 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.059179Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:35.046759Z","time spent":"9.012400115s","remote":"172.24.4.149:54904","response type":"/etcdserverpb.KV/Range","request count":0,"request size":84,"response count":0,"response size":29,"request content":"key:\"/registry/operator.tigera.io/imagesets/\" range_end:\"/registry/operator.tigera.io/imagesets0\" count_only:true "} Jul 15 10:58:44.067110 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.055159Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.655542009s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/stagedglobalnetworkpolicies/\" range_end:\"/registry/crd.projectcalico.org/stagedglobalnetworkpolicies0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:58:44.067658 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.059284Z","caller":"traceutil/trace.go:171","msg":"trace[776697865] range","detail":"{range_begin:/registry/crd.projectcalico.org/stagedglobalnetworkpolicies/; range_end:/registry/crd.projectcalico.org/stagedglobalnetworkpolicies0; response_count:0; response_revision:1240; }","duration":"3.659754839s","start":"2025-07-15T10:58:40.399504Z","end":"2025-07-15T10:58:44.059259Z","steps":["trace[776697865] 'agreement among raft nodes before linearized reading' (duration: 3.25649942s)","trace[776697865] 'count revisions from in-memory index tree' (duration: 399.139483ms)"],"step_count":2} Jul 15 10:58:44.067658 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.060277Z","caller":"traceutil/trace.go:171","msg":"trace[893608434] range","detail":"{range_begin:/registry/prioritylevelconfigurations/; range_end:/registry/prioritylevelconfigurations0; response_count:0; response_revision:1240; }","duration":"7.448456087s","start":"2025-07-15T10:58:36.610260Z","end":"2025-07-15T10:58:44.058716Z","steps":["trace[893608434] 'agreement among raft nodes before linearized reading' (duration: 7.044496641s)","trace[893608434] 'count revisions from in-memory index tree' (duration: 400.112912ms)"],"step_count":2} Jul 15 10:58:44.067658 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.061074Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:36.610185Z","time spent":"7.450855653s","remote":"172.24.4.149:33428","response type":"/etcdserverpb.KV/Range","request count":0,"request size":82,"response count":8,"response size":31,"request content":"key:\"/registry/prioritylevelconfigurations/\" range_end:\"/registry/prioritylevelconfigurations0\" count_only:true "} Jul 15 10:58:44.068147 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.060120Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:40.399466Z","time spent":"3.660457824s","remote":"172.24.4.149:54524","response type":"/etcdserverpb.KV/Range","request count":0,"request size":126,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/stagedglobalnetworkpolicies/\" range_end:\"/registry/crd.projectcalico.org/stagedglobalnetworkpolicies0\" count_only:true "} Jul 15 10:58:44.068147 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.058588Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:32.528690Z","time spent":"11.529860141s","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":36,"response count":0,"response size":0,"request content":"key:\"/registry/clusterroles/calico-crds\" "} Jul 15 10:58:44.112030 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.111025Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"124.411446ms","expected-duration":"100ms","prefix":"","request":"header: lease_revoke:","response":"size:29"} Jul 15 10:58:44.178950 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.177934Z","caller":"traceutil/trace.go:171","msg":"trace[765896057] transaction","detail":"{read_only:false; response_revision:1241; number_of_response:1; }","duration":"470.824259ms","start":"2025-07-15T10:58:43.707065Z","end":"2025-07-15T10:58:44.177889Z","steps":["trace[765896057] 'process raft request' (duration: 470.543706ms)"],"step_count":1} Jul 15 10:58:44.178950 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.178502Z","caller":"traceutil/trace.go:171","msg":"trace[74071841] transaction","detail":"{read_only:false; response_revision:1242; number_of_response:1; }","duration":"416.180755ms","start":"2025-07-15T10:58:43.762293Z","end":"2025-07-15T10:58:44.178474Z","steps":["trace[74071841] 'process raft request' (duration: 415.459365ms)"],"step_count":1} Jul 15 10:58:44.181425 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.179538Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:43.762252Z","time spent":"417.207064ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":419,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:44.181425 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.178527Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:43.707024Z","time spent":"471.018448ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":890,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:44.181425 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.179268Z","caller":"traceutil/trace.go:171","msg":"trace[1466918379] transaction","detail":"{read_only:false; response_revision:1243; number_of_response:1; }","duration":"364.450549ms","start":"2025-07-15T10:58:43.814787Z","end":"2025-07-15T10:58:44.179237Z","steps":["trace[1466918379] 'process raft request' (duration: 363.036383ms)"],"step_count":1} Jul 15 10:58:44.182608 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.179891Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:43.814628Z","time spent":"365.186407ms","remote":"172.24.4.149:33534","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":4949,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:58:44.182608 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.179350Z","caller":"traceutil/trace.go:171","msg":"trace[1302866172] linearizableReadLoop","detail":"{readStateIndex:1389; appliedIndex:1386; }","duration":"468.36441ms","start":"2025-07-15T10:58:43.710956Z","end":"2025-07-15T10:58:44.179320Z","steps":["trace[1302866172] 'read index received' (duration: 275.584033ms)","trace[1302866172] 'applied index is now lower than readState.Index' (duration: 192.779095ms)"],"step_count":2} Jul 15 10:58:44.182608 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.180265Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"462.277514ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/apiserver-jtty2bqg5eqwaymygkxixvrno4\" ","response":"range_response_count:1 size:713"} Jul 15 10:58:44.182608 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.180354Z","caller":"traceutil/trace.go:171","msg":"trace[422223295] range","detail":"{range_begin:/registry/leases/kube-system/apiserver-jtty2bqg5eqwaymygkxixvrno4; range_end:; response_count:1; response_revision:1243; }","duration":"462.383334ms","start":"2025-07-15T10:58:43.717948Z","end":"2025-07-15T10:58:44.180331Z","steps":["trace[422223295] 'agreement among raft nodes before linearized reading' (duration: 462.095608ms)"],"step_count":1} Jul 15 10:58:44.183081 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.180667Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"102.33628ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:58:44.183081 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.180737Z","caller":"traceutil/trace.go:171","msg":"trace[1898380192] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1243; }","duration":"102.469323ms","start":"2025-07-15T10:58:44.078248Z","end":"2025-07-15T10:58:44.180718Z","steps":["trace[1898380192] 'agreement among raft nodes before linearized reading' (duration: 102.33076ms)"],"step_count":1} Jul 15 10:58:44.183081 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.180873Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:58:43.717940Z","time spent":"462.894576ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":67,"response count":1,"response size":737,"request content":"key:\"/registry/leases/kube-system/apiserver-jtty2bqg5eqwaymygkxixvrno4\" "} Jul 15 10:58:44.183081 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.181053Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"109.284827ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal\" ","response":"range_response_count:1 size:6418"} Jul 15 10:58:44.183081 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.181105Z","caller":"traceutil/trace.go:171","msg":"trace[274147337] range","detail":"{range_begin:/registry/pods/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal; range_end:; response_count:1; response_revision:1243; }","duration":"109.337948ms","start":"2025-07-15T10:58:44.071751Z","end":"2025-07-15T10:58:44.181089Z","steps":["trace[274147337] 'agreement among raft nodes before linearized reading' (duration: 109.141965ms)"],"step_count":1} Jul 15 10:58:44.307477 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.306248Z","caller":"traceutil/trace.go:171","msg":"trace[1311321278] transaction","detail":"{read_only:false; response_revision:1244; number_of_response:1; }","duration":"117.469412ms","start":"2025-07-15T10:58:44.188740Z","end":"2025-07-15T10:58:44.306210Z","steps":["trace[1311321278] 'process raft request' (duration: 107.549914ms)"],"step_count":1} Jul 15 10:58:44.365274 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.363467Z","caller":"traceutil/trace.go:171","msg":"trace[195182330] linearizableReadLoop","detail":"{readStateIndex:1395; appliedIndex:1392; }","duration":"153.654472ms","start":"2025-07-15T10:58:44.209718Z","end":"2025-07-15T10:58:44.363373Z","steps":["trace[195182330] 'read index received' (duration: 86.585128ms)","trace[195182330] 'applied index is now lower than readState.Index' (duration: 67.068182ms)"],"step_count":2} Jul 15 10:58:44.365274 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.363675Z","caller":"traceutil/trace.go:171","msg":"trace[1615107194] transaction","detail":"{read_only:false; response_revision:1246; number_of_response:1; }","duration":"165.297644ms","start":"2025-07-15T10:58:44.198338Z","end":"2025-07-15T10:58:44.363636Z","steps":["trace[1615107194] 'process raft request' (duration: 164.801101ms)"],"step_count":1} Jul 15 10:58:44.365274 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.363947Z","caller":"traceutil/trace.go:171","msg":"trace[1157516113] transaction","detail":"{read_only:false; response_revision:1247; number_of_response:1; }","duration":"134.31854ms","start":"2025-07-15T10:58:44.229607Z","end":"2025-07-15T10:58:44.363925Z","steps":["trace[1157516113] 'process raft request' (duration: 133.6129ms)"],"step_count":1} Jul 15 10:58:44.365274 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.364239Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"154.492244ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678d1663474b\" ","response":"range_response_count:1 size:904"} Jul 15 10:58:44.365856 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.364479Z","caller":"traceutil/trace.go:171","msg":"trace[2054226624] range","detail":"{range_begin:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678d1663474b; range_end:; response_count:1; response_revision:1247; }","duration":"154.776464ms","start":"2025-07-15T10:58:44.209679Z","end":"2025-07-15T10:58:44.364456Z","steps":["trace[2054226624] 'agreement among raft nodes before linearized reading' (duration: 154.376303ms)"],"step_count":1} Jul 15 10:58:44.583219 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.580340Z","caller":"traceutil/trace.go:171","msg":"trace[608911207] linearizableReadLoop","detail":"{readStateIndex:1398; appliedIndex:1397; }","duration":"180.815956ms","start":"2025-07-15T10:58:44.399483Z","end":"2025-07-15T10:58:44.580298Z","steps":["trace[608911207] 'read index received' (duration: 131.164491ms)","trace[608911207] 'applied index is now lower than readState.Index' (duration: 49.650042ms)"],"step_count":2} Jul 15 10:58:44.583219 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.580660Z","caller":"traceutil/trace.go:171","msg":"trace[2027799929] transaction","detail":"{read_only:false; response_revision:1249; number_of_response:1; }","duration":"181.881659ms","start":"2025-07-15T10:58:44.398750Z","end":"2025-07-15T10:58:44.580632Z","steps":["trace[2027799929] 'process raft request' (duration: 132.039442ms)","trace[2027799929] 'compare' (duration: 49.22764ms)"],"step_count":2} Jul 15 10:58:44.583219 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:58:44.581015Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"181.502458ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76\" ","response":"range_response_count:1 size:905"} Jul 15 10:58:44.583219 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:44.581622Z","caller":"traceutil/trace.go:171","msg":"trace[70356680] range","detail":"{range_begin:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76; range_end:; response_count:1; response_revision:1249; }","duration":"181.635893ms","start":"2025-07-15T10:58:44.399438Z","end":"2025-07-15T10:58:44.581074Z","steps":["trace[70356680] 'agreement among raft nodes before linearized reading' (duration: 181.320613ms)"],"step_count":1} Jul 15 10:58:49.791669 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:49.788298Z","caller":"traceutil/trace.go:171","msg":"trace[1272641907] transaction","detail":"{read_only:false; response_revision:1266; number_of_response:1; }","duration":"121.616534ms","start":"2025-07-15T10:58:49.666624Z","end":"2025-07-15T10:58:49.788241Z","steps":["trace[1272641907] 'process raft request' (duration: 121.035198ms)"],"step_count":1} Jul 15 10:58:54.788641 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:54.785374Z","caller":"traceutil/trace.go:171","msg":"trace[1770436601] transaction","detail":"{read_only:false; response_revision:1282; number_of_response:1; }","duration":"100.117108ms","start":"2025-07-15T10:58:54.685145Z","end":"2025-07-15T10:58:54.785262Z","steps":["trace[1770436601] 'process raft request' (duration: 99.923068ms)"],"step_count":1} Jul 15 10:58:54.845432 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:58:54.844468Z","caller":"traceutil/trace.go:171","msg":"trace[632767561] transaction","detail":"{read_only:false; response_revision:1283; number_of_response:1; }","duration":"158.544142ms","start":"2025-07-15T10:58:54.685892Z","end":"2025-07-15T10:58:54.844436Z","steps":["trace[632767561] 'process raft request' (duration: 158.349241ms)"],"step_count":1} Jul 15 10:59:08.643810 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:08.642207Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074196,"retry-timeout":"500ms"} Jul 15 10:59:09.859263 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:09.692433Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074196,"retry-timeout":"500ms"} Jul 15 10:59:09.859263 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:09.789271Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"2.676621694s","expected-duration":"1s"} Jul 15 10:59:09.859263 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:09.789915Z","caller":"traceutil/trace.go:171","msg":"trace[994579941] transaction","detail":"{read_only:false; number_of_response:0; response_revision:1306; }","duration":"2.677349334s","start":"2025-07-15T10:59:07.112485Z","end":"2025-07-15T10:59:09.789835Z","steps":["trace[994579941] 'process raft request' (duration: 2.677230016s)"],"step_count":1} Jul 15 10:59:09.859263 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:09.790513Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:07.112450Z","time spent":"2.677571999s","remote":"172.24.4.149:54498","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":29,"request content":"compare: success:> failure:<>"} Jul 15 10:59:09.859263 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:09.790998Z","caller":"traceutil/trace.go:171","msg":"trace[561324729] linearizableReadLoop","detail":"{readStateIndex:1568; appliedIndex:1568; }","duration":"1.649325259s","start":"2025-07-15T10:59:08.141612Z","end":"2025-07-15T10:59:09.790938Z","steps":["trace[561324729] 'read index received' (duration: 1.649318085s)","trace[561324729] 'applied index is now lower than readState.Index' (duration: 5.551µs)"],"step_count":2} Jul 15 10:59:09.861632 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:09.791240Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.649603652s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/tigera-operator/operator-lock\" ","response":"range_response_count:1 size:505"} Jul 15 10:59:09.861632 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:09.791375Z","caller":"traceutil/trace.go:171","msg":"trace[1375445310] range","detail":"{range_begin:/registry/leases/tigera-operator/operator-lock; range_end:; response_count:1; response_revision:1306; }","duration":"1.649803253s","start":"2025-07-15T10:59:08.141547Z","end":"2025-07-15T10:59:09.791350Z","steps":["trace[1375445310] 'agreement among raft nodes before linearized reading' (duration: 1.649544749s)"],"step_count":1} Jul 15 10:59:09.861632 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:09.791854Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:08.141509Z","time spent":"1.650216402s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":1,"response size":529,"request content":"key:\"/registry/leases/tigera-operator/operator-lock\" "} Jul 15 10:59:09.861632 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:09.851655Z","caller":"traceutil/trace.go:171","msg":"trace[1503213208] transaction","detail":"{read_only:false; response_revision:1307; number_of_response:1; }","duration":"141.031898ms","start":"2025-07-15T10:59:09.710515Z","end":"2025-07-15T10:59:09.851547Z","steps":["trace[1503213208] 'process raft request' (duration: 140.533717ms)"],"step_count":1} Jul 15 10:59:09.861632 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:09.852008Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"894.616131ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:09.862244 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:09.853295Z","caller":"traceutil/trace.go:171","msg":"trace[950842525] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1307; }","duration":"895.952854ms","start":"2025-07-15T10:59:08.957312Z","end":"2025-07-15T10:59:09.853265Z","steps":["trace[950842525] 'agreement among raft nodes before linearized reading' (duration: 894.264458ms)"],"step_count":1} Jul 15 10:59:09.862244 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:09.853449Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:08.957283Z","time spent":"896.06579ms","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:59:09.862244 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:09.852254Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.051917074s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-controller-manager\" ","response":"range_response_count:1 size:531"} Jul 15 10:59:09.862244 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:09.854411Z","caller":"traceutil/trace.go:171","msg":"trace[1153567191] range","detail":"{range_begin:/registry/leases/kube-system/kube-controller-manager; range_end:; response_count:1; response_revision:1307; }","duration":"1.054081359s","start":"2025-07-15T10:59:08.800277Z","end":"2025-07-15T10:59:09.854358Z","steps":["trace[1153567191] 'agreement among raft nodes before linearized reading' (duration: 1.051847402s)"],"step_count":1} Jul 15 10:59:09.862244 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:09.854506Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:08.800232Z","time spent":"1.054248938s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":54,"response count":1,"response size":555,"request content":"key:\"/registry/leases/kube-system/kube-controller-manager\" "} Jul 15 10:59:09.863171 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:09.852316Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.190734975s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:09.863171 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:09.854604Z","caller":"traceutil/trace.go:171","msg":"trace[439168336] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1307; }","duration":"1.193042744s","start":"2025-07-15T10:59:08.661536Z","end":"2025-07-15T10:59:09.854579Z","steps":["trace[439168336] 'agreement among raft nodes before linearized reading' (duration: 1.190760605s)"],"step_count":1} Jul 15 10:59:09.863171 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:09.854667Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:08.661508Z","time spent":"1.193144148s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:59:19.172260 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:19.171344Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074477,"retry-timeout":"500ms"} Jul 15 10:59:19.599787 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:19.402641Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"2.692162946s","expected-duration":"1s"} Jul 15 10:59:19.599787 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:19.403190Z","caller":"traceutil/trace.go:171","msg":"trace[1885197933] linearizableReadLoop","detail":"{readStateIndex:1620; appliedIndex:1619; }","duration":"1.834129117s","start":"2025-07-15T10:59:17.569024Z","end":"2025-07-15T10:59:19.403153Z","steps":["trace[1885197933] 'read index received' (duration: 1.833802643s)","trace[1885197933] 'applied index is now lower than readState.Index' (duration: 325.032µs)"],"step_count":2} Jul 15 10:59:19.599787 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:19.404117Z","caller":"traceutil/trace.go:171","msg":"trace[1174085649] transaction","detail":"{read_only:false; response_revision:1353; number_of_response:1; }","duration":"2.693818434s","start":"2025-07-15T10:59:16.710270Z","end":"2025-07-15T10:59:19.404088Z","steps":["trace[1174085649] 'process raft request' (duration: 2.692642694s)"],"step_count":1} Jul 15 10:59:19.599787 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:19.404359Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:16.710231Z","time spent":"2.693949717s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":517,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:19.599787 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:19.404551Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.835519538s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/csinodes/172.24.4.29\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:19.600784 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:19.404602Z","caller":"traceutil/trace.go:171","msg":"trace[1332009307] range","detail":"{range_begin:/registry/csinodes/172.24.4.29; range_end:; response_count:0; response_revision:1353; }","duration":"1.835615972s","start":"2025-07-15T10:59:17.568969Z","end":"2025-07-15T10:59:19.404585Z","steps":["trace[1332009307] 'agreement among raft nodes before linearized reading' (duration: 1.835540999s)"],"step_count":1} Jul 15 10:59:19.600784 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:19.404704Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:17.568940Z","time spent":"1.835726865s","remote":"172.24.4.149:33400","response type":"/etcdserverpb.KV/Range","request count":0,"request size":32,"response count":0,"response size":29,"request content":"key:\"/registry/csinodes/172.24.4.29\" "} Jul 15 10:59:19.600784 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:19.404809Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.835539907s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/172.24.4.29\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:19.600784 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:19.404848Z","caller":"traceutil/trace.go:171","msg":"trace[1403095821] range","detail":"{range_begin:/registry/minions/172.24.4.29; range_end:; response_count:0; response_revision:1353; }","duration":"1.835600844s","start":"2025-07-15T10:59:17.569233Z","end":"2025-07-15T10:59:19.404834Z","steps":["trace[1403095821] 'agreement among raft nodes before linearized reading' (duration: 1.835548574s)"],"step_count":1} Jul 15 10:59:19.600784 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:19.404911Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:17.569219Z","time spent":"1.835673633s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Range","request count":0,"request size":31,"response count":0,"response size":29,"request content":"key:\"/registry/minions/172.24.4.29\" "} Jul 15 10:59:19.601324 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:19.405155Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.835799625s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/calico-system/whisker-cf9979c5b-vb78x\" ","response":"range_response_count:1 size:6482"} Jul 15 10:59:19.601324 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:19.405253Z","caller":"traceutil/trace.go:171","msg":"trace[1473552706] range","detail":"{range_begin:/registry/pods/calico-system/whisker-cf9979c5b-vb78x; range_end:; response_count:1; response_revision:1353; }","duration":"1.83595921s","start":"2025-07-15T10:59:17.569277Z","end":"2025-07-15T10:59:19.405237Z","steps":["trace[1473552706] 'agreement among raft nodes before linearized reading' (duration: 1.835706697s)"],"step_count":1} Jul 15 10:59:19.601324 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:19.405315Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:17.569214Z","time spent":"1.836083679s","remote":"172.24.4.149:33206","response type":"/etcdserverpb.KV/Range","request count":0,"request size":54,"response count":1,"response size":6506,"request content":"key:\"/registry/pods/calico-system/whisker-cf9979c5b-vb78x\" "} Jul 15 10:59:22.021458 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:19.997360Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074484,"retry-timeout":"500ms"} Jul 15 10:59:22.021458 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:20.498003Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074484,"retry-timeout":"500ms"} Jul 15 10:59:22.021458 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:20.998906Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074484,"retry-timeout":"500ms"} Jul 15 10:59:22.021458 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:21.497047Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.999751427s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 10:59:22.021458 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:21.497147Z","caller":"traceutil/trace.go:171","msg":"trace[1217159111] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"1.999966197s","start":"2025-07-15T10:59:19.497157Z","end":"2025-07-15T10:59:21.497123Z","steps":["trace[1217159111] 'agreement among raft nodes before linearized reading' (duration: 1.999785221s)"],"step_count":1} Jul 15 10:59:22.021458 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:21.497259Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:19.497133Z","time spent":"2.000107527s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:59:22.021458 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:21.499545Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074484,"retry-timeout":"500ms"} Jul 15 10:59:22.033477 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:21.499645Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000918642s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 10:59:22.033477 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:21.499863Z","caller":"traceutil/trace.go:171","msg":"trace[708249554] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.001161106s","start":"2025-07-15T10:59:19.498685Z","end":"2025-07-15T10:59:21.499846Z","steps":["trace[708249554] 'agreement among raft nodes before linearized reading' (duration: 2.000939441s)"],"step_count":1} Jul 15 10:59:22.033477 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:21.499999Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:19.498652Z","time spent":"2.001327124s","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:59:22.033477 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.000475Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074484,"retry-timeout":"500ms"} Jul 15 10:59:22.069358 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.068505Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"2.582986089s","expected-duration":"1s"} Jul 15 10:59:22.092411 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.088876Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.604122217s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-node-lease/172.24.4.29\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:22.092411 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.089302Z","caller":"traceutil/trace.go:171","msg":"trace[636508781] range","detail":"{range_begin:/registry/leases/kube-node-lease/172.24.4.29; range_end:; response_count:0; response_revision:1353; }","duration":"2.604667301s","start":"2025-07-15T10:59:19.484609Z","end":"2025-07-15T10:59:22.089276Z","steps":["trace[636508781] 'range keys from in-memory index tree' (duration: 2.603693096s)"],"step_count":1} Jul 15 10:59:22.092411 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.089425Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:19.484575Z","time spent":"2.604776149s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":46,"response count":0,"response size":29,"request content":"key:\"/registry/leases/kube-node-lease/172.24.4.29\" "} Jul 15 10:59:22.092411 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.088976Z","caller":"traceutil/trace.go:171","msg":"trace[1524025650] transaction","detail":"{read_only:false; response_revision:1354; number_of_response:1; }","duration":"2.603635796s","start":"2025-07-15T10:59:19.485245Z","end":"2025-07-15T10:59:22.088881Z","steps":["trace[1524025650] 'process raft request' (duration: 2.583966818s)","trace[1524025650] 'compare' (duration: 18.308404ms)"],"step_count":2} Jul 15 10:59:22.094917 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.089646Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:19.485218Z","time spent":"2.604346889s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1940,"response count":0,"response size":40,"request content":"compare: success:> failure:<>"} Jul 15 10:59:22.110157 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.106513Z","caller":"traceutil/trace.go:171","msg":"trace[464355579] linearizableReadLoop","detail":"{readStateIndex:1625; appliedIndex:1620; }","duration":"2.609279995s","start":"2025-07-15T10:59:19.497202Z","end":"2025-07-15T10:59:22.106482Z","steps":["trace[464355579] 'read index received' (duration: 2.572026267s)","trace[464355579] 'applied index is now lower than readState.Index' (duration: 37.252485ms)"],"step_count":2} Jul 15 10:59:22.110157 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.106692Z","caller":"traceutil/trace.go:171","msg":"trace[716291299] transaction","detail":"{read_only:false; response_revision:1356; number_of_response:1; }","duration":"2.603896101s","start":"2025-07-15T10:59:19.502774Z","end":"2025-07-15T10:59:22.106670Z","steps":["trace[716291299] 'process raft request' (duration: 2.603396805s)"],"step_count":1} Jul 15 10:59:22.110157 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.106850Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:19.502752Z","time spent":"2.603984971s","remote":"172.24.4.149:33206","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":6729,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:22.110157 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.107075Z","caller":"traceutil/trace.go:171","msg":"trace[133661036] transaction","detail":"{read_only:false; response_revision:1357; number_of_response:1; }","duration":"2.599932876s","start":"2025-07-15T10:59:19.507099Z","end":"2025-07-15T10:59:22.107032Z","steps":["trace[133661036] 'process raft request' (duration: 2.599149737s)"],"step_count":1} Jul 15 10:59:22.110987 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.107160Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:19.507079Z","time spent":"2.600030113s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":490,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:22.110987 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.107264Z","caller":"traceutil/trace.go:171","msg":"trace[1529569617] transaction","detail":"{read_only:false; response_revision:1358; number_of_response:1; }","duration":"262.613794ms","start":"2025-07-15T10:59:21.844620Z","end":"2025-07-15T10:59:22.107234Z","steps":["trace[1529569617] 'process raft request' (duration: 261.743086ms)"],"step_count":1} Jul 15 10:59:22.110987 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.107421Z","caller":"traceutil/trace.go:171","msg":"trace[1847414115] transaction","detail":"{read_only:false; response_revision:1355; number_of_response:1; }","duration":"2.608125636s","start":"2025-07-15T10:59:19.499225Z","end":"2025-07-15T10:59:22.107351Z","steps":["trace[1847414115] 'process raft request' (duration: 2.60674916s)"],"step_count":1} Jul 15 10:59:22.110987 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.107547Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"470.968166ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/csidrivers/\" range_end:\"/registry/csidrivers0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:59:22.111508 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.107591Z","caller":"traceutil/trace.go:171","msg":"trace[1822391349] range","detail":"{range_begin:/registry/csidrivers/; range_end:/registry/csidrivers0; response_count:0; response_revision:1358; }","duration":"471.067196ms","start":"2025-07-15T10:59:21.636509Z","end":"2025-07-15T10:59:22.107576Z","steps":["trace[1822391349] 'agreement among raft nodes before linearized reading' (duration: 470.992252ms)"],"step_count":1} Jul 15 10:59:22.111508 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.107760Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:21.636482Z","time spent":"471.176505ms","remote":"172.24.4.149:33412","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":1,"response size":31,"request content":"key:\"/registry/csidrivers/\" range_end:\"/registry/csidrivers0\" count_only:true "} Jul 15 10:59:22.111508 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.107990Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.353657482s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/whiskers/\" range_end:\"/registry/operator.tigera.io/whiskers0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:59:22.111508 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.108035Z","caller":"traceutil/trace.go:171","msg":"trace[34527503] range","detail":"{range_begin:/registry/operator.tigera.io/whiskers/; range_end:/registry/operator.tigera.io/whiskers0; response_count:0; response_revision:1358; }","duration":"2.353766421s","start":"2025-07-15T10:59:19.754254Z","end":"2025-07-15T10:59:22.108021Z","steps":["trace[34527503] 'agreement among raft nodes before linearized reading' (duration: 2.353698521s)"],"step_count":1} Jul 15 10:59:22.111978 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.108038Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.169294013s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"range_response_count:1 size:886"} Jul 15 10:59:22.111978 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.108077Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:19.754217Z","time spent":"2.35384476s","remote":"172.24.4.149:54978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":82,"response count":1,"response size":31,"request content":"key:\"/registry/operator.tigera.io/whiskers/\" range_end:\"/registry/operator.tigera.io/whiskers0\" count_only:true "} Jul 15 10:59:22.111978 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.108097Z","caller":"traceutil/trace.go:171","msg":"trace[1780084358] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; response_count:1; response_revision:1358; }","duration":"2.169400365s","start":"2025-07-15T10:59:19.938679Z","end":"2025-07-15T10:59:22.108080Z","steps":["trace[1780084358] 'agreement among raft nodes before linearized reading' (duration: 2.169186234s)"],"step_count":1} Jul 15 10:59:22.111978 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.108156Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:19.938614Z","time spent":"2.16952831s","remote":"172.24.4.149:54994","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":1,"response size":910,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" "} Jul 15 10:59:22.111978 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.108189Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.609241863s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/172.24.4.29\" ","response":"range_response_count:1 size:1955"} Jul 15 10:59:22.112598 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.108228Z","caller":"traceutil/trace.go:171","msg":"trace[326911530] range","detail":"{range_begin:/registry/minions/172.24.4.29; range_end:; response_count:1; response_revision:1358; }","duration":"2.609299363s","start":"2025-07-15T10:59:19.498916Z","end":"2025-07-15T10:59:22.108216Z","steps":["trace[326911530] 'agreement among raft nodes before linearized reading' (duration: 2.609218829s)"],"step_count":1} Jul 15 10:59:22.112598 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.108267Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"590.888345ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:22.112598 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.108310Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:19.498910Z","time spent":"2.609386749s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Range","request count":0,"request size":31,"response count":1,"response size":1979,"request content":"key:\"/registry/minions/172.24.4.29\" "} Jul 15 10:59:22.112598 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.107601Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:19.499201Z","time spent":"2.608304528s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:22.113080 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.109013Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"601.759916ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678d1663474b\" ","response":"range_response_count:1 size:904"} Jul 15 10:59:22.113080 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.108314Z","caller":"traceutil/trace.go:171","msg":"trace[271011944] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1358; }","duration":"590.948891ms","start":"2025-07-15T10:59:21.517352Z","end":"2025-07-15T10:59:22.108301Z","steps":["trace[271011944] 'agreement among raft nodes before linearized reading' (duration: 590.892002ms)"],"step_count":1} Jul 15 10:59:22.113080 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.109295Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:21.517342Z","time spent":"591.902547ms","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:59:22.113080 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.107458Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.608487729s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/172.24.4.29\" ","response":"range_response_count:1 size:1955"} Jul 15 10:59:22.113080 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.109475Z","caller":"traceutil/trace.go:171","msg":"trace[1584296051] range","detail":"{range_begin:/registry/minions/172.24.4.29; range_end:; response_count:1; response_revision:1358; }","duration":"2.610583512s","start":"2025-07-15T10:59:19.498868Z","end":"2025-07-15T10:59:22.109452Z","steps":["trace[1584296051] 'agreement among raft nodes before linearized reading' (duration: 2.608462611s)"],"step_count":1} Jul 15 10:59:22.113590 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.109525Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:19.498857Z","time spent":"2.610652123s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Range","request count":0,"request size":31,"response count":1,"response size":1979,"request content":"key:\"/registry/minions/172.24.4.29\" "} Jul 15 10:59:22.113590 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.109221Z","caller":"traceutil/trace.go:171","msg":"trace[693489666] range","detail":"{range_begin:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678d1663474b; range_end:; response_count:1; response_revision:1358; }","duration":"601.972274ms","start":"2025-07-15T10:59:21.507228Z","end":"2025-07-15T10:59:22.109200Z","steps":["trace[693489666] 'agreement among raft nodes before linearized reading' (duration: 601.107578ms)"],"step_count":1} Jul 15 10:59:22.113590 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.110812Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.129027244s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 10:59:22.113590 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.110879Z","caller":"traceutil/trace.go:171","msg":"trace[1943209462] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1358; }","duration":"2.129145962s","start":"2025-07-15T10:59:19.981718Z","end":"2025-07-15T10:59:22.110864Z","steps":["trace[1943209462] 'agreement among raft nodes before linearized reading' (duration: 2.128944897s)"],"step_count":1} Jul 15 10:59:22.114026 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.110922Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:19.981688Z","time spent":"2.129220194s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 10:59:22.114026 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.110102Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:21.507148Z","time spent":"602.912224ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":97,"response count":1,"response size":928,"request content":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678d1663474b\" "} Jul 15 10:59:22.287076 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.284030Z","caller":"traceutil/trace.go:171","msg":"trace[1115864020] transaction","detail":"{read_only:false; response_revision:1361; number_of_response:1; }","duration":"124.208286ms","start":"2025-07-15T10:59:22.159781Z","end":"2025-07-15T10:59:22.283989Z","steps":["trace[1115864020] 'process raft request' (duration: 123.55779ms)"],"step_count":1} Jul 15 10:59:22.287076 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.284962Z","caller":"traceutil/trace.go:171","msg":"trace[1563292082] transaction","detail":"{read_only:false; response_revision:1362; number_of_response:1; }","duration":"125.026262ms","start":"2025-07-15T10:59:22.159905Z","end":"2025-07-15T10:59:22.284931Z","steps":["trace[1563292082] 'process raft request' (duration: 123.969839ms)"],"step_count":1} Jul 15 10:59:22.291223 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.286296Z","caller":"traceutil/trace.go:171","msg":"trace[1004851930] transaction","detail":"{read_only:false; response_revision:1363; number_of_response:1; }","duration":"122.700187ms","start":"2025-07-15T10:59:22.163565Z","end":"2025-07-15T10:59:22.286266Z","steps":["trace[1004851930] 'process raft request' (duration: 121.218731ms)"],"step_count":1} Jul 15 10:59:22.291223 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.288286Z","caller":"traceutil/trace.go:171","msg":"trace[1267046174] transaction","detail":"{read_only:false; response_revision:1364; number_of_response:1; }","duration":"122.627508ms","start":"2025-07-15T10:59:22.165639Z","end":"2025-07-15T10:59:22.288266Z","steps":["trace[1267046174] 'process raft request' (duration: 120.359987ms)"],"step_count":1} Jul 15 10:59:22.291223 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.288648Z","caller":"traceutil/trace.go:171","msg":"trace[965089223] transaction","detail":"{read_only:false; response_revision:1365; number_of_response:1; }","duration":"121.415688ms","start":"2025-07-15T10:59:22.167214Z","end":"2025-07-15T10:59:22.288630Z","steps":["trace[965089223] 'process raft request' (duration: 119.49966ms)"],"step_count":1} Jul 15 10:59:22.291223 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.288966Z","caller":"traceutil/trace.go:171","msg":"trace[667938872] transaction","detail":"{read_only:false; response_revision:1367; number_of_response:1; }","duration":"116.709056ms","start":"2025-07-15T10:59:22.172237Z","end":"2025-07-15T10:59:22.288946Z","steps":["trace[667938872] 'process raft request' (duration: 114.745897ms)"],"step_count":1} Jul 15 10:59:22.291223 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.289235Z","caller":"traceutil/trace.go:171","msg":"trace[1743100895] transaction","detail":"{read_only:false; response_revision:1366; number_of_response:1; }","duration":"119.973908ms","start":"2025-07-15T10:59:22.169239Z","end":"2025-07-15T10:59:22.289213Z","steps":["trace[1743100895] 'process raft request' (duration: 117.672431ms)"],"step_count":1} Jul 15 10:59:22.292151 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.289476Z","caller":"traceutil/trace.go:171","msg":"trace[1832543129] linearizableReadLoop","detail":"{readStateIndex:1635; appliedIndex:1627; }","duration":"106.674104ms","start":"2025-07-15T10:59:22.180555Z","end":"2025-07-15T10:59:22.287229Z","steps":["trace[1832543129] 'read index received' (duration: 72.990642ms)","trace[1832543129] 'applied index is now lower than readState.Index' (duration: 33.681869ms)"],"step_count":2} Jul 15 10:59:22.292151 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.289813Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"128.450306ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76\" ","response":"range_response_count:1 size:905"} Jul 15 10:59:22.292151 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.289922Z","caller":"traceutil/trace.go:171","msg":"trace[397892246] range","detail":"{range_begin:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76; range_end:; response_count:1; response_revision:1367; }","duration":"128.607056ms","start":"2025-07-15T10:59:22.161263Z","end":"2025-07-15T10:59:22.289870Z","steps":["trace[397892246] 'agreement among raft nodes before linearized reading' (duration: 128.340606ms)"],"step_count":1} Jul 15 10:59:22.292151 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.290093Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"136.08308ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:22.292643 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.290144Z","caller":"traceutil/trace.go:171","msg":"trace[554762288] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1367; }","duration":"136.16123ms","start":"2025-07-15T10:59:22.153970Z","end":"2025-07-15T10:59:22.290131Z","steps":["trace[554762288] 'agreement among raft nodes before linearized reading' (duration: 136.091877ms)"],"step_count":1} Jul 15 10:59:22.292643 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.290217Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"135.841659ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/172.24.4.29\" ","response":"range_response_count:1 size:2127"} Jul 15 10:59:22.292643 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.290359Z","caller":"traceutil/trace.go:171","msg":"trace[463597039] range","detail":"{range_begin:/registry/minions/172.24.4.29; range_end:; response_count:1; response_revision:1367; }","duration":"136.127165ms","start":"2025-07-15T10:59:22.154211Z","end":"2025-07-15T10:59:22.290338Z","steps":["trace[463597039] 'agreement among raft nodes before linearized reading' (duration: 135.619763ms)"],"step_count":1} Jul 15 10:59:22.298770 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.298083Z","caller":"traceutil/trace.go:171","msg":"trace[1490277095] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1367; }","duration":"111.565869ms","start":"2025-07-15T10:59:22.186453Z","end":"2025-07-15T10:59:22.298019Z","steps":["trace[1490277095] 'process raft request' (duration: 111.026777ms)"],"step_count":1} Jul 15 10:59:22.299497 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.299068Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"111.80637ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/kube-system/ttl-controller\" ","response":"range_response_count:1 size:193"} Jul 15 10:59:22.299497 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:22.299209Z","caller":"traceutil/trace.go:171","msg":"trace[855058778] range","detail":"{range_begin:/registry/serviceaccounts/kube-system/ttl-controller; range_end:; response_count:1; response_revision:1368; }","duration":"111.993387ms","start":"2025-07-15T10:59:22.187196Z","end":"2025-07-15T10:59:22.299189Z","steps":["trace[855058778] 'agreement among raft nodes before linearized reading' (duration: 111.421582ms)"],"step_count":1} Jul 15 10:59:23.387318 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:22.851929Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074509,"retry-timeout":"500ms"} Jul 15 10:59:23.390365 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:23.386996Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074509,"retry-timeout":"500ms"} Jul 15 10:59:24.592187 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:23.888164Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074509,"retry-timeout":"500ms"} Jul 15 10:59:24.592187 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.388810Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074509,"retry-timeout":"500ms"} Jul 15 10:59:24.614933 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.614237Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.262623276s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/tiers/default\" ","response":"","error":"context canceled"} Jul 15 10:59:24.614933 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.614350Z","caller":"traceutil/trace.go:171","msg":"trace[1853420967] range","detail":"{range_begin:/registry/crd.projectcalico.org/tiers/default; range_end:; }","duration":"2.262844379s","start":"2025-07-15T10:59:22.351485Z","end":"2025-07-15T10:59:24.614330Z","steps":["trace[1853420967] 'agreement among raft nodes before linearized reading' (duration: 2.26267243s)"],"step_count":1} Jul 15 10:59:24.614933 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.614432Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.351453Z","time spent":"2.262963868s","remote":"172.24.4.149:54498","response type":"/etcdserverpb.KV/Range","request count":0,"request size":47,"response count":0,"response size":0,"request content":"key:\"/registry/crd.projectcalico.org/tiers/default\" "} Jul 15 10:59:24.617934 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.617149Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"2.272394827s","expected-duration":"1s"} Jul 15 10:59:24.630536 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.629075Z","caller":"traceutil/trace.go:171","msg":"trace[648041817] transaction","detail":"{read_only:false; response_revision:1374; number_of_response:1; }","duration":"2.285098938s","start":"2025-07-15T10:59:22.343953Z","end":"2025-07-15T10:59:24.629052Z","steps":["trace[648041817] 'process raft request' (duration: 2.285010137s)"],"step_count":1} Jul 15 10:59:24.630536 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.629237Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.343918Z","time spent":"2.285236782s","remote":"172.24.4.149:33488","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":4913,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:24.630536 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.629638Z","caller":"traceutil/trace.go:171","msg":"trace[591480157] transaction","detail":"{read_only:false; response_revision:1372; number_of_response:1; }","duration":"2.301913282s","start":"2025-07-15T10:59:22.327712Z","end":"2025-07-15T10:59:24.629625Z","steps":["trace[591480157] 'process raft request' (duration: 2.290524039s)","trace[591480157] 'compare' (duration: 10.389288ms)"],"step_count":2} Jul 15 10:59:24.631934 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.629706Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.327690Z","time spent":"2.301976071s","remote":"172.24.4.149:33462","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":5093,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:24.631934 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.629716Z","caller":"traceutil/trace.go:171","msg":"trace[831513967] transaction","detail":"{read_only:false; response_revision:1373; number_of_response:1; }","duration":"2.295942991s","start":"2025-07-15T10:59:22.333741Z","end":"2025-07-15T10:59:24.629684Z","steps":["trace[831513967] 'process raft request' (duration: 2.29507548s)"],"step_count":1} Jul 15 10:59:24.631934 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.630102Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.333720Z","time spent":"2.296163373s","remote":"172.24.4.149:33206","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":11963,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:24.646298 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.644323Z","caller":"traceutil/trace.go:171","msg":"trace[1285574866] transaction","detail":"{read_only:false; response_revision:1376; number_of_response:1; }","duration":"2.291603631s","start":"2025-07-15T10:59:22.352682Z","end":"2025-07-15T10:59:24.644285Z","steps":["trace[1285574866] 'process raft request' (duration: 2.291412705s)"],"step_count":1} Jul 15 10:59:24.646298 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.644632Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.352661Z","time spent":"2.291854301s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":532,"response count":0,"response size":40,"request content":"compare: success:> failure:<>"} Jul 15 10:59:24.646298 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.644818Z","caller":"traceutil/trace.go:171","msg":"trace[1113464899] transaction","detail":"{read_only:false; response_revision:1375; number_of_response:1; }","duration":"2.293557403s","start":"2025-07-15T10:59:22.351242Z","end":"2025-07-15T10:59:24.644799Z","steps":["trace[1113464899] 'process raft request' (duration: 2.292471393s)"],"step_count":1} Jul 15 10:59:24.646298 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.644960Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.351215Z","time spent":"2.293667383s","remote":"172.24.4.149:33488","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":11425,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:24.647084 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.645914Z","caller":"traceutil/trace.go:171","msg":"trace[421168537] transaction","detail":"{read_only:false; response_revision:1377; number_of_response:1; }","duration":"2.291681161s","start":"2025-07-15T10:59:22.354139Z","end":"2025-07-15T10:59:24.645820Z","steps":["trace[421168537] 'process raft request' (duration: 2.290048955s)"],"step_count":1} Jul 15 10:59:24.647744 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.647296Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.354121Z","time spent":"2.291950597s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":680,"response count":0,"response size":40,"request content":"compare: success:> failure:<>"} Jul 15 10:59:24.648798 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.648433Z","caller":"traceutil/trace.go:171","msg":"trace[623360432] transaction","detail":"{read_only:false; response_revision:1381; number_of_response:1; }","duration":"2.039551302s","start":"2025-07-15T10:59:22.608822Z","end":"2025-07-15T10:59:24.648374Z","steps":["trace[623360432] 'process raft request' (duration: 2.038240542s)"],"step_count":1} Jul 15 10:59:24.649408 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.649044Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.608791Z","time spent":"2.039770202s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":3779,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:24.650371 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.647707Z","caller":"traceutil/trace.go:171","msg":"trace[461213694] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1379; }","duration":"2.23221216s","start":"2025-07-15T10:59:22.415440Z","end":"2025-07-15T10:59:24.647652Z","steps":["trace[461213694] 'process raft request' (duration: 2.231184852s)"],"step_count":1} Jul 15 10:59:24.650775 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.650222Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.415345Z","time spent":"2.234810396s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":33,"response count":0,"response size":2484,"request content":"compare: success:> failure: >"} Jul 15 10:59:24.651570 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.649013Z","caller":"traceutil/trace.go:171","msg":"trace[1972527319] transaction","detail":"{read_only:false; response_revision:1378; number_of_response:1; }","duration":"2.294368117s","start":"2025-07-15T10:59:22.354620Z","end":"2025-07-15T10:59:24.648988Z","steps":["trace[1972527319] 'process raft request' (duration: 2.29058953s)"],"step_count":1} Jul 15 10:59:24.653900 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.648205Z","caller":"traceutil/trace.go:171","msg":"trace[1549583914] transaction","detail":"{read_only:false; response_revision:1382; number_of_response:1; }","duration":"582.012718ms","start":"2025-07-15T10:59:24.066112Z","end":"2025-07-15T10:59:24.648125Z","steps":["trace[1549583914] 'process raft request' (duration: 581.098157ms)"],"step_count":1} Jul 15 10:59:24.653900 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.648283Z","caller":"traceutil/trace.go:171","msg":"trace[1750826993] transaction","detail":"{read_only:false; response_revision:1380; number_of_response:1; }","duration":"2.055603146s","start":"2025-07-15T10:59:22.592666Z","end":"2025-07-15T10:59:24.648270Z","steps":["trace[1750826993] 'process raft request' (duration: 2.054134955s)"],"step_count":1} Jul 15 10:59:24.653900 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.649468Z","caller":"traceutil/trace.go:171","msg":"trace[703963182] transaction","detail":"{read_only:false; response_revision:1379; number_of_response:1; }","duration":"2.291365779s","start":"2025-07-15T10:59:22.357994Z","end":"2025-07-15T10:59:24.649360Z","steps":["trace[703963182] 'process raft request' (duration: 2.287650454s)"],"step_count":1} Jul 15 10:59:24.653900 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.651273Z","caller":"traceutil/trace.go:171","msg":"trace[1218946240] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1382; }","duration":"576.433092ms","start":"2025-07-15T10:59:24.074803Z","end":"2025-07-15T10:59:24.651236Z","steps":["trace[1218946240] 'process raft request' (duration: 572.557209ms)"],"step_count":1} Jul 15 10:59:24.654603 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.652149Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.074783Z","time spent":"577.269455ms","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":33,"response count":0,"response size":2484,"request content":"compare: success:> failure: >"} Jul 15 10:59:24.654603 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.652094Z","caller":"traceutil/trace.go:171","msg":"trace[1637526211] transaction","detail":"{read_only:false; response_revision:1383; number_of_response:1; }","duration":"492.382365ms","start":"2025-07-15T10:59:24.159639Z","end":"2025-07-15T10:59:24.652021Z","steps":["trace[1637526211] 'process raft request' (duration: 490.314164ms)"],"step_count":1} Jul 15 10:59:24.654603 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.652356Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.354602Z","time spent":"2.29768565s","remote":"172.24.4.149:33206","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":5313,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:24.655008 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.652541Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.066081Z","time spent":"586.392958ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":664,"response count":0,"response size":40,"request content":"compare: success:> failure:<>"} Jul 15 10:59:24.655008 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.652488Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.159617Z","time spent":"492.627003ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":516,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:24.655008 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.652650Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.592623Z","time spent":"2.059986692s","remote":"172.24.4.149:54882","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1466,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:24.655532 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.652770Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.357966Z","time spent":"2.294766742s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":2434,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:24.656406 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.655653Z","caller":"traceutil/trace.go:171","msg":"trace[1680226063] linearizableReadLoop","detail":"{readStateIndex:1655; appliedIndex:1641; }","duration":"2.303692914s","start":"2025-07-15T10:59:22.351545Z","end":"2025-07-15T10:59:24.655238Z","steps":["trace[1680226063] 'read index received' (duration: 2.266624279s)","trace[1680226063] 'applied index is now lower than readState.Index' (duration: 37.067303ms)"],"step_count":2} Jul 15 10:59:24.658458 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.657929Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.546785656s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/statefulsets/\" range_end:\"/registry/statefulsets0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:59:24.658458 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.658010Z","caller":"traceutil/trace.go:171","msg":"trace[753866811] range","detail":"{range_begin:/registry/statefulsets/; range_end:/registry/statefulsets0; response_count:0; response_revision:1386; }","duration":"1.546942938s","start":"2025-07-15T10:59:23.111049Z","end":"2025-07-15T10:59:24.657992Z","steps":["trace[753866811] 'agreement among raft nodes before linearized reading' (duration: 1.546398065s)"],"step_count":1} Jul 15 10:59:24.658925 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.658062Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:23.111022Z","time spent":"1.547022439s","remote":"172.24.4.149:33472","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":0,"response size":29,"request content":"key:\"/registry/statefulsets/\" range_end:\"/registry/statefulsets0\" count_only:true "} Jul 15 10:59:24.659488 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.658884Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.504723129s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:24.659488 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.658954Z","caller":"traceutil/trace.go:171","msg":"trace[2069849303] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1386; }","duration":"1.504894277s","start":"2025-07-15T10:59:23.154043Z","end":"2025-07-15T10:59:24.658938Z","steps":["trace[2069849303] 'agreement among raft nodes before linearized reading' (duration: 1.504812641s)"],"step_count":1} Jul 15 10:59:24.659488 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.658996Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:23.154021Z","time spent":"1.504961266s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:59:24.660756 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.659246Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.104779749s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 10:59:24.660756 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.659376Z","caller":"traceutil/trace.go:171","msg":"trace[646626626] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1386; }","duration":"2.104992977s","start":"2025-07-15T10:59:22.554356Z","end":"2025-07-15T10:59:24.659349Z","steps":["trace[646626626] 'agreement among raft nodes before linearized reading' (duration: 2.104796551s)"],"step_count":1} Jul 15 10:59:24.660756 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.659732Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.554336Z","time spent":"2.105372504s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":3818,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:59:24.660756 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.660061Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.070318789s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 10:59:24.660756 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.660348Z","caller":"traceutil/trace.go:171","msg":"trace[749029892] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1386; }","duration":"2.070639703s","start":"2025-07-15T10:59:22.589688Z","end":"2025-07-15T10:59:24.660328Z","steps":["trace[749029892] 'agreement among raft nodes before linearized reading' (duration: 2.070233745s)"],"step_count":1} Jul 15 10:59:24.663242 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.660737Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.306457811s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/endpoints/default/kubernetes\" ","response":"range_response_count:1 size:420"} Jul 15 10:59:24.663242 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.660805Z","caller":"traceutil/trace.go:171","msg":"trace[186712016] range","detail":"{range_begin:/registry/services/endpoints/default/kubernetes; range_end:; response_count:1; response_revision:1386; }","duration":"2.306565448s","start":"2025-07-15T10:59:22.354224Z","end":"2025-07-15T10:59:24.660789Z","steps":["trace[186712016] 'agreement among raft nodes before linearized reading' (duration: 2.30280747s)"],"step_count":1} Jul 15 10:59:24.663242 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.660851Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.354207Z","time spent":"2.306630752s","remote":"172.24.4.149:33194","response type":"/etcdserverpb.KV/Range","request count":0,"request size":49,"response count":1,"response size":444,"request content":"key:\"/registry/services/endpoints/default/kubernetes\" "} Jul 15 10:59:24.663242 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.661055Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.589667Z","time spent":"2.070784761s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":3818,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:59:24.663242 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.661518Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.107015681s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 10:59:24.663941 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.661583Z","caller":"traceutil/trace.go:171","msg":"trace[268759125] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1386; }","duration":"2.107109811s","start":"2025-07-15T10:59:22.554458Z","end":"2025-07-15T10:59:24.661568Z","steps":["trace[268759125] 'agreement among raft nodes before linearized reading' (duration: 2.106975593s)"],"step_count":1} Jul 15 10:59:24.663941 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.661630Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.554439Z","time spent":"2.107176448s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":3818,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:59:24.663941 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.662302Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.11570012s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/whiskers/default\" ","response":"range_response_count:1 size:948"} Jul 15 10:59:24.664826 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.662368Z","caller":"traceutil/trace.go:171","msg":"trace[815870155] range","detail":"{range_begin:/registry/operator.tigera.io/whiskers/default; range_end:; response_count:1; response_revision:1386; }","duration":"2.115799309s","start":"2025-07-15T10:59:22.546553Z","end":"2025-07-15T10:59:24.662352Z","steps":["trace[815870155] 'agreement among raft nodes before linearized reading' (duration: 2.115488563s)"],"step_count":1} Jul 15 10:59:24.664826 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.662906Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.546535Z","time spent":"2.116338232s","remote":"172.24.4.149:54978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":47,"response count":1,"response size":972,"request content":"key:\"/registry/operator.tigera.io/whiskers/default\" "} Jul 15 10:59:24.664826 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.663195Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.121926161s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/tigera-operator/tigera-ca-private\" ","response":"range_response_count:1 size:3369"} Jul 15 10:59:24.664826 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.663262Z","caller":"traceutil/trace.go:171","msg":"trace[1826151793] range","detail":"{range_begin:/registry/secrets/tigera-operator/tigera-ca-private; range_end:; response_count:1; response_revision:1386; }","duration":"2.122036893s","start":"2025-07-15T10:59:22.541205Z","end":"2025-07-15T10:59:24.663242Z","steps":["trace[1826151793] 'agreement among raft nodes before linearized reading' (duration: 2.121854013s)"],"step_count":1} Jul 15 10:59:24.665271 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.663315Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.541175Z","time spent":"2.122122887s","remote":"172.24.4.149:33122","response type":"/etcdserverpb.KV/Range","request count":0,"request size":53,"response count":1,"response size":3393,"request content":"key:\"/registry/secrets/tigera-operator/tigera-ca-private\" "} Jul 15 10:59:24.665271 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.663565Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.13472238s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/tigera-csr-creator\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:24.665271 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:24.663622Z","caller":"traceutil/trace.go:171","msg":"trace[1181711116] range","detail":"{range_begin:/registry/clusterroles/tigera-csr-creator; range_end:; response_count:0; response_revision:1386; }","duration":"2.134832181s","start":"2025-07-15T10:59:22.528772Z","end":"2025-07-15T10:59:24.663604Z","steps":["trace[1181711116] 'agreement among raft nodes before linearized reading' (duration: 2.134736147s)"],"step_count":1} Jul 15 10:59:24.665271 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:24.663671Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:22.528741Z","time spent":"2.134913617s","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":43,"response count":0,"response size":29,"request content":"key:\"/registry/clusterroles/tigera-csr-creator\" "} Jul 15 10:59:25.244172 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:25.243540Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074532,"retry-timeout":"500ms"} Jul 15 10:59:25.832445 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:25.831653Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074532,"retry-timeout":"500ms"} Jul 15 10:59:25.853050 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:25.852100Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"1.12563771s","expected-duration":"1s"} Jul 15 10:59:25.956480 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:25.953410Z","caller":"traceutil/trace.go:171","msg":"trace[1249747531] transaction","detail":"{read_only:false; response_revision:1389; number_of_response:1; }","duration":"1.227017274s","start":"2025-07-15T10:59:24.726179Z","end":"2025-07-15T10:59:25.953196Z","steps":["trace[1249747531] 'process raft request' (duration: 1.126920116s)","trace[1249747531] 'compare' (duration: 99.223564ms)"],"step_count":2} Jul 15 10:59:25.959138 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:25.957895Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.726144Z","time spent":"1.22872789s","remote":"172.24.4.149:33488","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":4913,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:26.518568 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.515103Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"323.069866ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure:<>>","response":"size:16"} Jul 15 10:59:26.518568 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.515713Z","caller":"traceutil/trace.go:171","msg":"trace[190169819] transaction","detail":"{read_only:false; response_revision:1391; number_of_response:1; }","duration":"1.770199657s","start":"2025-07-15T10:59:24.745496Z","end":"2025-07-15T10:59:26.515696Z","steps":["trace[190169819] 'process raft request' (duration: 1.769736391s)"],"step_count":1} Jul 15 10:59:26.518568 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.515831Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.745470Z","time spent":"1.770314949s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":6676,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:26.518568 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.516096Z","caller":"traceutil/trace.go:171","msg":"trace[766537272] transaction","detail":"{read_only:false; response_revision:1392; number_of_response:1; }","duration":"1.768318647s","start":"2025-07-15T10:59:24.747767Z","end":"2025-07-15T10:59:26.516085Z","steps":["trace[766537272] 'process raft request' (duration: 1.767572708s)"],"step_count":1} Jul 15 10:59:26.521512 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.516168Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.747743Z","time spent":"1.768401213s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":696,"response count":0,"response size":40,"request content":"compare: success:> failure:<>"} Jul 15 10:59:26.521512 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.516311Z","caller":"traceutil/trace.go:171","msg":"trace[1026089017] linearizableReadLoop","detail":"{readStateIndex:1664; appliedIndex:1660; }","duration":"1.773335303s","start":"2025-07-15T10:59:24.742925Z","end":"2025-07-15T10:59:26.516261Z","steps":["trace[1026089017] 'read index received' (duration: 1.110711562s)","trace[1026089017] 'applied index is now lower than readState.Index' (duration: 662.62281ms)"],"step_count":2} Jul 15 10:59:26.521512 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.516297Z","caller":"traceutil/trace.go:171","msg":"trace[1138424749] transaction","detail":"{read_only:false; response_revision:1390; number_of_response:1; }","duration":"1.773165748s","start":"2025-07-15T10:59:24.743020Z","end":"2025-07-15T10:59:26.516186Z","steps":["trace[1138424749] 'process raft request' (duration: 1.448950637s)","trace[1138424749] 'compare' (duration: 322.886676ms)"],"step_count":2} Jul 15 10:59:26.522986 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.516525Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.742986Z","time spent":"1.773434324s","remote":"172.24.4.149:33532","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":734,"response count":0,"response size":40,"request content":"compare: success:> failure:<>"} Jul 15 10:59:26.522986 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.517126Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.770914126s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/limitranges\" limit:1 ","response":"range_response_count:0 size:5"} Jul 15 10:59:26.522986 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.517161Z","caller":"traceutil/trace.go:171","msg":"trace[705016724] range","detail":"{range_begin:/registry/limitranges; range_end:; response_count:0; response_revision:1394; }","duration":"1.770991676s","start":"2025-07-15T10:59:24.746157Z","end":"2025-07-15T10:59:26.517148Z","steps":["trace[705016724] 'agreement among raft nodes before linearized reading' (duration: 1.770903447s)"],"step_count":1} Jul 15 10:59:26.522986 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.517201Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.746136Z","time spent":"1.771042232s","remote":"172.24.4.149:33184","response type":"/etcdserverpb.KV/Range","request count":0,"request size":25,"response count":0,"response size":29,"request content":"key:\"/registry/limitranges\" limit:1 "} Jul 15 10:59:26.523742 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.517264Z","caller":"traceutil/trace.go:171","msg":"trace[1248510697] transaction","detail":"{read_only:false; response_revision:1393; number_of_response:1; }","duration":"663.897732ms","start":"2025-07-15T10:59:25.853360Z","end":"2025-07-15T10:59:26.517257Z","steps":["trace[1248510697] 'process raft request' (duration: 662.206682ms)"],"step_count":1} Jul 15 10:59:26.523742 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.517255Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.774308829s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/\" range_end:\"/registry/masterleases0\" ","response":"range_response_count:1 size:131"} Jul 15 10:59:26.523742 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.517306Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:25.853297Z","time spent":"663.983415ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":699,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:26.523742 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.517313Z","caller":"traceutil/trace.go:171","msg":"trace[1267753284] range","detail":"{range_begin:/registry/masterleases/; range_end:/registry/masterleases0; response_count:1; response_revision:1393; }","duration":"1.774421575s","start":"2025-07-15T10:59:24.742875Z","end":"2025-07-15T10:59:26.517297Z","steps":["trace[1267753284] 'agreement among raft nodes before linearized reading' (duration: 1.773531049s)"],"step_count":1} Jul 15 10:59:26.524552 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.517579Z","caller":"traceutil/trace.go:171","msg":"trace[2127593024] transaction","detail":"{read_only:false; response_revision:1394; number_of_response:1; }","duration":"544.410663ms","start":"2025-07-15T10:59:25.973156Z","end":"2025-07-15T10:59:26.517567Z","steps":["trace[2127593024] 'process raft request' (duration: 543.386932ms)"],"step_count":1} Jul 15 10:59:26.524552 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.517660Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:25.973082Z","time spent":"544.522597ms","remote":"172.24.4.149:33488","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":11425,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:26.524552 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.517707Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"558.16035ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:26.524552 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.517724Z","caller":"traceutil/trace.go:171","msg":"trace[950299841] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1394; }","duration":"558.578201ms","start":"2025-07-15T10:59:25.959141Z","end":"2025-07-15T10:59:26.517719Z","steps":["trace[950299841] 'agreement among raft nodes before linearized reading' (duration: 558.553053ms)"],"step_count":1} Jul 15 10:59:26.524950 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.517741Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:25.958944Z","time spent":"558.791961ms","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:59:26.524950 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.517835Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"668.643622ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/\" range_end:\"/registry/serviceaccounts0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:59:26.524950 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.517855Z","caller":"traceutil/trace.go:171","msg":"trace[774813735] range","detail":"{range_begin:/registry/serviceaccounts/; range_end:/registry/serviceaccounts0; response_count:0; response_revision:1394; }","duration":"668.755367ms","start":"2025-07-15T10:59:25.849093Z","end":"2025-07-15T10:59:26.517849Z","steps":["trace[774813735] 'agreement among raft nodes before linearized reading' (duration: 668.67316ms)"],"step_count":1} Jul 15 10:59:26.524950 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.517886Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:25.848901Z","time spent":"668.964357ms","remote":"172.24.4.149:33226","response type":"/etcdserverpb.KV/Range","request count":0,"request size":58,"response count":51,"response size":31,"request content":"key:\"/registry/serviceaccounts/\" range_end:\"/registry/serviceaccounts0\" count_only:true "} Jul 15 10:59:26.524950 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.517931Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.248647901s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/cronjobs/\" range_end:\"/registry/cronjobs0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:59:26.525546 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.517958Z","caller":"traceutil/trace.go:171","msg":"trace[367368051] range","detail":"{range_begin:/registry/cronjobs/; range_end:/registry/cronjobs0; response_count:0; response_revision:1394; }","duration":"1.248708798s","start":"2025-07-15T10:59:25.269232Z","end":"2025-07-15T10:59:26.517941Z","steps":["trace[367368051] 'agreement among raft nodes before linearized reading' (duration: 1.248685203s)"],"step_count":1} Jul 15 10:59:26.525546 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.517980Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:25.269203Z","time spent":"1.248769796s","remote":"172.24.4.149:33276","response type":"/etcdserverpb.KV/Range","request count":0,"request size":44,"response count":0,"response size":29,"request content":"key:\"/registry/cronjobs/\" range_end:\"/registry/cronjobs0\" count_only:true "} Jul 15 10:59:26.525546 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.517365Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.742850Z","time spent":"1.774499534s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":50,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/\" range_end:\"/registry/masterleases0\" "} Jul 15 10:59:26.525546 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.518773Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.727215605s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 10:59:26.526072 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.518808Z","caller":"traceutil/trace.go:171","msg":"trace[1436385769] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1394; }","duration":"1.727442361s","start":"2025-07-15T10:59:24.791357Z","end":"2025-07-15T10:59:26.518799Z","steps":["trace[1436385769] 'agreement among raft nodes before linearized reading' (duration: 1.727256164s)"],"step_count":1} Jul 15 10:59:26.526072 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.518889Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.791325Z","time spent":"1.727554725s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":3818,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:59:26.526072 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.519084Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.544368122s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/ingress/\" range_end:\"/registry/ingress0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:59:26.526072 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.519176Z","caller":"traceutil/trace.go:171","msg":"trace[677081280] range","detail":"{range_begin:/registry/ingress/; range_end:/registry/ingress0; response_count:0; response_revision:1394; }","duration":"1.544501707s","start":"2025-07-15T10:59:24.974665Z","end":"2025-07-15T10:59:26.519167Z","steps":["trace[677081280] 'agreement among raft nodes before linearized reading' (duration: 1.544396455s)"],"step_count":1} Jul 15 10:59:26.526516 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.519269Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.974641Z","time spent":"1.544614833s","remote":"172.24.4.149:33320","response type":"/etcdserverpb.KV/Range","request count":0,"request size":42,"response count":0,"response size":29,"request content":"key:\"/registry/ingress/\" range_end:\"/registry/ingress0\" count_only:true "} Jul 15 10:59:26.526516 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.519724Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.650376774s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 10:59:26.526516 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.519788Z","caller":"traceutil/trace.go:171","msg":"trace[1617097452] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1394; }","duration":"1.650451707s","start":"2025-07-15T10:59:24.869328Z","end":"2025-07-15T10:59:26.519779Z","steps":["trace[1617097452] 'agreement among raft nodes before linearized reading' (duration: 1.65029131s)"],"step_count":1} Jul 15 10:59:26.526516 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.519895Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.869322Z","time spent":"1.650481144s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":3818,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:59:26.526516 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.520400Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.651116551s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/172.24.4.29\" ","response":"range_response_count:1 size:2638"} Jul 15 10:59:26.526962 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.520505Z","caller":"traceutil/trace.go:171","msg":"trace[1133550732] range","detail":"{range_begin:/registry/minions/172.24.4.29; range_end:; response_count:1; response_revision:1394; }","duration":"1.651157851s","start":"2025-07-15T10:59:24.869272Z","end":"2025-07-15T10:59:26.520430Z","steps":["trace[1133550732] 'agreement among raft nodes before linearized reading' (duration: 1.65093344s)"],"step_count":1} Jul 15 10:59:26.526962 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.520588Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.869220Z","time spent":"1.651355979s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Range","request count":0,"request size":31,"response count":1,"response size":2662,"request content":"key:\"/registry/minions/172.24.4.29\" "} Jul 15 10:59:26.526962 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.520871Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.726723415s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/configmaps/calico-system/goldmane-ca-bundle\" ","response":"range_response_count:1 size:1959"} Jul 15 10:59:26.526962 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.520903Z","caller":"traceutil/trace.go:171","msg":"trace[1414569425] range","detail":"{range_begin:/registry/configmaps/calico-system/goldmane-ca-bundle; range_end:; response_count:1; response_revision:1394; }","duration":"1.726780924s","start":"2025-07-15T10:59:24.794115Z","end":"2025-07-15T10:59:26.520896Z","steps":["trace[1414569425] 'agreement among raft nodes before linearized reading' (duration: 1.726606801s)"],"step_count":1} Jul 15 10:59:26.526962 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.520979Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.794099Z","time spent":"1.726869132s","remote":"172.24.4.149:33128","response type":"/etcdserverpb.KV/Range","request count":0,"request size":55,"response count":1,"response size":1983,"request content":"key:\"/registry/configmaps/calico-system/goldmane-ca-bundle\" "} Jul 15 10:59:26.527519 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.521035Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.72955091s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/tigera-csr-creator\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:26.527519 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.521970Z","caller":"traceutil/trace.go:171","msg":"trace[266555232] range","detail":"{range_begin:/registry/clusterroles/tigera-csr-creator; range_end:; response_count:0; response_revision:1394; }","duration":"1.730634345s","start":"2025-07-15T10:59:24.791300Z","end":"2025-07-15T10:59:26.521935Z","steps":["trace[266555232] 'agreement among raft nodes before linearized reading' (duration: 1.729526593s)"],"step_count":1} Jul 15 10:59:26.527519 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.522064Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.791286Z","time spent":"1.730753193s","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":43,"response count":0,"response size":29,"request content":"key:\"/registry/clusterroles/tigera-csr-creator\" "} Jul 15 10:59:26.527519 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.521287Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.728346554s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/tigera-operator/tigera-ca-private\" ","response":"range_response_count:1 size:3369"} Jul 15 10:59:26.527519 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.522202Z","caller":"traceutil/trace.go:171","msg":"trace[691504149] range","detail":"{range_begin:/registry/secrets/tigera-operator/tigera-ca-private; range_end:; response_count:1; response_revision:1394; }","duration":"1.72927909s","start":"2025-07-15T10:59:24.792908Z","end":"2025-07-15T10:59:26.522187Z","steps":["trace[691504149] 'agreement among raft nodes before linearized reading' (duration: 1.728178852s)"],"step_count":1} Jul 15 10:59:26.527993 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.522250Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.792891Z","time spent":"1.729341699s","remote":"172.24.4.149:33122","response type":"/etcdserverpb.KV/Range","request count":0,"request size":53,"response count":1,"response size":3393,"request content":"key:\"/registry/secrets/tigera-operator/tigera-ca-private\" "} Jul 15 10:59:26.527993 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.521513Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.746158227s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 10:59:26.527993 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.522326Z","caller":"traceutil/trace.go:171","msg":"trace[841472596] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1394; }","duration":"1.74701016s","start":"2025-07-15T10:59:24.775300Z","end":"2025-07-15T10:59:26.522310Z","steps":["trace[841472596] 'agreement among raft nodes before linearized reading' (duration: 1.746060199s)"],"step_count":1} Jul 15 10:59:26.527993 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.522393Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.775284Z","time spent":"1.747081486s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":3818,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:59:26.527993 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.521660Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.732912779s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 10:59:26.528581 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.522529Z","caller":"traceutil/trace.go:171","msg":"trace[1703285816] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1394; }","duration":"1.73384254s","start":"2025-07-15T10:59:24.788678Z","end":"2025-07-15T10:59:26.522521Z","steps":["trace[1703285816] 'agreement among raft nodes before linearized reading' (duration: 1.732852453s)"],"step_count":1} Jul 15 10:59:26.528581 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.522572Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.788652Z","time spent":"1.733897164s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":3818,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:59:26.528581 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.521826Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.747636197s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 10:59:26.528581 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:26.522698Z","caller":"traceutil/trace.go:171","msg":"trace[1632086199] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1394; }","duration":"1.748551391s","start":"2025-07-15T10:59:24.774138Z","end":"2025-07-15T10:59:26.522689Z","steps":["trace[1632086199] 'agreement among raft nodes before linearized reading' (duration: 1.74757549s)"],"step_count":1} Jul 15 10:59:26.528965 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:26.522768Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:24.774109Z","time spent":"1.748629962s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":3818,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:59:27.074048 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.072724Z","caller":"traceutil/trace.go:171","msg":"trace[1576174529] transaction","detail":"{read_only:false; response_revision:1395; number_of_response:1; }","duration":"498.995024ms","start":"2025-07-15T10:59:26.573673Z","end":"2025-07-15T10:59:27.072669Z","steps":["trace[1576174529] 'process raft request' (duration: 498.636818ms)"],"step_count":1} Jul 15 10:59:27.074048 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.072939Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:26.573343Z","time spent":"499.506304ms","remote":"172.24.4.149:33206","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":3358,"response count":0,"response size":40,"request content":"compare: success:> failure:<>"} Jul 15 10:59:27.095607 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.094208Z","caller":"traceutil/trace.go:171","msg":"trace[1460593761] linearizableReadLoop","detail":"{readStateIndex:1669; appliedIndex:1668; }","duration":"493.017658ms","start":"2025-07-15T10:59:26.601153Z","end":"2025-07-15T10:59:27.094171Z","steps":["trace[1460593761] 'read index received' (duration: 472.049242ms)","trace[1460593761] 'applied index is now lower than readState.Index' (duration: 20.967114ms)"],"step_count":2} Jul 15 10:59:27.095607 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.094659Z","caller":"traceutil/trace.go:171","msg":"trace[20187903] transaction","detail":"{read_only:false; response_revision:1397; number_of_response:1; }","duration":"357.13801ms","start":"2025-07-15T10:59:26.737452Z","end":"2025-07-15T10:59:27.094590Z","steps":["trace[20187903] 'process raft request' (duration: 356.469019ms)"],"step_count":1} Jul 15 10:59:27.095607 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.094926Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:26.737359Z","time spent":"357.446441ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":517,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:27.098075 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.095196Z","caller":"traceutil/trace.go:171","msg":"trace[1185066074] transaction","detail":"{read_only:false; response_revision:1398; number_of_response:1; }","duration":"357.571271ms","start":"2025-07-15T10:59:26.737606Z","end":"2025-07-15T10:59:27.095177Z","steps":["trace[1185066074] 'process raft request' (duration: 356.416889ms)"],"step_count":1} Jul 15 10:59:27.098075 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.096526Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:26.737591Z","time spent":"357.726448ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:27.098075 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.096657Z","caller":"traceutil/trace.go:171","msg":"trace[1799360499] transaction","detail":"{read_only:false; response_revision:1399; number_of_response:1; }","duration":"316.148757ms","start":"2025-07-15T10:59:26.780490Z","end":"2025-07-15T10:59:27.096639Z","steps":["trace[1799360499] 'process raft request' (duration: 313.628407ms)"],"step_count":1} Jul 15 10:59:27.098075 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.096744Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:26.780453Z","time spent":"316.241425ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":490,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:27.101740 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.096713Z","caller":"traceutil/trace.go:171","msg":"trace[439943504] transaction","detail":"{read_only:false; response_revision:1396; number_of_response:1; }","duration":"507.790963ms","start":"2025-07-15T10:59:26.588852Z","end":"2025-07-15T10:59:27.096643Z","steps":["trace[439943504] 'process raft request' (duration: 504.778029ms)"],"step_count":1} Jul 15 10:59:27.101740 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.096928Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:26.588810Z","time spent":"508.001495ms","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":2693,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:27.101740 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.097010Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"495.861868ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/tigera-operator/goldmane-key-pair\" ","response":"range_response_count:1 size:4711"} Jul 15 10:59:27.101740 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.097071Z","caller":"traceutil/trace.go:171","msg":"trace[1058906120] range","detail":"{range_begin:/registry/secrets/tigera-operator/goldmane-key-pair; range_end:; response_count:1; response_revision:1399; }","duration":"496.02991ms","start":"2025-07-15T10:59:26.601023Z","end":"2025-07-15T10:59:27.097053Z","steps":["trace[1058906120] 'agreement among raft nodes before linearized reading' (duration: 495.767177ms)"],"step_count":1} Jul 15 10:59:27.102531 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.097119Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:26.600957Z","time spent":"496.147706ms","remote":"172.24.4.149:33122","response type":"/etcdserverpb.KV/Range","request count":0,"request size":53,"response count":1,"response size":4735,"request content":"key:\"/registry/secrets/tigera-operator/goldmane-key-pair\" "} Jul 15 10:59:27.112087 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.110259Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"505.949688ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/configmaps/calico-system/whisker-ca-bundle\" ","response":"range_response_count:1 size:1956"} Jul 15 10:59:27.112087 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.110364Z","caller":"traceutil/trace.go:171","msg":"trace[44829182] range","detail":"{range_begin:/registry/configmaps/calico-system/whisker-ca-bundle; range_end:; response_count:1; response_revision:1400; }","duration":"506.061092ms","start":"2025-07-15T10:59:26.604281Z","end":"2025-07-15T10:59:27.110342Z","steps":["trace[44829182] 'agreement among raft nodes before linearized reading' (duration: 505.780715ms)"],"step_count":1} Jul 15 10:59:27.112087 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.110649Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:26.604225Z","time spent":"506.396985ms","remote":"172.24.4.149:33128","response type":"/etcdserverpb.KV/Range","request count":0,"request size":54,"response count":1,"response size":1980,"request content":"key:\"/registry/configmaps/calico-system/whisker-ca-bundle\" "} Jul 15 10:59:27.112087 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.110916Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"373.296891ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"range_response_count:1 size:886"} Jul 15 10:59:27.112087 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.111054Z","caller":"traceutil/trace.go:171","msg":"trace[248191213] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; response_count:1; response_revision:1400; }","duration":"373.447732ms","start":"2025-07-15T10:59:26.737573Z","end":"2025-07-15T10:59:27.111021Z","steps":["trace[248191213] 'agreement among raft nodes before linearized reading' (duration: 372.985245ms)"],"step_count":1} Jul 15 10:59:27.112918 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.111239Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:26.737533Z","time spent":"373.598219ms","remote":"172.24.4.149:54994","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":1,"response size":910,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" "} Jul 15 10:59:27.112918 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.111612Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"374.314693ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/globalnetworkpolicies.crd.projectcalico.org\" ","response":"range_response_count:1 size:35079"} Jul 15 10:59:27.112918 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.111679Z","caller":"traceutil/trace.go:171","msg":"trace[2020878655] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/globalnetworkpolicies.crd.projectcalico.org; range_end:; response_count:1; response_revision:1400; }","duration":"374.433471ms","start":"2025-07-15T10:59:26.737227Z","end":"2025-07-15T10:59:27.111661Z","steps":["trace[2020878655] 'agreement among raft nodes before linearized reading' (duration: 373.851295ms)"],"step_count":1} Jul 15 10:59:27.114680 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.113462Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:26.737200Z","time spent":"374.511099ms","remote":"172.24.4.149:33078","response type":"/etcdserverpb.KV/Range","request count":0,"request size":102,"response count":1,"response size":35103,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/globalnetworkpolicies.crd.projectcalico.org\" "} Jul 15 10:59:27.114680 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.113495Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"123.119032ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/\" range_end:\"/registry/minions0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:59:27.115864 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.114502Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"490.537232ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 10:59:27.117448 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.115746Z","caller":"traceutil/trace.go:171","msg":"trace[1938944986] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1400; }","duration":"491.804741ms","start":"2025-07-15T10:59:26.623908Z","end":"2025-07-15T10:59:27.115712Z","steps":["trace[1938944986] 'agreement among raft nodes before linearized reading' (duration: 490.384008ms)"],"step_count":1} Jul 15 10:59:27.117448 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.116011Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:26.623852Z","time spent":"492.080018ms","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":3818,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:59:27.117448 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.114519Z","caller":"traceutil/trace.go:171","msg":"trace[129904600] range","detail":"{range_begin:/registry/minions/; range_end:/registry/minions0; response_count:0; response_revision:1400; }","duration":"123.749039ms","start":"2025-07-15T10:59:26.990290Z","end":"2025-07-15T10:59:27.114039Z","steps":["trace[129904600] 'agreement among raft nodes before linearized reading' (duration: 123.071622ms)"],"step_count":1} Jul 15 10:59:27.119312 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.115262Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"494.279261ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/calico-crds\" ","response":"range_response_count:1 size:1255"} Jul 15 10:59:27.119312 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.117234Z","caller":"traceutil/trace.go:171","msg":"trace[1422699537] range","detail":"{range_begin:/registry/clusterroles/calico-crds; range_end:; response_count:1; response_revision:1400; }","duration":"496.268062ms","start":"2025-07-15T10:59:26.620941Z","end":"2025-07-15T10:59:27.117209Z","steps":["trace[1422699537] 'agreement among raft nodes before linearized reading' (duration: 494.16879ms)"],"step_count":1} Jul 15 10:59:27.119312 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.117488Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:26.620890Z","time spent":"496.571163ms","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":36,"response count":1,"response size":1279,"request content":"key:\"/registry/clusterroles/calico-crds\" "} Jul 15 10:59:27.119312 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.117606Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"511.831197ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/tigera-operator/tigera-ca-private\" ","response":"range_response_count:1 size:3369"} Jul 15 10:59:27.119312 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.118003Z","caller":"traceutil/trace.go:171","msg":"trace[1501952806] range","detail":"{range_begin:/registry/secrets/tigera-operator/tigera-ca-private; range_end:; response_count:1; response_revision:1400; }","duration":"512.497184ms","start":"2025-07-15T10:59:26.605483Z","end":"2025-07-15T10:59:27.117980Z","steps":["trace[1501952806] 'agreement among raft nodes before linearized reading' (duration: 511.031767ms)"],"step_count":1} Jul 15 10:59:27.120765 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.118321Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:26.605454Z","time spent":"512.81382ms","remote":"172.24.4.149:33122","response type":"/etcdserverpb.KV/Range","request count":0,"request size":53,"response count":1,"response size":3393,"request content":"key:\"/registry/secrets/tigera-operator/tigera-ca-private\" "} Jul 15 10:59:27.120765 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.116148Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"500.903292ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 10:59:27.120765 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.118781Z","caller":"traceutil/trace.go:171","msg":"trace[2140986104] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1400; }","duration":"503.509846ms","start":"2025-07-15T10:59:26.615202Z","end":"2025-07-15T10:59:27.118712Z","steps":["trace[2140986104] 'agreement among raft nodes before linearized reading' (duration: 500.107588ms)"],"step_count":1} Jul 15 10:59:27.120765 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:27.119031Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:26.615184Z","time spent":"503.818228ms","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":1,"response size":3818,"request content":"key:\"/registry/operator.tigera.io/installations/default\" "} Jul 15 10:59:27.390925 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.389976Z","caller":"traceutil/trace.go:171","msg":"trace[1794646805] transaction","detail":"{read_only:false; response_revision:1401; number_of_response:1; }","duration":"198.023659ms","start":"2025-07-15T10:59:27.191926Z","end":"2025-07-15T10:59:27.389949Z","steps":["trace[1794646805] 'process raft request' (duration: 197.650744ms)"],"step_count":1} Jul 15 10:59:27.411999 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.409506Z","caller":"traceutil/trace.go:171","msg":"trace[1279306488] transaction","detail":"{read_only:false; response_revision:1402; number_of_response:1; }","duration":"213.876555ms","start":"2025-07-15T10:59:27.195603Z","end":"2025-07-15T10:59:27.409479Z","steps":["trace[1279306488] 'process raft request' (duration: 213.516805ms)"],"step_count":1} Jul 15 10:59:27.411999 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:27.410712Z","caller":"traceutil/trace.go:171","msg":"trace[805055869] transaction","detail":"{read_only:false; response_revision:1403; number_of_response:1; }","duration":"210.205619ms","start":"2025-07-15T10:59:27.200467Z","end":"2025-07-15T10:59:27.410672Z","steps":["trace[805055869] 'process raft request' (duration: 208.797762ms)"],"step_count":1} Jul 15 10:59:32.252957 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:31.933993Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"446.725077ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 10:59:32.252957 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:31.934449Z","caller":"traceutil/trace.go:171","msg":"trace[1419770194] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"447.195399ms","start":"2025-07-15T10:59:31.487067Z","end":"2025-07-15T10:59:31.934262Z","steps":["trace[1419770194] 'agreement among raft nodes before linearized reading' (duration: 446.817935ms)"],"step_count":1} Jul 15 10:59:32.252957 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:31.934592Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.487042Z","time spent":"447.512066ms","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:59:32.252957 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:31.934649Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"445.717034ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 10:59:32.252957 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:31.934679Z","caller":"traceutil/trace.go:171","msg":"trace[614043675] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"445.921717ms","start":"2025-07-15T10:59:31.488746Z","end":"2025-07-15T10:59:31.934668Z","steps":["trace[614043675] 'agreement among raft nodes before linearized reading' (duration: 445.892951ms)"],"step_count":1} Jul 15 10:59:32.255995 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:31.934727Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.488726Z","time spent":"445.990219ms","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:59:32.255995 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:31.977883Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074584,"retry-timeout":"500ms"} Jul 15 10:59:32.255995 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:32.060248Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"2.157814923s","expected-duration":"1s"} Jul 15 10:59:32.255995 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:32.073008Z","caller":"traceutil/trace.go:171","msg":"trace[1075840920] transaction","detail":"{read_only:false; response_revision:1404; number_of_response:1; }","duration":"2.17112294s","start":"2025-07-15T10:59:29.901851Z","end":"2025-07-15T10:59:32.072974Z","steps":["trace[1075840920] 'process raft request' (duration: 2.170158832s)"],"step_count":1} Jul 15 10:59:32.255995 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:32.073314Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:29.901372Z","time spent":"2.171725454s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":490,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:33.943631 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:33.943021Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000756745s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 10:59:33.943631 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:33.943237Z","caller":"traceutil/trace.go:171","msg":"trace[764776389] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.001011071s","start":"2025-07-15T10:59:31.942162Z","end":"2025-07-15T10:59:33.943173Z","steps":["trace[764776389] 'agreement among raft nodes before linearized reading' (duration: 2.000776521s)"],"step_count":1} Jul 15 10:59:34.045848 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:33.943777Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.942146Z","time spent":"2.00140666s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:59:34.112977 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.111211Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"2.050515436s","expected-duration":"1s"} Jul 15 10:59:34.113278 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.111826Z","caller":"traceutil/trace.go:171","msg":"trace[332620578] linearizableReadLoop","detail":"{readStateIndex:1683; appliedIndex:1680; }","duration":"2.634709146s","start":"2025-07-15T10:59:31.477088Z","end":"2025-07-15T10:59:34.111797Z","steps":["trace[332620578] 'read index received' (duration: 594.732306ms)","trace[332620578] 'applied index is now lower than readState.Index' (duration: 2.039975858s)"],"step_count":2} Jul 15 10:59:34.113278 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.112095Z","caller":"traceutil/trace.go:171","msg":"trace[1601907170] transaction","detail":"{read_only:false; response_revision:1405; number_of_response:1; }","duration":"2.625584203s","start":"2025-07-15T10:59:31.486496Z","end":"2025-07-15T10:59:34.112080Z","steps":["trace[1601907170] 'process raft request' (duration: 2.625041752s)"],"step_count":1} Jul 15 10:59:34.113278 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.112250Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.486340Z","time spent":"2.62579116s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:34.113278 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.112349Z","caller":"traceutil/trace.go:171","msg":"trace[1393977061] transaction","detail":"{read_only:false; response_revision:1406; number_of_response:1; }","duration":"2.618127346s","start":"2025-07-15T10:59:31.494196Z","end":"2025-07-15T10:59:34.112324Z","steps":["trace[1393977061] 'process raft request' (duration: 2.617534109s)"],"step_count":1} Jul 15 10:59:34.113687 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.112435Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.494165Z","time spent":"2.618205617s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":517,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:34.113687 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.112632Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.635668255s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rolebindings/kube-system/calico-apiserver-auth-reader\" ","response":"range_response_count:1 size:679"} Jul 15 10:59:34.113687 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.112693Z","caller":"traceutil/trace.go:171","msg":"trace[12087077] range","detail":"{range_begin:/registry/rolebindings/kube-system/calico-apiserver-auth-reader; range_end:; response_count:1; response_revision:1406; }","duration":"2.635855846s","start":"2025-07-15T10:59:31.476808Z","end":"2025-07-15T10:59:34.112663Z","steps":["trace[12087077] 'agreement among raft nodes before linearized reading' (duration: 2.635663578s)"],"step_count":1} Jul 15 10:59:34.113687 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.112737Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.476526Z","time spent":"2.636201267s","remote":"172.24.4.149:33358","response type":"/etcdserverpb.KV/Range","request count":0,"request size":65,"response count":1,"response size":703,"request content":"key:\"/registry/rolebindings/kube-system/calico-apiserver-auth-reader\" "} Jul 15 10:59:34.123181 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.122558Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.194237476s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/deployments/\" range_end:\"/registry/deployments0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:59:34.123547 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.122697Z","caller":"traceutil/trace.go:171","msg":"trace[2117312407] range","detail":"{range_begin:/registry/deployments/; range_end:/registry/deployments0; response_count:0; response_revision:1409; }","duration":"2.194464311s","start":"2025-07-15T10:59:31.928212Z","end":"2025-07-15T10:59:34.122677Z","steps":["trace[2117312407] 'agreement among raft nodes before linearized reading' (duration: 2.194250862s)"],"step_count":1} Jul 15 10:59:34.123547 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.123170Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.928180Z","time spent":"2.194780216s","remote":"172.24.4.149:33462","response type":"/etcdserverpb.KV/Range","request count":0,"request size":50,"response count":7,"response size":31,"request content":"key:\"/registry/deployments/\" range_end:\"/registry/deployments0\" count_only:true "} Jul 15 10:59:34.123899 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.123368Z","caller":"traceutil/trace.go:171","msg":"trace[1699280966] transaction","detail":"{read_only:false; response_revision:1408; number_of_response:1; }","duration":"1.709738878s","start":"2025-07-15T10:59:32.413612Z","end":"2025-07-15T10:59:34.123351Z","steps":["trace[1699280966] 'process raft request' (duration: 1.707892025s)"],"step_count":1} Jul 15 10:59:34.123899 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.123604Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:32.413584Z","time spent":"1.709898303s","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1139,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:34.124343 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.124209Z","caller":"traceutil/trace.go:171","msg":"trace[554005918] transaction","detail":"{read_only:false; response_revision:1407; number_of_response:1; }","duration":"1.711831761s","start":"2025-07-15T10:59:32.410598Z","end":"2025-07-15T10:59:34.122429Z","steps":["trace[554005918] 'process raft request' (duration: 1.710607111s)"],"step_count":1} Jul 15 10:59:34.126846 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.124671Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"112.438136ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/blockaffinities/\" range_end:\"/registry/crd.projectcalico.org/blockaffinities0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:59:34.126846 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.124749Z","caller":"traceutil/trace.go:171","msg":"trace[188312925] range","detail":"{range_begin:/registry/crd.projectcalico.org/blockaffinities/; range_end:/registry/crd.projectcalico.org/blockaffinities0; response_count:0; response_revision:1409; }","duration":"112.650073ms","start":"2025-07-15T10:59:34.012066Z","end":"2025-07-15T10:59:34.124716Z","steps":["trace[188312925] 'agreement among raft nodes before linearized reading' (duration: 112.553447ms)"],"step_count":1} Jul 15 10:59:34.126846 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.124869Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"117.708895ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/networkpolicies/\" range_end:\"/registry/crd.projectcalico.org/networkpolicies0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:59:34.126846 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.124895Z","caller":"traceutil/trace.go:171","msg":"trace[1228635376] range","detail":"{range_begin:/registry/crd.projectcalico.org/networkpolicies/; range_end:/registry/crd.projectcalico.org/networkpolicies0; response_count:0; response_revision:1409; }","duration":"117.755564ms","start":"2025-07-15T10:59:34.007132Z","end":"2025-07-15T10:59:34.124887Z","steps":["trace[1228635376] 'agreement among raft nodes before linearized reading' (duration: 117.711299ms)"],"step_count":1} Jul 15 10:59:34.127176 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.124988Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"118.730764ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:34.127176 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.125020Z","caller":"traceutil/trace.go:171","msg":"trace[1907788956] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1409; }","duration":"118.803443ms","start":"2025-07-15T10:59:34.006200Z","end":"2025-07-15T10:59:34.125004Z","steps":["trace[1907788956] 'agreement among raft nodes before linearized reading' (duration: 118.771532ms)"],"step_count":1} Jul 15 10:59:34.127176 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.125150Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"494.663514ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rolebindings/\" range_end:\"/registry/rolebindings0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:59:34.127176 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.125174Z","caller":"traceutil/trace.go:171","msg":"trace[671568917] range","detail":"{range_begin:/registry/rolebindings/; range_end:/registry/rolebindings0; response_count:0; response_revision:1409; }","duration":"494.804234ms","start":"2025-07-15T10:59:33.630363Z","end":"2025-07-15T10:59:34.125167Z","steps":["trace[671568917] 'agreement among raft nodes before linearized reading' (duration: 494.735472ms)"],"step_count":1} Jul 15 10:59:34.127176 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.125200Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:33.630334Z","time spent":"494.855953ms","remote":"172.24.4.149:33358","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":16,"response size":31,"request content":"key:\"/registry/rolebindings/\" range_end:\"/registry/rolebindings0\" count_only:true "} Jul 15 10:59:34.127473 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.125288Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"810.203059ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/csinodes/\" range_end:\"/registry/csinodes0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:59:34.127473 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.125318Z","caller":"traceutil/trace.go:171","msg":"trace[2038155243] range","detail":"{range_begin:/registry/csinodes/; range_end:/registry/csinodes0; response_count:0; response_revision:1409; }","duration":"810.289785ms","start":"2025-07-15T10:59:33.315020Z","end":"2025-07-15T10:59:34.125310Z","steps":["trace[2038155243] 'agreement among raft nodes before linearized reading' (duration: 810.228558ms)"],"step_count":1} Jul 15 10:59:34.127473 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.125359Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:33.314993Z","time spent":"810.341244ms","remote":"172.24.4.149:33400","response type":"/etcdserverpb.KV/Range","request count":0,"request size":44,"response count":2,"response size":31,"request content":"key:\"/registry/csinodes/\" range_end:\"/registry/csinodes0\" count_only:true "} Jul 15 10:59:34.127473 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.125561Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.18473513s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76\" ","response":"range_response_count:1 size:905"} Jul 15 10:59:34.127678 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.125592Z","caller":"traceutil/trace.go:171","msg":"trace[1949683795] range","detail":"{range_begin:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76; range_end:; response_count:1; response_revision:1409; }","duration":"2.184800486s","start":"2025-07-15T10:59:31.940782Z","end":"2025-07-15T10:59:34.125583Z","steps":["trace[1949683795] 'agreement among raft nodes before linearized reading' (duration: 2.184674705s)"],"step_count":1} Jul 15 10:59:34.127678 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.125617Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.940763Z","time spent":"2.184845623s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":97,"response count":1,"response size":929,"request content":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76\" "} Jul 15 10:59:34.127678 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.125956Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.627745808s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/roles/calico-system/goldmane\" ","response":"range_response_count:1 size:531"} Jul 15 10:59:34.127678 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.125986Z","caller":"traceutil/trace.go:171","msg":"trace[463806979] range","detail":"{range_begin:/registry/roles/calico-system/goldmane; range_end:; response_count:1; response_revision:1409; }","duration":"2.627846821s","start":"2025-07-15T10:59:31.498131Z","end":"2025-07-15T10:59:34.125978Z","steps":["trace[463806979] 'agreement among raft nodes before linearized reading' (duration: 2.627714879s)"],"step_count":1} Jul 15 10:59:34.127921 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.126021Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.498037Z","time spent":"2.62796636s","remote":"172.24.4.149:33348","response type":"/etcdserverpb.KV/Range","request count":0,"request size":40,"response count":1,"response size":555,"request content":"key:\"/registry/roles/calico-system/goldmane\" "} Jul 15 10:59:34.127921 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.126171Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.551324496s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/endpoints/calico-system/calico-typha\" ","response":"range_response_count:1 size:688"} Jul 15 10:59:34.127921 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.126210Z","caller":"traceutil/trace.go:171","msg":"trace[1036106827] range","detail":"{range_begin:/registry/services/endpoints/calico-system/calico-typha; range_end:; response_count:1; response_revision:1409; }","duration":"2.551352981s","start":"2025-07-15T10:59:31.574839Z","end":"2025-07-15T10:59:34.126192Z","steps":["trace[1036106827] 'agreement among raft nodes before linearized reading' (duration: 2.551255333s)"],"step_count":1} Jul 15 10:59:34.127921 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.126240Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.574765Z","time spent":"2.551467071s","remote":"172.24.4.149:33194","response type":"/etcdserverpb.KV/Range","request count":0,"request size":57,"response count":1,"response size":712,"request content":"key:\"/registry/services/endpoints/calico-system/calico-typha\" "} Jul 15 10:59:34.127921 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.126337Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.560034214s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/\" range_end:\"/registry/operator.tigera.io/installations0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:59:34.132088 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.124553Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:32.410555Z","time spent":"1.713943289s","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1147,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:34.132088 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.126638Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.635098915s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/networkpolicies/calico-system/whisker\" ","response":"range_response_count:1 size:739"} Jul 15 10:59:34.132088 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.128466Z","caller":"traceutil/trace.go:171","msg":"trace[1668555245] range","detail":"{range_begin:/registry/networkpolicies/calico-system/whisker; range_end:; response_count:1; response_revision:1409; }","duration":"2.637070306s","start":"2025-07-15T10:59:31.491372Z","end":"2025-07-15T10:59:34.128443Z","steps":["trace[1668555245] 'agreement among raft nodes before linearized reading' (duration: 2.635181374s)"],"step_count":1} Jul 15 10:59:34.132088 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.128731Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.491356Z","time spent":"2.63735874s","remote":"172.24.4.149:33316","response type":"/etcdserverpb.KV/Range","request count":0,"request size":49,"response count":1,"response size":763,"request content":"key:\"/registry/networkpolicies/calico-system/whisker\" "} Jul 15 10:59:34.132422 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.129174Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.639906564s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/endpointslices/\" range_end:\"/registry/endpointslices0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:59:34.132422 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.129366Z","caller":"traceutil/trace.go:171","msg":"trace[2092807292] range","detail":"{range_begin:/registry/endpointslices/; range_end:/registry/endpointslices0; response_count:0; response_revision:1409; }","duration":"2.640137688s","start":"2025-07-15T10:59:31.489217Z","end":"2025-07-15T10:59:34.129354Z","steps":["trace[2092807292] 'agreement among raft nodes before linearized reading' (duration: 2.639864263s)"],"step_count":1} Jul 15 10:59:34.132422 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.129661Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.489203Z","time spent":"2.640439224s","remote":"172.24.4.149:33310","response type":"/etcdserverpb.KV/Range","request count":0,"request size":56,"response count":6,"response size":31,"request content":"key:\"/registry/endpointslices/\" range_end:\"/registry/endpointslices0\" count_only:true "} Jul 15 10:59:34.132422 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.129906Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.651830985s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/felixconfigurations/\" range_end:\"/registry/crd.projectcalico.org/felixconfigurations0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:59:34.132645 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.129948Z","caller":"traceutil/trace.go:171","msg":"trace[1863248170] range","detail":"{range_begin:/registry/crd.projectcalico.org/felixconfigurations/; range_end:/registry/crd.projectcalico.org/felixconfigurations0; response_count:0; response_revision:1409; }","duration":"2.651913443s","start":"2025-07-15T10:59:31.478015Z","end":"2025-07-15T10:59:34.129928Z","steps":["trace[1863248170] 'agreement among raft nodes before linearized reading' (duration: 2.651843409s)"],"step_count":1} Jul 15 10:59:34.132645 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.129988Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.478001Z","time spent":"2.651967417s","remote":"172.24.4.149:54930","response type":"/etcdserverpb.KV/Range","request count":0,"request size":110,"response count":1,"response size":31,"request content":"key:\"/registry/crd.projectcalico.org/felixconfigurations/\" range_end:\"/registry/crd.projectcalico.org/felixconfigurations0\" count_only:true "} Jul 15 10:59:34.132645 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.126366Z","caller":"traceutil/trace.go:171","msg":"trace[60499287] range","detail":"{range_begin:/registry/operator.tigera.io/installations/; range_end:/registry/operator.tigera.io/installations0; response_count:0; response_revision:1409; }","duration":"2.560064372s","start":"2025-07-15T10:59:31.566291Z","end":"2025-07-15T10:59:34.126356Z","steps":["trace[60499287] 'agreement among raft nodes before linearized reading' (duration: 2.560019626s)"],"step_count":1} Jul 15 10:59:34.132645 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.130251Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.566261Z","time spent":"2.563976124s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":1,"response size":31,"request content":"key:\"/registry/operator.tigera.io/installations/\" range_end:\"/registry/operator.tigera.io/installations0\" count_only:true "} Jul 15 10:59:34.132913 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.127836Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.619720012s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/hostendpoints.crd.projectcalico.org\" ","response":"range_response_count:1 size:5520"} Jul 15 10:59:34.132913 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.130341Z","caller":"traceutil/trace.go:171","msg":"trace[2027484314] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/hostendpoints.crd.projectcalico.org; range_end:; response_count:1; response_revision:1409; }","duration":"2.622374773s","start":"2025-07-15T10:59:31.507957Z","end":"2025-07-15T10:59:34.130331Z","steps":["trace[2027484314] 'agreement among raft nodes before linearized reading' (duration: 2.619711766s)"],"step_count":1} Jul 15 10:59:34.132913 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.130371Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.507920Z","time spent":"2.62244095s","remote":"172.24.4.149:33078","response type":"/etcdserverpb.KV/Range","request count":0,"request size":94,"response count":1,"response size":5544,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/hostendpoints.crd.projectcalico.org\" "} Jul 15 10:59:34.132913 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.127958Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.624966621s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/runtimeclasses/\" range_end:\"/registry/runtimeclasses0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:59:34.133178 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.131064Z","caller":"traceutil/trace.go:171","msg":"trace[990732981] range","detail":"{range_begin:/registry/runtimeclasses/; range_end:/registry/runtimeclasses0; response_count:0; response_revision:1409; }","duration":"2.628117644s","start":"2025-07-15T10:59:31.502935Z","end":"2025-07-15T10:59:34.131052Z","steps":["trace[990732981] 'agreement among raft nodes before linearized reading' (duration: 2.624975006s)"],"step_count":1} Jul 15 10:59:34.133178 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.131192Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.502914Z","time spent":"2.628195653s","remote":"172.24.4.149:33336","response type":"/etcdserverpb.KV/Range","request count":0,"request size":56,"response count":0,"response size":29,"request content":"key:\"/registry/runtimeclasses/\" range_end:\"/registry/runtimeclasses0\" count_only:true "} Jul 15 10:59:34.133178 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.128138Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.627792558s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 10:59:34.133178 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.131363Z","caller":"traceutil/trace.go:171","msg":"trace[528121887] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1409; }","duration":"2.631074553s","start":"2025-07-15T10:59:31.500277Z","end":"2025-07-15T10:59:34.131352Z","steps":["trace[528121887] 'agreement among raft nodes before linearized reading' (duration: 2.62773606s)"],"step_count":1} Jul 15 10:59:34.133376 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.131445Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:31.500255Z","time spent":"2.631177881s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 10:59:34.288779 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.288182Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"111.487632ms","expected-duration":"100ms","prefix":"","request":"header: lease_grant:","response":"size:41"} Jul 15 10:59:34.290310 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.289119Z","caller":"traceutil/trace.go:171","msg":"trace[1963340325] linearizableReadLoop","detail":"{readStateIndex:1688; appliedIndex:1686; }","duration":"148.826114ms","start":"2025-07-15T10:59:34.140236Z","end":"2025-07-15T10:59:34.289062Z","steps":["trace[1963340325] 'read index received' (duration: 17.199828ms)","trace[1963340325] 'applied index is now lower than readState.Index' (duration: 131.624903ms)"],"step_count":2} Jul 15 10:59:34.290310 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.289236Z","caller":"traceutil/trace.go:171","msg":"trace[219474502] transaction","detail":"{read_only:false; response_revision:1411; number_of_response:1; }","duration":"139.91846ms","start":"2025-07-15T10:59:34.149301Z","end":"2025-07-15T10:59:34.289220Z","steps":["trace[219474502] 'process raft request' (duration: 139.033683ms)"],"step_count":1} Jul 15 10:59:34.290970 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.290498Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"150.247861ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:34.290970 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.290573Z","caller":"traceutil/trace.go:171","msg":"trace[1516679043] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1411; }","duration":"150.367049ms","start":"2025-07-15T10:59:34.140188Z","end":"2025-07-15T10:59:34.290555Z","steps":["trace[1516679043] 'agreement among raft nodes before linearized reading' (duration: 149.158432ms)"],"step_count":1} Jul 15 10:59:34.295328 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.294883Z","caller":"traceutil/trace.go:171","msg":"trace[1870633920] transaction","detail":"{read_only:false; response_revision:1413; number_of_response:1; }","duration":"127.036563ms","start":"2025-07-15T10:59:34.167815Z","end":"2025-07-15T10:59:34.294851Z","steps":["trace[1870633920] 'process raft request' (duration: 126.95698ms)"],"step_count":1} Jul 15 10:59:34.296884 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.296556Z","caller":"traceutil/trace.go:171","msg":"trace[1759607334] transaction","detail":"{read_only:false; response_revision:1412; number_of_response:1; }","duration":"138.759818ms","start":"2025-07-15T10:59:34.157727Z","end":"2025-07-15T10:59:34.296487Z","steps":["trace[1759607334] 'process raft request' (duration: 136.762387ms)"],"step_count":1} Jul 15 10:59:34.298670 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.298242Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"134.412941ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678d1663474b\" ","response":"range_response_count:1 size:904"} Jul 15 10:59:34.298670 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.298429Z","caller":"traceutil/trace.go:171","msg":"trace[1086837970] range","detail":"{range_begin:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678d1663474b; range_end:; response_count:1; response_revision:1413; }","duration":"134.602845ms","start":"2025-07-15T10:59:34.163769Z","end":"2025-07-15T10:59:34.298372Z","steps":["trace[1086837970] 'agreement among raft nodes before linearized reading' (duration: 134.368917ms)"],"step_count":1} Jul 15 10:59:34.300825 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.299440Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"140.717613ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/ipamblocks.crd.projectcalico.org\" ","response":"range_response_count:1 size:5469"} Jul 15 10:59:34.300825 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.299526Z","caller":"traceutil/trace.go:171","msg":"trace[1317738180] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/ipamblocks.crd.projectcalico.org; range_end:; response_count:1; response_revision:1413; }","duration":"140.804328ms","start":"2025-07-15T10:59:34.158705Z","end":"2025-07-15T10:59:34.299509Z","steps":["trace[1317738180] 'agreement among raft nodes before linearized reading' (duration: 140.508461ms)"],"step_count":1} Jul 15 10:59:34.300825 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.299827Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"142.516923ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/calico-apiserver-webhook-reader\" ","response":"range_response_count:1 size:663"} Jul 15 10:59:34.300825 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.299877Z","caller":"traceutil/trace.go:171","msg":"trace[914210026] range","detail":"{range_begin:/registry/clusterrolebindings/calico-apiserver-webhook-reader; range_end:; response_count:1; response_revision:1413; }","duration":"142.568903ms","start":"2025-07-15T10:59:34.157293Z","end":"2025-07-15T10:59:34.299862Z","steps":["trace[914210026] 'agreement among raft nodes before linearized reading' (duration: 142.403475ms)"],"step_count":1} Jul 15 10:59:34.300825 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.300114Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"156.508589ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rolebindings/calico-system/goldmane\" ","response":"range_response_count:1 size:598"} Jul 15 10:59:34.301310 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.300181Z","caller":"traceutil/trace.go:171","msg":"trace[1965223926] range","detail":"{range_begin:/registry/rolebindings/calico-system/goldmane; range_end:; response_count:1; response_revision:1413; }","duration":"156.550718ms","start":"2025-07-15T10:59:34.143592Z","end":"2025-07-15T10:59:34.300143Z","steps":["trace[1965223926] 'agreement among raft nodes before linearized reading' (duration: 156.434806ms)"],"step_count":1} Jul 15 10:59:34.516230 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.514911Z","caller":"traceutil/trace.go:171","msg":"trace[490777493] transaction","detail":"{read_only:false; response_revision:1417; number_of_response:1; }","duration":"153.978345ms","start":"2025-07-15T10:59:34.360889Z","end":"2025-07-15T10:59:34.514868Z","steps":["trace[490777493] 'process raft request' (duration: 91.529013ms)","trace[490777493] 'compare' (duration: 61.946057ms)"],"step_count":2} Jul 15 10:59:34.516230 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.515794Z","caller":"traceutil/trace.go:171","msg":"trace[1768612203] linearizableReadLoop","detail":"{readStateIndex:1695; appliedIndex:1694; }","duration":"151.904738ms","start":"2025-07-15T10:59:34.363830Z","end":"2025-07-15T10:59:34.515735Z","steps":["trace[1768612203] 'read index received' (duration: 88.601298ms)","trace[1768612203] 'applied index is now lower than readState.Index' (duration: 63.301576ms)"],"step_count":2} Jul 15 10:59:34.518061 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.516824Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"179.531855ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/\" range_end:\"/registry/masterleases0\" ","response":"range_response_count:1 size:131"} Jul 15 10:59:34.518061 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.516921Z","caller":"traceutil/trace.go:171","msg":"trace[937146597] range","detail":"{range_begin:/registry/masterleases/; range_end:/registry/masterleases0; response_count:1; response_revision:1418; }","duration":"179.666372ms","start":"2025-07-15T10:59:34.337234Z","end":"2025-07-15T10:59:34.516900Z","steps":["trace[937146597] 'agreement among raft nodes before linearized reading' (duration: 179.37855ms)"],"step_count":1} Jul 15 10:59:34.518061 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.515478Z","caller":"traceutil/trace.go:171","msg":"trace[949661079] transaction","detail":"{read_only:false; response_revision:1418; number_of_response:1; }","duration":"140.666445ms","start":"2025-07-15T10:59:34.374698Z","end":"2025-07-15T10:59:34.515365Z","steps":["trace[949661079] 'process raft request' (duration: 140.033891ms)"],"step_count":1} Jul 15 10:59:34.518061 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.517206Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"179.020304ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76\" ","response":"range_response_count:1 size:905"} Jul 15 10:59:34.519861 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.517712Z","caller":"traceutil/trace.go:171","msg":"trace[1386239051] range","detail":"{range_begin:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76; range_end:; response_count:1; response_revision:1418; }","duration":"179.085378ms","start":"2025-07-15T10:59:34.338164Z","end":"2025-07-15T10:59:34.517250Z","steps":["trace[1386239051] 'agreement among raft nodes before linearized reading' (duration: 178.918057ms)"],"step_count":1} Jul 15 10:59:34.520761 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.516856Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"145.627208ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/configmaps/calico-system/goldmane\" ","response":"range_response_count:1 size:494"} Jul 15 10:59:34.520761 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.518820Z","caller":"traceutil/trace.go:171","msg":"trace[1426660444] range","detail":"{range_begin:/registry/configmaps/calico-system/goldmane; range_end:; response_count:1; response_revision:1418; }","duration":"147.51039ms","start":"2025-07-15T10:59:34.371215Z","end":"2025-07-15T10:59:34.518726Z","steps":["trace[1426660444] 'agreement among raft nodes before linearized reading' (duration: 145.540151ms)"],"step_count":1} Jul 15 10:59:34.520761 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.519226Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"120.891247ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/networkpolicies.crd.projectcalico.org\" ","response":"range_response_count:1 size:34255"} Jul 15 10:59:34.520761 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.519301Z","caller":"traceutil/trace.go:171","msg":"trace[8032658] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/networkpolicies.crd.projectcalico.org; range_end:; response_count:1; response_revision:1418; }","duration":"121.000488ms","start":"2025-07-15T10:59:34.398279Z","end":"2025-07-15T10:59:34.519279Z","steps":["trace[8032658] 'agreement among raft nodes before linearized reading' (duration: 120.510828ms)"],"step_count":1} Jul 15 10:59:34.522825 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:34.520114Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"132.24342ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/whiskers/default\" ","response":"range_response_count:1 size:948"} Jul 15 10:59:34.522825 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:34.521906Z","caller":"traceutil/trace.go:171","msg":"trace[1429347155] range","detail":"{range_begin:/registry/operator.tigera.io/whiskers/default; range_end:; response_count:1; response_revision:1418; }","duration":"134.312097ms","start":"2025-07-15T10:59:34.387566Z","end":"2025-07-15T10:59:34.521878Z","steps":["trace[1429347155] 'agreement among raft nodes before linearized reading' (duration: 131.778588ms)"],"step_count":1} Jul 15 10:59:38.531141 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:37.454966Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074931,"retry-timeout":"500ms"} Jul 15 10:59:38.531141 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:37.955824Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074931,"retry-timeout":"500ms"} Jul 15 10:59:38.531141 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:38.456439Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885074931,"retry-timeout":"500ms"} Jul 15 10:59:39.103159 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:38.948771Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"2.089096691s","expected-duration":"1s"} Jul 15 10:59:39.103159 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:38.954402Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.999977207s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 10:59:39.103159 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:38.954500Z","caller":"traceutil/trace.go:171","msg":"trace[1030527819] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.000149519s","start":"2025-07-15T10:59:36.954331Z","end":"2025-07-15T10:59:38.954480Z","steps":["trace[1030527819] 'agreement among raft nodes before linearized reading' (duration: 2.000021684s)"],"step_count":1} Jul 15 10:59:39.103159 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:38.954588Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:36.954298Z","time spent":"2.000276643s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 10:59:39.119621 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.118921Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.42723586s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/csistoragecapacities/\" range_end:\"/registry/csistoragecapacities0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:59:39.119621 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.119005Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.26322253s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/managementclusterconnections.operator.tigera.io\" ","response":"range_response_count:1 size:12873"} Jul 15 10:59:39.119621 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.119022Z","caller":"traceutil/trace.go:171","msg":"trace[1341608028] range","detail":"{range_begin:/registry/csistoragecapacities/; range_end:/registry/csistoragecapacities0; response_count:0; response_revision:1426; }","duration":"2.427384356s","start":"2025-07-15T10:59:36.691597Z","end":"2025-07-15T10:59:39.118981Z","steps":["trace[1341608028] 'count revisions from in-memory index tree' (duration: 2.427060845s)"],"step_count":1} Jul 15 10:59:39.119621 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.119039Z","caller":"traceutil/trace.go:171","msg":"trace[233674376] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/managementclusterconnections.operator.tigera.io; range_end:; response_count:1; response_revision:1426; }","duration":"2.263307924s","start":"2025-07-15T10:59:36.855722Z","end":"2025-07-15T10:59:39.119030Z","steps":["trace[233674376] 'range keys from in-memory index tree' (duration: 2.26300948s)"],"step_count":1} Jul 15 10:59:39.126147 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.119064Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:36.691567Z","time spent":"2.427484167s","remote":"172.24.4.149:33422","response type":"/etcdserverpb.KV/Range","request count":0,"request size":68,"response count":0,"response size":29,"request content":"key:\"/registry/csistoragecapacities/\" range_end:\"/registry/csistoragecapacities0\" count_only:true "} Jul 15 10:59:39.126147 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.119064Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:36.855694Z","time spent":"2.263361917s","remote":"172.24.4.149:33078","response type":"/etcdserverpb.KV/Range","request count":0,"request size":106,"response count":1,"response size":12897,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/managementclusterconnections.operator.tigera.io\" "} Jul 15 10:59:39.126147 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.119167Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.872245221s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/tigera-extension-apiserver-auth-access\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:39.126147 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.119187Z","caller":"traceutil/trace.go:171","msg":"trace[328061640] range","detail":"{range_begin:/registry/clusterroles/tigera-extension-apiserver-auth-access; range_end:; response_count:0; response_revision:1426; }","duration":"2.872295718s","start":"2025-07-15T10:59:36.246885Z","end":"2025-07-15T10:59:39.119181Z","steps":["trace[328061640] 'range keys from in-memory index tree' (duration: 2.872118397s)"],"step_count":1} Jul 15 10:59:39.126273 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.119205Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:36.246869Z","time spent":"2.872329974s","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":63,"response count":0,"response size":29,"request content":"key:\"/registry/clusterroles/tigera-extension-apiserver-auth-access\" "} Jul 15 10:59:39.126273 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.119374Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"170.007878ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 10:59:39.126273 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.120021Z","caller":"traceutil/trace.go:171","msg":"trace[872508891] transaction","detail":"{read_only:false; response_revision:1427; number_of_response:1; }","duration":"2.260448308s","start":"2025-07-15T10:59:36.859542Z","end":"2025-07-15T10:59:39.119990Z","steps":["trace[872508891] 'process raft request' (duration: 2.08970613s)","trace[872508891] 'compare' (duration: 169.49843ms)"],"step_count":2} Jul 15 10:59:39.126449 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.120194Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:36.859493Z","time spent":"2.260592754s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":699,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:39.155775 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.153762Z","caller":"traceutil/trace.go:171","msg":"trace[1359152404] linearizableReadLoop","detail":"{readStateIndex:1711; appliedIndex:1704; }","duration":"2.199311167s","start":"2025-07-15T10:59:36.954427Z","end":"2025-07-15T10:59:39.153739Z","steps":["trace[1359152404] 'read index received' (duration: 1.994737113s)","trace[1359152404] 'applied index is now lower than readState.Index' (duration: 204.573383ms)"],"step_count":2} Jul 15 10:59:39.155775 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.153899Z","caller":"traceutil/trace.go:171","msg":"trace[1622355756] transaction","detail":"{read_only:false; response_revision:1430; number_of_response:1; }","duration":"1.737756728s","start":"2025-07-15T10:59:37.416115Z","end":"2025-07-15T10:59:39.153872Z","steps":["trace[1622355756] 'process raft request' (duration: 1.737504825s)"],"step_count":1} Jul 15 10:59:39.155775 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.154103Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:37.416094Z","time spent":"1.737880294s","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1140,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:39.155775 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.154610Z","caller":"traceutil/trace.go:171","msg":"trace[2059636786] transaction","detail":"{read_only:false; response_revision:1428; number_of_response:1; }","duration":"1.74146276s","start":"2025-07-15T10:59:37.413125Z","end":"2025-07-15T10:59:39.154588Z","steps":["trace[2059636786] 'process raft request' (duration: 1.740287164s)"],"step_count":1} Jul 15 10:59:39.157347 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.154912Z","caller":"traceutil/trace.go:171","msg":"trace[732659725] transaction","detail":"{read_only:false; response_revision:1431; number_of_response:1; }","duration":"1.010317763s","start":"2025-07-15T10:59:38.144579Z","end":"2025-07-15T10:59:39.154897Z","steps":["trace[732659725] 'process raft request' (duration: 1.009073766s)"],"step_count":1} Jul 15 10:59:39.157347 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.155027Z","caller":"traceutil/trace.go:171","msg":"trace[608310992] transaction","detail":"{read_only:false; response_revision:1429; number_of_response:1; }","duration":"1.739699627s","start":"2025-07-15T10:59:37.415300Z","end":"2025-07-15T10:59:39.154999Z","steps":["trace[608310992] 'process raft request' (duration: 1.738254263s)"],"step_count":1} Jul 15 10:59:39.157347 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.155153Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:37.415280Z","time spent":"1.739824214s","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1142,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:39.157347 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.155077Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:38.144549Z","time spent":"1.010487407s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:39.159053 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.155285Z","caller":"traceutil/trace.go:171","msg":"trace[1290655684] transaction","detail":"{read_only:false; response_revision:1432; number_of_response:1; }","duration":"1.000709556s","start":"2025-07-15T10:59:38.154565Z","end":"2025-07-15T10:59:39.155275Z","steps":["trace[1290655684] 'process raft request' (duration: 999.125496ms)"],"step_count":1} Jul 15 10:59:39.159053 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.155579Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:38.154519Z","time spent":"1.000954626s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":516,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:39.159053 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.154925Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:37.413096Z","time spent":"1.741580105s","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1183,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:39.159053 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.156156Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"486.609855ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:39.159255 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.156421Z","caller":"traceutil/trace.go:171","msg":"trace[228746770] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1433; }","duration":"486.927213ms","start":"2025-07-15T10:59:38.669482Z","end":"2025-07-15T10:59:39.156410Z","steps":["trace[228746770] 'agreement among raft nodes before linearized reading' (duration: 486.644801ms)"],"step_count":1} Jul 15 10:59:39.159255 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.156960Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:38.669449Z","time spent":"487.502508ms","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:59:39.159255 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.156964Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.941315563s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a\" ","response":"range_response_count:1 size:873"} Jul 15 10:59:39.159255 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.157045Z","caller":"traceutil/trace.go:171","msg":"trace[1439744313] range","detail":"{range_begin:/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a; range_end:; response_count:1; response_revision:1433; }","duration":"1.941426485s","start":"2025-07-15T10:59:37.215580Z","end":"2025-07-15T10:59:39.157007Z","steps":["trace[1439744313] 'agreement among raft nodes before linearized reading' (duration: 1.939759156s)"],"step_count":1} Jul 15 10:59:39.159255 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.156596Z","caller":"traceutil/trace.go:171","msg":"trace[1324835141] transaction","detail":"{read_only:false; response_revision:1433; number_of_response:1; }","duration":"988.963146ms","start":"2025-07-15T10:59:38.167623Z","end":"2025-07-15T10:59:39.156586Z","steps":["trace[1324835141] 'process raft request' (duration: 986.092257ms)"],"step_count":1} Jul 15 10:59:39.159450 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.157321Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:38.167602Z","time spent":"989.551626ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":488,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:39.159450 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.156907Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"201.426104ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:39.159450 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.157514Z","caller":"traceutil/trace.go:171","msg":"trace[571823945] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1433; }","duration":"201.904562ms","start":"2025-07-15T10:59:38.955456Z","end":"2025-07-15T10:59:39.157361Z","steps":["trace[571823945] 'agreement among raft nodes before linearized reading' (duration: 201.417227ms)"],"step_count":1} Jul 15 10:59:39.159450 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.157694Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:37.215551Z","time spent":"1.941535174s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":71,"response count":1,"response size":897,"request content":"key:\"/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a\" "} Jul 15 10:59:39.159674 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.157928Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"832.422347ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/validatingadmissionpolicies/\" range_end:\"/registry/validatingadmissionpolicies0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:59:39.159674 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.158007Z","caller":"traceutil/trace.go:171","msg":"trace[1775970555] range","detail":"{range_begin:/registry/validatingadmissionpolicies/; range_end:/registry/validatingadmissionpolicies0; response_count:0; response_revision:1433; }","duration":"832.554771ms","start":"2025-07-15T10:59:38.325435Z","end":"2025-07-15T10:59:39.157990Z","steps":["trace[1775970555] 'agreement among raft nodes before linearized reading' (duration: 832.397589ms)"],"step_count":1} Jul 15 10:59:39.159674 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.158066Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:38.325372Z","time spent":"832.675263ms","remote":"172.24.4.149:33528","response type":"/etcdserverpb.KV/Range","request count":0,"request size":82,"response count":0,"response size":29,"request content":"key:\"/registry/validatingadmissionpolicies/\" range_end:\"/registry/validatingadmissionpolicies0\" count_only:true "} Jul 15 10:59:39.159674 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.158301Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.515702832s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"range_response_count:1 size:886"} Jul 15 10:59:39.159864 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.158358Z","caller":"traceutil/trace.go:171","msg":"trace[1909672781] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; response_count:1; response_revision:1433; }","duration":"1.515808345s","start":"2025-07-15T10:59:37.642534Z","end":"2025-07-15T10:59:39.158343Z","steps":["trace[1909672781] 'agreement among raft nodes before linearized reading' (duration: 1.515667013s)"],"step_count":1} Jul 15 10:59:39.159864 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.158461Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:37.642510Z","time spent":"1.515932022s","remote":"172.24.4.149:54994","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":1,"response size":910,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" "} Jul 15 10:59:39.352214 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.347963Z","caller":"traceutil/trace.go:171","msg":"trace[397360039] transaction","detail":"{read_only:false; response_revision:1435; number_of_response:1; }","duration":"140.419919ms","start":"2025-07-15T10:59:39.207513Z","end":"2025-07-15T10:59:39.347933Z","steps":["trace[397360039] 'process raft request' (duration: 140.147867ms)"],"step_count":1} Jul 15 10:59:39.359773 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.358169Z","caller":"traceutil/trace.go:171","msg":"trace[969503325] transaction","detail":"{read_only:false; response_revision:1436; number_of_response:1; }","duration":"148.362512ms","start":"2025-07-15T10:59:39.209744Z","end":"2025-07-15T10:59:39.358106Z","steps":["trace[969503325] 'process raft request' (duration: 148.027519ms)"],"step_count":1} Jul 15 10:59:39.362142 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.359282Z","caller":"traceutil/trace.go:171","msg":"trace[1694385369] linearizableReadLoop","detail":"{readStateIndex:1715; appliedIndex:1714; }","duration":"135.417369ms","start":"2025-07-15T10:59:39.223842Z","end":"2025-07-15T10:59:39.359260Z","steps":["trace[1694385369] 'read index received' (duration: 124.243005ms)","trace[1694385369] 'applied index is now lower than readState.Index' (duration: 11.173111ms)"],"step_count":2} Jul 15 10:59:39.363753 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.360155Z","caller":"traceutil/trace.go:171","msg":"trace[1693603214] transaction","detail":"{read_only:false; response_revision:1437; number_of_response:1; }","duration":"122.318582ms","start":"2025-07-15T10:59:39.237815Z","end":"2025-07-15T10:59:39.360133Z","steps":["trace[1693603214] 'process raft request' (duration: 121.252546ms)"],"step_count":1} Jul 15 10:59:39.363753 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.360292Z","caller":"traceutil/trace.go:171","msg":"trace[1288143381] transaction","detail":"{read_only:false; number_of_response:0; response_revision:1436; }","duration":"130.801601ms","start":"2025-07-15T10:59:39.229478Z","end":"2025-07-15T10:59:39.360280Z","steps":["trace[1288143381] 'process raft request' (duration: 128.531575ms)"],"step_count":1} Jul 15 10:59:39.363753 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.361146Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"137.094406ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/tigerastatuses/apiserver\" ","response":"range_response_count:1 size:1198"} Jul 15 10:59:39.363753 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.361233Z","caller":"traceutil/trace.go:171","msg":"trace[1538089225] range","detail":"{range_begin:/registry/operator.tigera.io/tigerastatuses/apiserver; range_end:; response_count:1; response_revision:1437; }","duration":"137.217382ms","start":"2025-07-15T10:59:39.223990Z","end":"2025-07-15T10:59:39.361207Z","steps":["trace[1538089225] 'agreement among raft nodes before linearized reading' (duration: 136.936544ms)"],"step_count":1} Jul 15 10:59:39.363753 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.361551Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"137.7005ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/tigerastatuses/whisker\" ","response":"range_response_count:1 size:1155"} Jul 15 10:59:39.364515 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.361640Z","caller":"traceutil/trace.go:171","msg":"trace[294842017] range","detail":"{range_begin:/registry/operator.tigera.io/tigerastatuses/whisker; range_end:; response_count:1; response_revision:1437; }","duration":"137.791104ms","start":"2025-07-15T10:59:39.223796Z","end":"2025-07-15T10:59:39.361587Z","steps":["trace[294842017] 'agreement among raft nodes before linearized reading' (duration: 137.65352ms)"],"step_count":1} Jul 15 10:59:39.366761 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.362710Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"128.10183ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/tigera-webhook-reader\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:39.366761 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.362803Z","caller":"traceutil/trace.go:171","msg":"trace[2066698995] range","detail":"{range_begin:/registry/clusterroles/tigera-webhook-reader; range_end:; response_count:0; response_revision:1437; }","duration":"128.214716ms","start":"2025-07-15T10:59:39.234568Z","end":"2025-07-15T10:59:39.362783Z","steps":["trace[2066698995] 'agreement among raft nodes before linearized reading' (duration: 128.070679ms)"],"step_count":1} Jul 15 10:59:39.366761 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.364848Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"131.367887ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/deployments/kube-system/calico-kube-controllers\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:39.367348 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.366373Z","caller":"traceutil/trace.go:171","msg":"trace[1339905209] range","detail":"{range_begin:/registry/deployments/kube-system/calico-kube-controllers; range_end:; response_count:0; response_revision:1437; }","duration":"132.940525ms","start":"2025-07-15T10:59:39.233407Z","end":"2025-07-15T10:59:39.366348Z","steps":["trace[1339905209] 'agreement among raft nodes before linearized reading' (duration: 131.346856ms)"],"step_count":1} Jul 15 10:59:39.818039 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.817064Z","caller":"traceutil/trace.go:171","msg":"trace[1401914335] linearizableReadLoop","detail":"{readStateIndex:1726; appliedIndex:1725; }","duration":"104.551524ms","start":"2025-07-15T10:59:39.712480Z","end":"2025-07-15T10:59:39.817032Z","steps":["trace[1401914335] 'read index received' (duration: 88.301249ms)","trace[1401914335] 'applied index is now lower than readState.Index' (duration: 16.208475ms)"],"step_count":2} Jul 15 10:59:39.819696 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.818559Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"106.051411ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/calico-system/whisker\" ","response":"range_response_count:1 size:426"} Jul 15 10:59:39.819696 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.818642Z","caller":"traceutil/trace.go:171","msg":"trace[872089858] range","detail":"{range_begin:/registry/serviceaccounts/calico-system/whisker; range_end:; response_count:1; response_revision:1444; }","duration":"106.147356ms","start":"2025-07-15T10:59:39.712474Z","end":"2025-07-15T10:59:39.818621Z","steps":["trace[872089858] 'agreement among raft nodes before linearized reading' (duration: 105.81021ms)"],"step_count":1} Jul 15 10:59:39.819696 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.818880Z","caller":"traceutil/trace.go:171","msg":"trace[413558212] transaction","detail":"{read_only:false; response_revision:1444; number_of_response:1; }","duration":"107.812201ms","start":"2025-07-15T10:59:39.709937Z","end":"2025-07-15T10:59:39.817749Z","steps":["trace[413558212] 'process raft request' (duration: 106.79064ms)"],"step_count":1} Jul 15 10:59:39.891249 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.890304Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"163.706156ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/calico-system/calico-typha\" ","response":"range_response_count:1 size:441"} Jul 15 10:59:39.892245 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.890754Z","caller":"traceutil/trace.go:171","msg":"trace[1520361946] range","detail":"{range_begin:/registry/serviceaccounts/calico-system/calico-typha; range_end:; response_count:1; response_revision:1445; }","duration":"164.178513ms","start":"2025-07-15T10:59:39.726541Z","end":"2025-07-15T10:59:39.890719Z","steps":["trace[1520361946] 'agreement among raft nodes before linearized reading' (duration: 163.117557ms)"],"step_count":1} Jul 15 10:59:39.893157 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:39.892454Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"178.835522ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/roles/calico-system/goldmane\" ","response":"range_response_count:1 size:531"} Jul 15 10:59:39.893157 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:39.892635Z","caller":"traceutil/trace.go:171","msg":"trace[1552577563] range","detail":"{range_begin:/registry/roles/calico-system/goldmane; range_end:; response_count:1; response_revision:1445; }","duration":"179.097914ms","start":"2025-07-15T10:59:39.713474Z","end":"2025-07-15T10:59:39.892572Z","steps":["trace[1552577563] 'agreement among raft nodes before linearized reading' (duration: 177.095422ms)"],"step_count":1} Jul 15 10:59:40.188481 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:40.185873Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"123.973894ms","expected-duration":"100ms","prefix":"","request":"header: lease_grant:","response":"size:41"} Jul 15 10:59:40.188481 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:40.186022Z","caller":"traceutil/trace.go:171","msg":"trace[973266412] linearizableReadLoop","detail":"{readStateIndex:1728; appliedIndex:1727; }","duration":"180.764373ms","start":"2025-07-15T10:59:40.005234Z","end":"2025-07-15T10:59:40.185999Z","steps":["trace[973266412] 'read index received' (duration: 56.570377ms)","trace[973266412] 'applied index is now lower than readState.Index' (duration: 124.191902ms)"],"step_count":2} Jul 15 10:59:40.188481 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:40.186372Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"181.093887ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/calico-apiserver-delegate-auth\" ","response":"range_response_count:1 size:661"} Jul 15 10:59:40.188481 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:40.186464Z","caller":"traceutil/trace.go:171","msg":"trace[501490268] range","detail":"{range_begin:/registry/clusterrolebindings/calico-apiserver-delegate-auth; range_end:; response_count:1; response_revision:1445; }","duration":"181.250897ms","start":"2025-07-15T10:59:40.005195Z","end":"2025-07-15T10:59:40.186446Z","steps":["trace[501490268] 'agreement among raft nodes before linearized reading' (duration: 180.941774ms)"],"step_count":1} Jul 15 10:59:40.188481 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:40.186481Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"181.11194ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/calico-system/calico-node\" ","response":"range_response_count:1 size:525"} Jul 15 10:59:40.189114 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:40.186543Z","caller":"traceutil/trace.go:171","msg":"trace[1935862822] range","detail":"{range_begin:/registry/serviceaccounts/calico-system/calico-node; range_end:; response_count:1; response_revision:1445; }","duration":"181.196994ms","start":"2025-07-15T10:59:40.005328Z","end":"2025-07-15T10:59:40.186525Z","steps":["trace[1935862822] 'agreement among raft nodes before linearized reading' (duration: 180.9554ms)"],"step_count":1} Jul 15 10:59:40.189114 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:40.186700Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"112.469894ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/ippools/default-ipv4-ippool\" ","response":"range_response_count:1 size:945"} Jul 15 10:59:40.189114 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:40.186741Z","caller":"traceutil/trace.go:171","msg":"trace[1738509478] range","detail":"{range_begin:/registry/crd.projectcalico.org/ippools/default-ipv4-ippool; range_end:; response_count:1; response_revision:1445; }","duration":"112.559347ms","start":"2025-07-15T10:59:40.074168Z","end":"2025-07-15T10:59:40.186727Z","steps":["trace[1738509478] 'agreement among raft nodes before linearized reading' (duration: 112.453493ms)"],"step_count":1} Jul 15 10:59:42.565716 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:42.564637Z","caller":"traceutil/trace.go:171","msg":"trace[155394524] linearizableReadLoop","detail":"{readStateIndex:1738; appliedIndex:1737; }","duration":"308.766892ms","start":"2025-07-15T10:59:42.255845Z","end":"2025-07-15T10:59:42.564612Z","steps":["trace[155394524] 'read index received' (duration: 308.465814ms)","trace[155394524] 'applied index is now lower than readState.Index' (duration: 300.076µs)"],"step_count":2} Jul 15 10:59:42.565716 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:42.564858Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"308.988708ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/calico-crds\" ","response":"range_response_count:1 size:1255"} Jul 15 10:59:42.565716 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:42.564908Z","caller":"traceutil/trace.go:171","msg":"trace[1539924292] range","detail":"{range_begin:/registry/clusterroles/calico-crds; range_end:; response_count:1; response_revision:1455; }","duration":"309.077378ms","start":"2025-07-15T10:59:42.255801Z","end":"2025-07-15T10:59:42.564878Z","steps":["trace[1539924292] 'agreement among raft nodes before linearized reading' (duration: 308.91154ms)"],"step_count":1} Jul 15 10:59:42.565716 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:42.564942Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:42.255747Z","time spent":"309.184604ms","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":36,"response count":1,"response size":1279,"request content":"key:\"/registry/clusterroles/calico-crds\" "} Jul 15 10:59:42.565716 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:42.565110Z","caller":"traceutil/trace.go:171","msg":"trace[330898766] transaction","detail":"{read_only:false; response_revision:1455; number_of_response:1; }","duration":"349.153936ms","start":"2025-07-15T10:59:42.215945Z","end":"2025-07-15T10:59:42.565099Z","steps":["trace[330898766] 'process raft request' (duration: 348.487957ms)"],"step_count":1} Jul 15 10:59:42.566575 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:42.565196Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:42.215917Z","time spent":"349.221816ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":858,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:47.353521 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:47.350671Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075407,"retry-timeout":"500ms"} Jul 15 10:59:48.036167 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:47.973492Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075407,"retry-timeout":"500ms"} Jul 15 10:59:48.109935 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.094718Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"4.289419846s","expected-duration":"1s"} Jul 15 10:59:48.111683 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.096511Z","caller":"traceutil/trace.go:171","msg":"trace[786197052] transaction","detail":"{read_only:false; response_revision:1459; number_of_response:1; }","duration":"4.291288226s","start":"2025-07-15T10:59:43.805083Z","end":"2025-07-15T10:59:48.096371Z","steps":["trace[786197052] 'process raft request' (duration: 4.290420488s)"],"step_count":1} Jul 15 10:59:48.111683 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.108519Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:43.805045Z","time spent":"4.291728981s","remote":"172.24.4.149:54882","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1521,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:48.120171 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.118434Z","caller":"traceutil/trace.go:171","msg":"trace[964340612] transaction","detail":"{read_only:false; response_revision:1460; number_of_response:1; }","duration":"1.435488537s","start":"2025-07-15T10:59:46.682837Z","end":"2025-07-15T10:59:48.118326Z","steps":["trace[964340612] 'process raft request' (duration: 1.434800386s)"],"step_count":1} Jul 15 10:59:48.120171 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.118714Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:46.682731Z","time spent":"1.43581223s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":490,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:48.120171 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.118862Z","caller":"traceutil/trace.go:171","msg":"trace[1924168967] transaction","detail":"{read_only:false; response_revision:1461; number_of_response:1; }","duration":"1.435587598s","start":"2025-07-15T10:59:46.683234Z","end":"2025-07-15T10:59:48.118822Z","steps":["trace[1924168967] 'process raft request' (duration: 1.434940304s)"],"step_count":1} Jul 15 10:59:48.120171 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.119145Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:46.683218Z","time spent":"1.435736374s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":517,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:48.122701 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.119233Z","caller":"traceutil/trace.go:171","msg":"trace[1288625413] transaction","detail":"{read_only:false; response_revision:1462; number_of_response:1; }","duration":"1.435878746s","start":"2025-07-15T10:59:46.683335Z","end":"2025-07-15T10:59:48.119213Z","steps":["trace[1288625413] 'process raft request' (duration: 1.435334732s)"],"step_count":1} Jul 15 10:59:48.122701 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.119358Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:46.683319Z","time spent":"1.435966806s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":532,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:48.122701 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.121066Z","caller":"traceutil/trace.go:171","msg":"trace[805665605] transaction","detail":"{read_only:false; response_revision:1463; number_of_response:1; }","duration":"1.437560949s","start":"2025-07-15T10:59:46.683482Z","end":"2025-07-15T10:59:48.121043Z","steps":["trace[805665605] 'process raft request' (duration: 1.435638025s)"],"step_count":1} Jul 15 10:59:48.122701 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.121755Z","caller":"traceutil/trace.go:171","msg":"trace[775304915] linearizableReadLoop","detail":"{readStateIndex:1747; appliedIndex:1741; }","duration":"1.439737218s","start":"2025-07-15T10:59:46.681956Z","end":"2025-07-15T10:59:48.121693Z","steps":["trace[775304915] 'read index received' (duration: 1.414331714s)","trace[775304915] 'applied index is now lower than readState.Index' (duration: 25.402809ms)"],"step_count":2} Jul 15 10:59:48.124612 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.122503Z","caller":"traceutil/trace.go:171","msg":"trace[791749339] transaction","detail":"{read_only:false; response_revision:1464; number_of_response:1; }","duration":"1.43888959s","start":"2025-07-15T10:59:46.683592Z","end":"2025-07-15T10:59:48.122482Z","steps":["trace[791749339] 'process raft request' (duration: 1.437326558s)"],"step_count":1} Jul 15 10:59:48.124612 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.122637Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:46.683572Z","time spent":"1.438989994s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:48.124612 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.122798Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.440864806s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:48.124612 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.122880Z","caller":"traceutil/trace.go:171","msg":"trace[1533125088] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1464; }","duration":"1.441244213s","start":"2025-07-15T10:59:46.681611Z","end":"2025-07-15T10:59:48.122855Z","steps":["trace[1533125088] 'agreement among raft nodes before linearized reading' (duration: 1.441111237s)"],"step_count":1} Jul 15 10:59:48.126168 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.122923Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:46.681479Z","time spent":"1.441430511s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:59:48.126168 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.123485Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.440485996s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/daemonsets/calico-system/calico-node\" ","response":"range_response_count:1 size:11440"} Jul 15 10:59:48.126168 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.123538Z","caller":"traceutil/trace.go:171","msg":"trace[835728381] range","detail":"{range_begin:/registry/daemonsets/calico-system/calico-node; range_end:; response_count:1; response_revision:1464; }","duration":"1.440577563s","start":"2025-07-15T10:59:46.682946Z","end":"2025-07-15T10:59:48.123523Z","steps":["trace[835728381] 'agreement among raft nodes before linearized reading' (duration: 1.440048467s)"],"step_count":1} Jul 15 10:59:48.126168 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.123580Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:46.682921Z","time spent":"1.440645614s","remote":"172.24.4.149:33488","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":1,"response size":11464,"request content":"key:\"/registry/daemonsets/calico-system/calico-node\" "} Jul 15 10:59:48.127141 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.123577Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:46.683459Z","time spent":"1.43820711s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":625,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:48.127141 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.124425Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.441290202s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a\" ","response":"range_response_count:1 size:873"} Jul 15 10:59:48.127141 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.124856Z","caller":"traceutil/trace.go:171","msg":"trace[1256149059] range","detail":"{range_begin:/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a; range_end:; response_count:1; response_revision:1464; }","duration":"1.441394442s","start":"2025-07-15T10:59:46.683083Z","end":"2025-07-15T10:59:48.124477Z","steps":["trace[1256149059] 'agreement among raft nodes before linearized reading' (duration: 1.440670281s)"],"step_count":1} Jul 15 10:59:48.127141 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.124944Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:46.683045Z","time spent":"1.441871729s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":71,"response count":1,"response size":897,"request content":"key:\"/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a\" "} Jul 15 10:59:48.284811 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.283860Z","caller":"traceutil/trace.go:171","msg":"trace[1459871161] transaction","detail":"{read_only:false; response_revision:1465; number_of_response:1; }","duration":"147.692147ms","start":"2025-07-15T10:59:48.136149Z","end":"2025-07-15T10:59:48.283841Z","steps":["trace[1459871161] 'process raft request' (duration: 59.508987ms)","trace[1459871161] 'compare' (duration: 87.898263ms)"],"step_count":2} Jul 15 10:59:48.285675 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.283802Z","caller":"traceutil/trace.go:171","msg":"trace[417130510] linearizableReadLoop","detail":"{readStateIndex:1748; appliedIndex:1747; }","duration":"147.2306ms","start":"2025-07-15T10:59:48.136545Z","end":"2025-07-15T10:59:48.283776Z","steps":["trace[417130510] 'read index received' (duration: 59.011512ms)","trace[417130510] 'applied index is now lower than readState.Index' (duration: 88.217876ms)"],"step_count":2} Jul 15 10:59:48.285675 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.284578Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"148.02687ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:48.285675 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.284737Z","caller":"traceutil/trace.go:171","msg":"trace[677424747] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1465; }","duration":"148.254467ms","start":"2025-07-15T10:59:48.136470Z","end":"2025-07-15T10:59:48.284724Z","steps":["trace[677424747] 'agreement among raft nodes before linearized reading' (duration: 147.936937ms)"],"step_count":1} Jul 15 10:59:48.285675 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.284859Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"105.912478ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/poddisruptionbudgets/\" range_end:\"/registry/poddisruptionbudgets0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 10:59:48.285675 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.284882Z","caller":"traceutil/trace.go:171","msg":"trace[1722353994] range","detail":"{range_begin:/registry/poddisruptionbudgets/; range_end:/registry/poddisruptionbudgets0; response_count:0; response_revision:1465; }","duration":"105.990598ms","start":"2025-07-15T10:59:48.178883Z","end":"2025-07-15T10:59:48.284873Z","steps":["trace[1722353994] 'agreement among raft nodes before linearized reading' (duration: 105.944449ms)"],"step_count":1} Jul 15 10:59:48.286696 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.284992Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"126.192416ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 10:59:48.286696 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.285012Z","caller":"traceutil/trace.go:171","msg":"trace[735212767] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1465; }","duration":"126.213647ms","start":"2025-07-15T10:59:48.158792Z","end":"2025-07-15T10:59:48.285006Z","steps":["trace[735212767] 'agreement among raft nodes before linearized reading' (duration: 126.144003ms)"],"step_count":1} Jul 15 10:59:48.286696 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:48.285253Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"131.803965ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 10:59:48.286696 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:48.285276Z","caller":"traceutil/trace.go:171","msg":"trace[1813602844] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1465; }","duration":"131.922894ms","start":"2025-07-15T10:59:48.153347Z","end":"2025-07-15T10:59:48.285270Z","steps":["trace[1813602844] 'agreement among raft nodes before linearized reading' (duration: 131.273758ms)"],"step_count":1} Jul 15 10:59:51.015667 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:50.837677Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075530,"retry-timeout":"500ms"} Jul 15 10:59:51.162279 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.161426Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"1.852447169s","expected-duration":"1s"} Jul 15 10:59:51.163523 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.162603Z","caller":"traceutil/trace.go:171","msg":"trace[105305469] transaction","detail":"{read_only:false; response_revision:1468; number_of_response:1; }","duration":"1.853391132s","start":"2025-07-15T10:59:49.308671Z","end":"2025-07-15T10:59:51.162062Z","steps":["trace[105305469] 'process raft request' (duration: 1.853190926s)"],"step_count":1} Jul 15 10:59:51.164695 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.163312Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:49.308641Z","time spent":"1.854165189s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":858,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:51.209589 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.207163Z","caller":"traceutil/trace.go:171","msg":"trace[1471834853] transaction","detail":"{read_only:false; response_revision:1470; number_of_response:1; }","duration":"870.227966ms","start":"2025-07-15T10:59:50.336905Z","end":"2025-07-15T10:59:51.207133Z","steps":["trace[1471834853] 'process raft request' (duration: 870.127143ms)"],"step_count":1} Jul 15 10:59:51.210629 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.207504Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:50.336856Z","time spent":"870.410397ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":488,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:51.210629 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.207457Z","caller":"traceutil/trace.go:171","msg":"trace[451012875] transaction","detail":"{read_only:false; response_revision:1469; number_of_response:1; }","duration":"1.723547145s","start":"2025-07-15T10:59:49.483660Z","end":"2025-07-15T10:59:51.207207Z","steps":["trace[451012875] 'process raft request' (duration: 1.72320625s)"],"step_count":1} Jul 15 10:59:51.210629 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.208017Z","caller":"traceutil/trace.go:171","msg":"trace[1299988850] transaction","detail":"{read_only:false; response_revision:1471; number_of_response:1; }","duration":"870.929836ms","start":"2025-07-15T10:59:50.337065Z","end":"2025-07-15T10:59:51.207994Z","steps":["trace[1299988850] 'process raft request' (duration: 870.02111ms)"],"step_count":1} Jul 15 10:59:51.210629 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.208120Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:50.337049Z","time spent":"871.016602ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:51.211982 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.208156Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:49.483628Z","time spent":"1.724371038s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":699,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:51.211982 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.208363Z","caller":"traceutil/trace.go:171","msg":"trace[231030184] linearizableReadLoop","detail":"{readStateIndex:1756; appliedIndex:1751; }","duration":"871.732307ms","start":"2025-07-15T10:59:50.336613Z","end":"2025-07-15T10:59:51.208345Z","steps":["trace[231030184] 'read index received' (duration: 826.448211ms)","trace[231030184] 'applied index is now lower than readState.Index' (duration: 45.283104ms)"],"step_count":2} Jul 15 10:59:51.211982 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.208776Z","caller":"traceutil/trace.go:171","msg":"trace[905852432] transaction","detail":"{read_only:false; response_revision:1472; number_of_response:1; }","duration":"871.604221ms","start":"2025-07-15T10:59:50.337149Z","end":"2025-07-15T10:59:51.208753Z","steps":["trace[905852432] 'process raft request' (duration: 870.791549ms)"],"step_count":1} Jul 15 10:59:51.212438 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.209007Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:50.337137Z","time spent":"871.804898ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":516,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 10:59:51.212438 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.209032Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"872.391304ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/persistentvolumeclaims/\" range_end:\"/registry/persistentvolumeclaims0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 10:59:51.212438 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.209100Z","caller":"traceutil/trace.go:171","msg":"trace[1749777811] range","detail":"{range_begin:/registry/persistentvolumeclaims/; range_end:/registry/persistentvolumeclaims0; response_count:0; response_revision:1472; }","duration":"872.509341ms","start":"2025-07-15T10:59:50.336570Z","end":"2025-07-15T10:59:51.209079Z","steps":["trace[1749777811] 'agreement among raft nodes before linearized reading' (duration: 872.394891ms)"],"step_count":1} Jul 15 10:59:51.212438 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.209158Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:50.336547Z","time spent":"872.588623ms","remote":"172.24.4.149:33190","response type":"/etcdserverpb.KV/Range","request count":0,"request size":72,"response count":0,"response size":29,"request content":"key:\"/registry/persistentvolumeclaims/\" range_end:\"/registry/persistentvolumeclaims0\" count_only:true "} Jul 15 10:59:51.213998 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.209828Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"872.710417ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/\" range_end:\"/registry/events0\" count_only:true ","response":"range_response_count:0 size:8"} Jul 15 10:59:51.213998 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.209909Z","caller":"traceutil/trace.go:171","msg":"trace[897452906] range","detail":"{range_begin:/registry/events/; range_end:/registry/events0; response_count:0; response_revision:1472; }","duration":"872.817302ms","start":"2025-07-15T10:59:50.337075Z","end":"2025-07-15T10:59:51.209893Z","steps":["trace[897452906] 'agreement among raft nodes before linearized reading' (duration: 872.362758ms)"],"step_count":1} Jul 15 10:59:51.213998 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.209965Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:50.337067Z","time spent":"872.874563ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":40,"response count":224,"response size":32,"request content":"key:\"/registry/events/\" range_end:\"/registry/events0\" count_only:true "} Jul 15 10:59:51.213998 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.211274Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"195.144486ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/daemonsets/calico-system/calico-node\" ","response":"range_response_count:1 size:11440"} Jul 15 10:59:51.213998 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.211348Z","caller":"traceutil/trace.go:171","msg":"trace[1410139815] range","detail":"{range_begin:/registry/daemonsets/calico-system/calico-node; range_end:; response_count:1; response_revision:1472; }","duration":"195.278074ms","start":"2025-07-15T10:59:51.016053Z","end":"2025-07-15T10:59:51.211331Z","steps":["trace[1410139815] 'agreement among raft nodes before linearized reading' (duration: 195.029034ms)"],"step_count":1} Jul 15 10:59:51.214567 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.209372Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"872.633069ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 10:59:51.214567 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.211893Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"874.932137ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"range_response_count:1 size:886"} Jul 15 10:59:51.214567 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.212786Z","caller":"traceutil/trace.go:171","msg":"trace[2028714594] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; response_count:1; response_revision:1472; }","duration":"875.83378ms","start":"2025-07-15T10:59:50.336921Z","end":"2025-07-15T10:59:51.212755Z","steps":["trace[2028714594] 'agreement among raft nodes before linearized reading' (duration: 874.818308ms)"],"step_count":1} Jul 15 10:59:51.214567 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.212926Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:50.336910Z","time spent":"875.98595ms","remote":"172.24.4.149:54994","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":1,"response size":910,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" "} Jul 15 10:59:51.214567 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.211972Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"875.180965ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:51.215711 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.213099Z","caller":"traceutil/trace.go:171","msg":"trace[2128124999] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1472; }","duration":"876.310625ms","start":"2025-07-15T10:59:50.336770Z","end":"2025-07-15T10:59:51.213081Z","steps":["trace[2128124999] 'agreement among raft nodes before linearized reading' (duration: 875.173359ms)"],"step_count":1} Jul 15 10:59:51.215711 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.213150Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:50.336764Z","time spent":"876.369198ms","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 10:59:51.215711 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.214628Z","caller":"traceutil/trace.go:171","msg":"trace[106663402] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1472; }","duration":"875.868885ms","start":"2025-07-15T10:59:50.336707Z","end":"2025-07-15T10:59:51.212576Z","steps":["trace[106663402] 'agreement among raft nodes before linearized reading' (duration: 872.466027ms)"],"step_count":1} Jul 15 10:59:51.215711 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.214817Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T10:59:50.336701Z","time spent":"878.057031ms","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 10:59:51.374272 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.373261Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"114.184776ms","expected-duration":"100ms","prefix":"","request":"header: lease_grant:","response":"size:41"} Jul 15 10:59:51.390909 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.390004Z","caller":"traceutil/trace.go:171","msg":"trace[1234712657] linearizableReadLoop","detail":"{readStateIndex:1759; appliedIndex:1756; }","duration":"144.131209ms","start":"2025-07-15T10:59:51.245835Z","end":"2025-07-15T10:59:51.389966Z","steps":["trace[1234712657] 'read index received' (duration: 13.230272ms)","trace[1234712657] 'applied index is now lower than readState.Index' (duration: 130.899545ms)"],"step_count":2} Jul 15 10:59:51.391426 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.390617Z","caller":"traceutil/trace.go:171","msg":"trace[57287176] transaction","detail":"{read_only:false; response_revision:1473; number_of_response:1; }","duration":"166.503229ms","start":"2025-07-15T10:59:51.224086Z","end":"2025-07-15T10:59:51.390589Z","steps":["trace[57287176] 'process raft request' (duration: 165.579454ms)"],"step_count":1} Jul 15 10:59:51.392345 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:51.391544Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"145.676069ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 10:59:51.392345 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.391641Z","caller":"traceutil/trace.go:171","msg":"trace[2042381774] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1473; }","duration":"145.81676ms","start":"2025-07-15T10:59:51.245788Z","end":"2025-07-15T10:59:51.391605Z","steps":["trace[2042381774] 'agreement among raft nodes before linearized reading' (duration: 144.330864ms)"],"step_count":1} Jul 15 10:59:51.392345 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:51.391794Z","caller":"traceutil/trace.go:171","msg":"trace[1639345111] transaction","detail":"{read_only:false; number_of_response:0; response_revision:1473; }","duration":"151.138054ms","start":"2025-07-15T10:59:51.240618Z","end":"2025-07-15T10:59:51.391756Z","steps":["trace[1639345111] 'process raft request' (duration: 149.267931ms)"],"step_count":1} Jul 15 10:59:55.717107 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:55.715496Z","caller":"traceutil/trace.go:171","msg":"trace[959757436] transaction","detail":"{read_only:false; response_revision:1489; number_of_response:1; }","duration":"153.575533ms","start":"2025-07-15T10:59:55.561592Z","end":"2025-07-15T10:59:55.715168Z","steps":["trace[959757436] 'process raft request' (duration: 151.777207ms)"],"step_count":1} Jul 15 10:59:58.992448 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T10:59:58.991091Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"142.253829ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 10:59:58.992448 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T10:59:58.991305Z","caller":"traceutil/trace.go:171","msg":"trace[225708527] transaction","detail":"{read_only:false; response_revision:1499; number_of_response:1; }","duration":"200.475181ms","start":"2025-07-15T10:59:58.790785Z","end":"2025-07-15T10:59:58.991261Z","steps":["trace[225708527] 'process raft request' (duration: 57.464395ms)","trace[225708527] 'compare' (duration: 141.960677ms)"],"step_count":2} Jul 15 11:00:10.005450 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:10.004316Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075894,"retry-timeout":"500ms"} Jul 15 11:00:10.506325 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:10.505530Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075894,"retry-timeout":"500ms"} Jul 15 11:00:11.007766 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.006898Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075894,"retry-timeout":"500ms"} Jul 15 11:00:11.508372 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.507740Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075894,"retry-timeout":"500ms"} Jul 15 11:00:11.767957 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.766764Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"2.397711101s","expected-duration":"1s"} Jul 15 11:00:11.769684 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.768537Z","caller":"traceutil/trace.go:171","msg":"trace[515798944] transaction","detail":"{read_only:false; response_revision:1525; number_of_response:1; }","duration":"2.399661956s","start":"2025-07-15T11:00:09.368823Z","end":"2025-07-15T11:00:11.768485Z","steps":["trace[515798944] 'process raft request' (duration: 2.398528133s)"],"step_count":1} Jul 15 11:00:11.769684 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.768978Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:09.368780Z","time spent":"2.399938768s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":533,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:00:11.876366 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.874719Z","caller":"traceutil/trace.go:171","msg":"trace[1822891664] transaction","detail":"{read_only:false; response_revision:1526; number_of_response:1; }","duration":"2.503983529s","start":"2025-07-15T11:00:09.370660Z","end":"2025-07-15T11:00:11.874644Z","steps":["trace[1822891664] 'process raft request' (duration: 2.485993652s)","trace[1822891664] 'compare' (duration: 17.479224ms)"],"step_count":2} Jul 15 11:00:11.878323 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.874973Z","caller":"traceutil/trace.go:171","msg":"trace[1646262158] linearizableReadLoop","detail":"{readStateIndex:1825; appliedIndex:1819; }","duration":"2.370884093s","start":"2025-07-15T11:00:09.503929Z","end":"2025-07-15T11:00:11.874813Z","steps":["trace[1646262158] 'read index received' (duration: 2.264367496s)","trace[1646262158] 'applied index is now lower than readState.Index' (duration: 106.515565ms)"],"step_count":2} Jul 15 11:00:11.878323 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.875640Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:09.370592Z","time spent":"2.504408477s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":858,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:00:11.878323 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.876028Z","caller":"traceutil/trace.go:171","msg":"trace[910792437] transaction","detail":"{read_only:false; response_revision:1527; number_of_response:1; }","duration":"2.504720087s","start":"2025-07-15T11:00:09.371288Z","end":"2025-07-15T11:00:11.876008Z","steps":["trace[910792437] 'process raft request' (duration: 2.503026197s)"],"step_count":1} Jul 15 11:00:11.878882 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.876188Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:09.371263Z","time spent":"2.504826782s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":624,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:00:11.878882 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.876334Z","caller":"traceutil/trace.go:171","msg":"trace[1319191050] transaction","detail":"{read_only:false; response_revision:1528; number_of_response:1; }","duration":"2.391841396s","start":"2025-07-15T11:00:09.484478Z","end":"2025-07-15T11:00:11.876319Z","steps":["trace[1319191050] 'process raft request' (duration: 2.390094604s)"],"step_count":1} Jul 15 11:00:11.878882 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.876563Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:09.484432Z","time spent":"2.392008839s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:00:11.879643 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.876668Z","caller":"traceutil/trace.go:171","msg":"trace[1030390474] transaction","detail":"{read_only:false; response_revision:1529; number_of_response:1; }","duration":"2.384728146s","start":"2025-07-15T11:00:09.491925Z","end":"2025-07-15T11:00:11.876653Z","steps":["trace[1030390474] 'process raft request' (duration: 2.382721814s)"],"step_count":1} Jul 15 11:00:11.879643 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.876776Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:09.491896Z","time spent":"2.384816286s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":490,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:00:11.879643 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.876906Z","caller":"traceutil/trace.go:171","msg":"trace[720776669] transaction","detail":"{read_only:false; response_revision:1530; number_of_response:1; }","duration":"2.378133244s","start":"2025-07-15T11:00:09.498710Z","end":"2025-07-15T11:00:11.876843Z","steps":["trace[720776669] 'process raft request' (duration: 2.376009907s)"],"step_count":1} Jul 15 11:00:11.879643 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.877038Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:09.498658Z","time spent":"2.378320005s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":517,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:00:11.880133 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.877138Z","caller":"traceutil/trace.go:171","msg":"trace[1019852865] transaction","detail":"{read_only:false; response_revision:1531; number_of_response:1; }","duration":"592.49261ms","start":"2025-07-15T11:00:11.284605Z","end":"2025-07-15T11:00:11.877098Z","steps":["trace[1019852865] 'process raft request' (duration: 590.15956ms)"],"step_count":1} Jul 15 11:00:11.880133 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.877225Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:11.284568Z","time spent":"592.610116ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":699,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:00:11.880133 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.877299Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.373367745s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions\" limit:1 ","response":"range_response_count:0 size:5"} Jul 15 11:00:11.880133 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.877779Z","caller":"traceutil/trace.go:171","msg":"trace[862251446] range","detail":"{range_begin:/registry/minions; range_end:; response_count:0; response_revision:1531; }","duration":"2.373481483s","start":"2025-07-15T11:00:09.503887Z","end":"2025-07-15T11:00:11.877369Z","steps":["trace[862251446] 'agreement among raft nodes before linearized reading' (duration: 2.373383504s)"],"step_count":1} Jul 15 11:00:11.883132 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.878099Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:09.503865Z","time spent":"2.374206989s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Range","request count":0,"request size":21,"response count":0,"response size":29,"request content":"key:\"/registry/minions\" limit:1 "} Jul 15 11:00:11.883132 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.879035Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"915.271747ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:00:11.883132 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.879194Z","caller":"traceutil/trace.go:171","msg":"trace[924414573] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1531; }","duration":"915.493805ms","start":"2025-07-15T11:00:10.963641Z","end":"2025-07-15T11:00:11.879135Z","steps":["trace[924414573] 'agreement among raft nodes before linearized reading' (duration: 915.185992ms)"],"step_count":1} Jul 15 11:00:11.883132 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.879522Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:10.963579Z","time spent":"915.75599ms","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 11:00:11.883132 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.880059Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"141.324628ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"range_response_count:1 size:886"} Jul 15 11:00:11.883624 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.880177Z","caller":"traceutil/trace.go:171","msg":"trace[1747395397] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; response_count:1; response_revision:1531; }","duration":"141.555413ms","start":"2025-07-15T11:00:11.738603Z","end":"2025-07-15T11:00:11.880158Z","steps":["trace[1747395397] 'agreement among raft nodes before linearized reading' (duration: 141.131216ms)"],"step_count":1} Jul 15 11:00:11.883624 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.880550Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"147.785354ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/configmaps/\" range_end:\"/registry/configmaps0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:11.883624 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.880679Z","caller":"traceutil/trace.go:171","msg":"trace[1843586533] range","detail":"{range_begin:/registry/configmaps/; range_end:/registry/configmaps0; response_count:0; response_revision:1531; }","duration":"148.0479ms","start":"2025-07-15T11:00:11.732614Z","end":"2025-07-15T11:00:11.880662Z","steps":["trace[1843586533] 'agreement among raft nodes before linearized reading' (duration: 147.699338ms)"],"step_count":1} Jul 15 11:00:11.883624 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.880858Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.890593282s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 11:00:11.883624 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.881018Z","caller":"traceutil/trace.go:171","msg":"trace[436190226] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1531; }","duration":"1.890916103s","start":"2025-07-15T11:00:09.990073Z","end":"2025-07-15T11:00:11.880989Z","steps":["trace[436190226] 'agreement among raft nodes before linearized reading' (duration: 1.888180147s)"],"step_count":1} Jul 15 11:00:11.884008 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.881496Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.23900424s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/daemonsets/calico-system/calico-node\" ","response":"range_response_count:1 size:11440"} Jul 15 11:00:11.884008 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.881504Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:09.990000Z","time spent":"1.891258751s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 11:00:11.884008 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.881553Z","caller":"traceutil/trace.go:171","msg":"trace[671385025] range","detail":"{range_begin:/registry/daemonsets/calico-system/calico-node; range_end:; response_count:1; response_revision:1531; }","duration":"1.239179026s","start":"2025-07-15T11:00:10.642359Z","end":"2025-07-15T11:00:11.881538Z","steps":["trace[671385025] 'agreement among raft nodes before linearized reading' (duration: 1.238939455s)"],"step_count":1} Jul 15 11:00:11.884008 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.881598Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:10.642325Z","time spent":"1.239258298s","remote":"172.24.4.149:33488","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":1,"response size":11464,"request content":"key:\"/registry/daemonsets/calico-system/calico-node\" "} Jul 15 11:00:11.884008 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.881698Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.61539535s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/prioritylevelconfigurations/\" range_end:\"/registry/prioritylevelconfigurations0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:11.884570 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.881773Z","caller":"traceutil/trace.go:171","msg":"trace[1409987378] range","detail":"{range_begin:/registry/prioritylevelconfigurations/; range_end:/registry/prioritylevelconfigurations0; response_count:0; response_revision:1531; }","duration":"1.615542063s","start":"2025-07-15T11:00:10.266210Z","end":"2025-07-15T11:00:11.881753Z","steps":["trace[1409987378] 'agreement among raft nodes before linearized reading' (duration: 1.615430588s)"],"step_count":1} Jul 15 11:00:11.884570 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.881835Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.161716706s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/tiers/\" range_end:\"/registry/crd.projectcalico.org/tiers0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:11.884570 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:11.881883Z","caller":"traceutil/trace.go:171","msg":"trace[2071256955] range","detail":"{range_begin:/registry/crd.projectcalico.org/tiers/; range_end:/registry/crd.projectcalico.org/tiers0; response_count:0; response_revision:1531; }","duration":"1.161848269s","start":"2025-07-15T11:00:10.720021Z","end":"2025-07-15T11:00:11.881870Z","steps":["trace[2071256955] 'agreement among raft nodes before linearized reading' (duration: 1.161774357s)"],"step_count":1} Jul 15 11:00:11.884570 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.881952Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:10.266143Z","time spent":"1.615781674s","remote":"172.24.4.149:33428","response type":"/etcdserverpb.KV/Range","request count":0,"request size":82,"response count":8,"response size":31,"request content":"key:\"/registry/prioritylevelconfigurations/\" range_end:\"/registry/prioritylevelconfigurations0\" count_only:true "} Jul 15 11:00:11.884942 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:11.881977Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:10.719991Z","time spent":"1.161918985s","remote":"172.24.4.149:54498","response type":"/etcdserverpb.KV/Range","request count":0,"request size":82,"response count":3,"response size":31,"request content":"key:\"/registry/crd.projectcalico.org/tiers/\" range_end:\"/registry/crd.projectcalico.org/tiers0\" count_only:true "} Jul 15 11:00:13.367172 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:13.365743Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"407.844238ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:00:13.367172 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:13.365914Z","caller":"traceutil/trace.go:171","msg":"trace[1419427835] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1535; }","duration":"408.121333ms","start":"2025-07-15T11:00:12.957767Z","end":"2025-07-15T11:00:13.365889Z","steps":["trace[1419427835] 'range keys from in-memory index tree' (duration: 407.704621ms)"],"step_count":1} Jul 15 11:00:13.367172 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:13.365988Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:12.957733Z","time spent":"408.218029ms","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 11:00:13.367172 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:13.366260Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"140.280432ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a\" ","response":"range_response_count:1 size:873"} Jul 15 11:00:13.367172 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:13.366305Z","caller":"traceutil/trace.go:171","msg":"trace[1324273411] range","detail":"{range_begin:/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a; range_end:; response_count:1; response_revision:1535; }","duration":"140.371057ms","start":"2025-07-15T11:00:13.225920Z","end":"2025-07-15T11:00:13.366291Z","steps":["trace[1324273411] 'range keys from in-memory index tree' (duration: 140.037494ms)"],"step_count":1} Jul 15 11:00:17.115062 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:17.112767Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"2.759999041s","expected-duration":"1s"} Jul 15 11:00:17.116338 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.114818Z","caller":"traceutil/trace.go:171","msg":"trace[949813387] transaction","detail":"{read_only:false; response_revision:1538; number_of_response:1; }","duration":"2.762140485s","start":"2025-07-15T11:00:14.352620Z","end":"2025-07-15T11:00:17.114761Z","steps":["trace[949813387] 'process raft request' (duration: 2.761299165s)"],"step_count":1} Jul 15 11:00:17.117494 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:17.116796Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:14.352580Z","time spent":"2.763883502s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":517,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:00:17.342405 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.340169Z","caller":"traceutil/trace.go:171","msg":"trace[1303481292] transaction","detail":"{read_only:false; response_revision:1540; number_of_response:1; }","duration":"262.913523ms","start":"2025-07-15T11:00:17.077217Z","end":"2025-07-15T11:00:17.340130Z","steps":["trace[1303481292] 'process raft request' (duration: 262.812399ms)"],"step_count":1} Jul 15 11:00:17.342405 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.340251Z","caller":"traceutil/trace.go:171","msg":"trace[951083310] linearizableReadLoop","detail":"{readStateIndex:1838; appliedIndex:1838; }","duration":"225.988291ms","start":"2025-07-15T11:00:17.114224Z","end":"2025-07-15T11:00:17.340212Z","steps":["trace[951083310] 'read index received' (duration: 225.90473ms)","trace[951083310] 'applied index is now lower than readState.Index' (duration: 15.198µs)"],"step_count":2} Jul 15 11:00:17.342405 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:17.340666Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"266.147642ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/ipreservations/\" range_end:\"/registry/crd.projectcalico.org/ipreservations0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:00:17.342405 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.340728Z","caller":"traceutil/trace.go:171","msg":"trace[898363670] transaction","detail":"{read_only:false; response_revision:1539; number_of_response:1; }","duration":"263.670541ms","start":"2025-07-15T11:00:17.077039Z","end":"2025-07-15T11:00:17.340709Z","steps":["trace[898363670] 'process raft request' (duration: 262.744036ms)"],"step_count":1} Jul 15 11:00:17.342973 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.340820Z","caller":"traceutil/trace.go:171","msg":"trace[1502200729] range","detail":"{range_begin:/registry/crd.projectcalico.org/ipreservations/; range_end:/registry/crd.projectcalico.org/ipreservations0; response_count:0; response_revision:1540; }","duration":"266.358718ms","start":"2025-07-15T11:00:17.074350Z","end":"2025-07-15T11:00:17.340708Z","steps":["trace[1502200729] 'agreement among raft nodes before linearized reading' (duration: 265.994135ms)"],"step_count":1} Jul 15 11:00:17.342973 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:17.340977Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"264.754949ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/volumeattachments/\" range_end:\"/registry/volumeattachments0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:00:17.342973 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.341033Z","caller":"traceutil/trace.go:171","msg":"trace[1475211839] range","detail":"{range_begin:/registry/volumeattachments/; range_end:/registry/volumeattachments0; response_count:0; response_revision:1540; }","duration":"264.852928ms","start":"2025-07-15T11:00:17.076165Z","end":"2025-07-15T11:00:17.341018Z","steps":["trace[1475211839] 'agreement among raft nodes before linearized reading' (duration: 264.752495ms)"],"step_count":1} Jul 15 11:00:17.342973 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:17.341269Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"265.268779ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/daemonsets/calico-system/calico-node\" ","response":"range_response_count:1 size:11440"} Jul 15 11:00:17.343560 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.341328Z","caller":"traceutil/trace.go:171","msg":"trace[985102299] range","detail":"{range_begin:/registry/daemonsets/calico-system/calico-node; range_end:; response_count:1; response_revision:1540; }","duration":"265.346528ms","start":"2025-07-15T11:00:17.075962Z","end":"2025-07-15T11:00:17.341308Z","steps":["trace[985102299] 'agreement among raft nodes before linearized reading' (duration: 265.021702ms)"],"step_count":1} Jul 15 11:00:17.343560 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:17.341330Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"257.849282ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/calico-system/calico-node-n6jj9\" ","response":"range_response_count:1 size:14567"} Jul 15 11:00:17.345570 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.344172Z","caller":"traceutil/trace.go:171","msg":"trace[711637643] range","detail":"{range_begin:/registry/pods/calico-system/calico-node-n6jj9; range_end:; response_count:1; response_revision:1540; }","duration":"260.715781ms","start":"2025-07-15T11:00:17.083431Z","end":"2025-07-15T11:00:17.344146Z","steps":["trace[711637643] 'agreement among raft nodes before linearized reading' (duration: 257.726405ms)"],"step_count":1} Jul 15 11:00:17.345570 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:17.343131Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"261.948645ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a\" ","response":"range_response_count:1 size:873"} Jul 15 11:00:17.345570 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:17.343461Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"266.787653ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/persistentvolumes/\" range_end:\"/registry/persistentvolumes0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:00:17.345570 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:17.343783Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"267.335759ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/\" range_end:\"/registry/apiextensions.k8s.io/customresourcedefinitions0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:17.347257 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:17.344301Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"268.747478ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/prioritylevelconfigurations/\" range_end:\"/registry/prioritylevelconfigurations0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:17.347257 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.345928Z","caller":"traceutil/trace.go:171","msg":"trace[1682602765] range","detail":"{range_begin:/registry/prioritylevelconfigurations/; range_end:/registry/prioritylevelconfigurations0; response_count:0; response_revision:1540; }","duration":"270.451869ms","start":"2025-07-15T11:00:17.075462Z","end":"2025-07-15T11:00:17.345914Z","steps":["trace[1682602765] 'agreement among raft nodes before linearized reading' (duration: 268.636132ms)"],"step_count":1} Jul 15 11:00:17.347257 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:17.344882Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"269.940014ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/\" range_end:\"/registry/secrets0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:17.348730 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.346638Z","caller":"traceutil/trace.go:171","msg":"trace[1228455916] range","detail":"{range_begin:/registry/secrets/; range_end:/registry/secrets0; response_count:0; response_revision:1540; }","duration":"271.694312ms","start":"2025-07-15T11:00:17.074903Z","end":"2025-07-15T11:00:17.346597Z","steps":["trace[1228455916] 'agreement among raft nodes before linearized reading' (duration: 269.93264ms)"],"step_count":1} Jul 15 11:00:17.348730 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:17.344738Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"269.526908ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:00:17.348730 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.347472Z","caller":"traceutil/trace.go:171","msg":"trace[64357544] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1540; }","duration":"272.403838ms","start":"2025-07-15T11:00:17.075044Z","end":"2025-07-15T11:00:17.347448Z","steps":["trace[64357544] 'agreement among raft nodes before linearized reading' (duration: 269.553238ms)"],"step_count":1} Jul 15 11:00:17.348730 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.345694Z","caller":"traceutil/trace.go:171","msg":"trace[606574064] range","detail":"{range_begin:/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a; range_end:; response_count:1; response_revision:1540; }","duration":"264.676118ms","start":"2025-07-15T11:00:17.080992Z","end":"2025-07-15T11:00:17.345668Z","steps":["trace[606574064] 'agreement among raft nodes before linearized reading' (duration: 261.682122ms)"],"step_count":1} Jul 15 11:00:17.349213 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.345779Z","caller":"traceutil/trace.go:171","msg":"trace[644151260] range","detail":"{range_begin:/registry/persistentvolumes/; range_end:/registry/persistentvolumes0; response_count:0; response_revision:1540; }","duration":"269.184158ms","start":"2025-07-15T11:00:17.076581Z","end":"2025-07-15T11:00:17.345766Z","steps":["trace[644151260] 'agreement among raft nodes before linearized reading' (duration: 266.743118ms)"],"step_count":1} Jul 15 11:00:17.349213 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.345848Z","caller":"traceutil/trace.go:171","msg":"trace[388742431] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/; range_end:/registry/apiextensions.k8s.io/customresourcedefinitions0; response_count:0; response_revision:1540; }","duration":"269.462734ms","start":"2025-07-15T11:00:17.076339Z","end":"2025-07-15T11:00:17.345802Z","steps":["trace[388742431] 'agreement among raft nodes before linearized reading' (duration: 267.235766ms)"],"step_count":1} Jul 15 11:00:17.373655 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:17.372162Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"297.276071ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/\" range_end:\"/registry/pods0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:17.373655 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:17.373211Z","caller":"traceutil/trace.go:171","msg":"trace[1641658637] range","detail":"{range_begin:/registry/pods/; range_end:/registry/pods0; response_count:0; response_revision:1540; }","duration":"298.312047ms","start":"2025-07-15T11:00:17.074802Z","end":"2025-07-15T11:00:17.373114Z","steps":["trace[1641658637] 'agreement among raft nodes before linearized reading' (duration: 272.438395ms)","trace[1641658637] 'count revisions from in-memory index tree' (duration: 24.835823ms)"],"step_count":2} Jul 15 11:00:19.333822 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:19.332805Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"116.518069ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a\" ","response":"range_response_count:1 size:873"} Jul 15 11:00:19.333822 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:19.332931Z","caller":"traceutil/trace.go:171","msg":"trace[581058451] range","detail":"{range_begin:/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a; range_end:; response_count:1; response_revision:1544; }","duration":"116.72127ms","start":"2025-07-15T11:00:19.216178Z","end":"2025-07-15T11:00:19.332899Z","steps":["trace[581058451] 'range keys from in-memory index tree' (duration: 116.11917ms)"],"step_count":1} Jul 15 11:00:27.172024 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:27.171020Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075951,"retry-timeout":"500ms"} Jul 15 11:00:31.475143 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:27.672299Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075951,"retry-timeout":"500ms"} Jul 15 11:00:31.475143 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:28.092983Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.424233647s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 11:00:31.475143 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:28.093107Z","caller":"traceutil/trace.go:171","msg":"trace[1796219119] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"1.424404375s","start":"2025-07-15T11:00:26.668676Z","end":"2025-07-15T11:00:28.093081Z","steps":["trace[1796219119] 'agreement among raft nodes before linearized reading' (duration: 1.424252102s)"],"step_count":1} Jul 15 11:00:31.475143 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:28.093509Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:26.668663Z","time spent":"1.424750522s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:00:31.475143 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:28.173037Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075951,"retry-timeout":"500ms"} Jul 15 11:00:31.475143 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:28.673830Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075951,"retry-timeout":"500ms"} Jul 15 11:00:31.475143 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.357950Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075951,"retry-timeout":"500ms"} Jul 15 11:00:31.476697 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.358722Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.393809589s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 11:00:31.476697 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.358897Z","caller":"traceutil/trace.go:171","msg":"trace[103270957] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"3.394124737s","start":"2025-07-15T11:00:27.964743Z","end":"2025-07-15T11:00:31.358868Z","steps":["trace[103270957] 'agreement among raft nodes before linearized reading' (duration: 3.393870647s)"],"step_count":1} Jul 15 11:00:31.476697 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.359300Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:27.964712Z","time spent":"3.394492245s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:00:31.476697 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.368736Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.697364277s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-scheduler\" ","response":"","error":"context canceled"} Jul 15 11:00:31.476697 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.368805Z","caller":"traceutil/trace.go:171","msg":"trace[276143649] range","detail":"{range_begin:/registry/leases/kube-system/kube-scheduler; range_end:; }","duration":"4.697478839s","start":"2025-07-15T11:00:26.671310Z","end":"2025-07-15T11:00:31.368789Z","steps":["trace[276143649] 'agreement among raft nodes before linearized reading' (duration: 4.697389306s)"],"step_count":1} Jul 15 11:00:31.477120 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.368842Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:26.671282Z","time spent":"4.697550777s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":45,"response count":0,"response size":0,"request content":"key:\"/registry/leases/kube-system/kube-scheduler\" "} Jul 15 11:00:31.477120 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.377059Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.705083205s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/tigera-operator/operator-lock\" ","response":"","error":"context canceled"} Jul 15 11:00:31.477120 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.377134Z","caller":"traceutil/trace.go:171","msg":"trace[1360880252] range","detail":"{range_begin:/registry/leases/tigera-operator/operator-lock; range_end:; }","duration":"4.705195632s","start":"2025-07-15T11:00:26.671922Z","end":"2025-07-15T11:00:31.377117Z","steps":["trace[1360880252] 'agreement among raft nodes before linearized reading' (duration: 4.705077545s)"],"step_count":1} Jul 15 11:00:31.477120 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.377534Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:26.671653Z","time spent":"4.705707398s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":0,"response size":0,"request content":"key:\"/registry/leases/tigera-operator/operator-lock\" "} Jul 15 11:00:31.477120 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.393715Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.723408201s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/daemonsets/calico-system/calico-node\" ","response":"","error":"context canceled"} Jul 15 11:00:31.478204 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.393789Z","caller":"traceutil/trace.go:171","msg":"trace[1947004447] range","detail":"{range_begin:/registry/daemonsets/calico-system/calico-node; range_end:; }","duration":"4.723519155s","start":"2025-07-15T11:00:26.670254Z","end":"2025-07-15T11:00:31.393773Z","steps":["trace[1947004447] 'agreement among raft nodes before linearized reading' (duration: 4.723426096s)"],"step_count":1} Jul 15 11:00:31.478204 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.394052Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:26.670244Z","time spent":"4.723793063s","remote":"172.24.4.149:33488","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":0,"response size":0,"request content":"key:\"/registry/daemonsets/calico-system/calico-node\" "} Jul 15 11:00:31.478204 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.418262Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:26.672489Z","time spent":"4.745765208s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:00:31.682452 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.681274Z","caller":"traceutil/trace.go:171","msg":"trace[893939019] transaction","detail":"{read_only:false; response_revision:1548; number_of_response:1; }","duration":"5.011027946s","start":"2025-07-15T11:00:26.670213Z","end":"2025-07-15T11:00:31.681241Z","steps":["trace[893939019] 'process raft request' (duration: 5.010807501s)"],"step_count":1} Jul 15 11:00:31.682452 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.681542Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:26.670189Z","time spent":"5.011241798s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":699,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:00:31.830316 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.826994Z","caller":"traceutil/trace.go:171","msg":"trace[370574179] transaction","detail":"{read_only:false; response_revision:1550; number_of_response:1; }","duration":"5.15479327s","start":"2025-07-15T11:00:26.672135Z","end":"2025-07-15T11:00:31.826928Z","steps":["trace[370574179] 'process raft request' (duration: 5.15449746s)"],"step_count":1} Jul 15 11:00:31.830316 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.827212Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:26.672102Z","time spent":"5.155003594s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":625,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:00:31.830316 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.827997Z","caller":"traceutil/trace.go:171","msg":"trace[2060980380] transaction","detail":"{read_only:false; response_revision:1552; number_of_response:1; }","duration":"469.369361ms","start":"2025-07-15T11:00:31.358602Z","end":"2025-07-15T11:00:31.827971Z","steps":["trace[2060980380] 'process raft request' (duration: 468.699129ms)"],"step_count":1} Jul 15 11:00:31.830961 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.828150Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:31.358579Z","time spent":"469.490494ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":533,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:00:31.830961 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.828492Z","caller":"traceutil/trace.go:171","msg":"trace[1114318376] transaction","detail":"{read_only:false; response_revision:1553; number_of_response:1; }","duration":"469.070686ms","start":"2025-07-15T11:00:31.359287Z","end":"2025-07-15T11:00:31.828358Z","steps":["trace[1114318376] 'process raft request' (duration: 468.535835ms)"],"step_count":1} Jul 15 11:00:31.830961 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.828693Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:31.359267Z","time spent":"469.323293ms","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":3298,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:00:31.830961 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.829004Z","caller":"traceutil/trace.go:171","msg":"trace[1437515708] transaction","detail":"{read_only:false; response_revision:1554; number_of_response:1; }","duration":"450.842576ms","start":"2025-07-15T11:00:31.378142Z","end":"2025-07-15T11:00:31.828984Z","steps":["trace[1437515708] 'process raft request' (duration: 450.091719ms)"],"step_count":1} Jul 15 11:00:31.831642 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.829126Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:31.378091Z","time spent":"450.960022ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":419,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:00:31.841738 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.839104Z","caller":"traceutil/trace.go:171","msg":"trace[1266979032] linearizableReadLoop","detail":"{readStateIndex:1850; appliedIndex:1847; }","duration":"5.16823918s","start":"2025-07-15T11:00:26.670831Z","end":"2025-07-15T11:00:31.839070Z","steps":["trace[1266979032] 'read index received' (duration: 5.010390938s)","trace[1266979032] 'applied index is now lower than readState.Index' (duration: 157.846008ms)"],"step_count":2} Jul 15 11:00:31.841738 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.839245Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.169050724s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/stagedkubernetesnetworkpolicies/\" range_end:\"/registry/crd.projectcalico.org/stagedkubernetesnetworkpolicies0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:00:31.841738 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.839337Z","caller":"traceutil/trace.go:171","msg":"trace[1187445373] range","detail":"{range_begin:/registry/crd.projectcalico.org/stagedkubernetesnetworkpolicies/; range_end:/registry/crd.projectcalico.org/stagedkubernetesnetworkpolicies0; response_count:0; response_revision:1554; }","duration":"5.169205182s","start":"2025-07-15T11:00:26.670103Z","end":"2025-07-15T11:00:31.839308Z","steps":["trace[1187445373] 'agreement among raft nodes before linearized reading' (duration: 5.169041277s)"],"step_count":1} Jul 15 11:00:31.841738 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.839363Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.1707822s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/controllers/\" range_end:\"/registry/controllers0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:00:31.843915 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.839497Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:26.670087Z","time spent":"5.169349931s","remote":"172.24.4.149:54568","response type":"/etcdserverpb.KV/Range","request count":0,"request size":134,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/stagedkubernetesnetworkpolicies/\" range_end:\"/registry/crd.projectcalico.org/stagedkubernetesnetworkpolicies0\" count_only:true "} Jul 15 11:00:31.843915 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.839532Z","caller":"traceutil/trace.go:171","msg":"trace[1247796927] range","detail":"{range_begin:/registry/controllers/; range_end:/registry/controllers0; response_count:0; response_revision:1554; }","duration":"5.170944734s","start":"2025-07-15T11:00:26.668522Z","end":"2025-07-15T11:00:31.839467Z","steps":["trace[1247796927] 'agreement among raft nodes before linearized reading' (duration: 5.170724359s)"],"step_count":1} Jul 15 11:00:31.843915 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.839650Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:26.668509Z","time spent":"5.171109831s","remote":"172.24.4.149:33232","response type":"/etcdserverpb.KV/Range","request count":0,"request size":50,"response count":0,"response size":29,"request content":"key:\"/registry/controllers/\" range_end:\"/registry/controllers0\" count_only:true "} Jul 15 11:00:31.843915 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.839739Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.169217505s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/managementclusterconnections/\" range_end:\"/registry/operator.tigera.io/managementclusterconnections0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:00:31.845651 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.839804Z","caller":"traceutil/trace.go:171","msg":"trace[1147472898] range","detail":"{range_begin:/registry/operator.tigera.io/managementclusterconnections/; range_end:/registry/operator.tigera.io/managementclusterconnections0; response_count:0; response_revision:1554; }","duration":"5.169305196s","start":"2025-07-15T11:00:26.670477Z","end":"2025-07-15T11:00:31.839783Z","steps":["trace[1147472898] 'agreement among raft nodes before linearized reading' (duration: 5.169199662s)"],"step_count":1} Jul 15 11:00:31.845651 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.839906Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:26.670466Z","time spent":"5.169392243s","remote":"172.24.4.149:54944","response type":"/etcdserverpb.KV/Range","request count":0,"request size":122,"response count":0,"response size":29,"request content":"key:\"/registry/operator.tigera.io/managementclusterconnections/\" range_end:\"/registry/operator.tigera.io/managementclusterconnections0\" count_only:true "} Jul 15 11:00:31.845651 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.839998Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.171756158s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 11:00:31.845651 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.840077Z","caller":"traceutil/trace.go:171","msg":"trace[847419814] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1554; }","duration":"5.171858475s","start":"2025-07-15T11:00:26.668196Z","end":"2025-07-15T11:00:31.840055Z","steps":["trace[847419814] 'agreement among raft nodes before linearized reading' (duration: 5.171623813s)"],"step_count":1} Jul 15 11:00:31.848544 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.840139Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:26.668184Z","time spent":"5.17193418s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 11:00:31.848544 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.840798Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"482.9631ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/csidrivers/\" range_end:\"/registry/csidrivers0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:31.848544 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.840891Z","caller":"traceutil/trace.go:171","msg":"trace[1543781873] range","detail":"{range_begin:/registry/csidrivers/; range_end:/registry/csidrivers0; response_count:0; response_revision:1554; }","duration":"483.130332ms","start":"2025-07-15T11:00:31.357737Z","end":"2025-07-15T11:00:31.840867Z","steps":["trace[1543781873] 'agreement among raft nodes before linearized reading' (duration: 483.001643ms)"],"step_count":1} Jul 15 11:00:31.848544 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.842202Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:31.357711Z","time spent":"483.343322ms","remote":"172.24.4.149:33412","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":1,"response size":31,"request content":"key:\"/registry/csidrivers/\" range_end:\"/registry/csidrivers0\" count_only:true "} Jul 15 11:00:31.848544 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.843091Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"420.987268ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-controller-manager\" ","response":"range_response_count:1 size:532"} Jul 15 11:00:31.849272 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.843415Z","caller":"traceutil/trace.go:171","msg":"trace[114202501] range","detail":"{range_begin:/registry/leases/kube-system/kube-controller-manager; range_end:; response_count:1; response_revision:1554; }","duration":"421.331101ms","start":"2025-07-15T11:00:31.422024Z","end":"2025-07-15T11:00:31.843355Z","steps":["trace[114202501] 'agreement among raft nodes before linearized reading' (duration: 421.002778ms)"],"step_count":1} Jul 15 11:00:31.849272 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.843844Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:31.422005Z","time spent":"421.780938ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":54,"response count":1,"response size":556,"request content":"key:\"/registry/leases/kube-system/kube-controller-manager\" "} Jul 15 11:00:31.849272 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.844367Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"484.452841ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/apiservers/\" range_end:\"/registry/operator.tigera.io/apiservers0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:31.849272 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.844470Z","caller":"traceutil/trace.go:171","msg":"trace[482191420] range","detail":"{range_begin:/registry/operator.tigera.io/apiservers/; range_end:/registry/operator.tigera.io/apiservers0; response_count:0; response_revision:1554; }","duration":"484.601227ms","start":"2025-07-15T11:00:31.359850Z","end":"2025-07-15T11:00:31.844452Z","steps":["trace[482191420] 'agreement among raft nodes before linearized reading' (duration: 484.464995ms)"],"step_count":1} Jul 15 11:00:31.851065 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.844653Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:31.359803Z","time spent":"484.70174ms","remote":"172.24.4.149:54882","response type":"/etcdserverpb.KV/Range","request count":0,"request size":86,"response count":1,"response size":31,"request content":"key:\"/registry/operator.tigera.io/apiservers/\" range_end:\"/registry/operator.tigera.io/apiservers0\" count_only:true "} Jul 15 11:00:31.851065 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.845268Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"487.185046ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a\" ","response":"range_response_count:1 size:873"} Jul 15 11:00:31.851065 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.845462Z","caller":"traceutil/trace.go:171","msg":"trace[453593919] range","detail":"{range_begin:/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a; range_end:; response_count:1; response_revision:1554; }","duration":"487.349923ms","start":"2025-07-15T11:00:31.358023Z","end":"2025-07-15T11:00:31.845373Z","steps":["trace[453593919] 'agreement among raft nodes before linearized reading' (duration: 487.10369ms)"],"step_count":1} Jul 15 11:00:31.851065 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.845766Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:31.357912Z","time spent":"487.824158ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":71,"response count":1,"response size":897,"request content":"key:\"/registry/events/calico-system/csi-node-driver-ltdb7.185267a77c26816a\" "} Jul 15 11:00:31.851065 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.846175Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"488.158492ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/jobs/\" range_end:\"/registry/jobs0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:00:31.851940 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.846475Z","caller":"traceutil/trace.go:171","msg":"trace[175105921] range","detail":"{range_begin:/registry/jobs/; range_end:/registry/jobs0; response_count:0; response_revision:1554; }","duration":"488.435847ms","start":"2025-07-15T11:00:31.357879Z","end":"2025-07-15T11:00:31.846315Z","steps":["trace[175105921] 'agreement among raft nodes before linearized reading' (duration: 488.190514ms)"],"step_count":1} Jul 15 11:00:31.851940 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.846816Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:31.357865Z","time spent":"488.770472ms","remote":"172.24.4.149:33262","response type":"/etcdserverpb.KV/Range","request count":0,"request size":36,"response count":0,"response size":29,"request content":"key:\"/registry/jobs/\" range_end:\"/registry/jobs0\" count_only:true "} Jul 15 11:00:31.851940 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.847139Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"490.086808ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/networksets/\" range_end:\"/registry/crd.projectcalico.org/networksets0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:00:31.851940 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.847220Z","caller":"traceutil/trace.go:171","msg":"trace[1625092955] range","detail":"{range_begin:/registry/crd.projectcalico.org/networksets/; range_end:/registry/crd.projectcalico.org/networksets0; response_count:0; response_revision:1554; }","duration":"490.24303ms","start":"2025-07-15T11:00:31.356953Z","end":"2025-07-15T11:00:31.847196Z","steps":["trace[1625092955] 'agreement among raft nodes before linearized reading' (duration: 490.07201ms)"],"step_count":1} Jul 15 11:00:31.852647 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.847320Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:31.356923Z","time spent":"490.353963ms","remote":"172.24.4.149:54570","response type":"/etcdserverpb.KV/Range","request count":0,"request size":94,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/networksets/\" range_end:\"/registry/crd.projectcalico.org/networksets0\" count_only:true "} Jul 15 11:00:31.852647 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.844959Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"487.355114ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:00:31.852647 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.847474Z","caller":"traceutil/trace.go:171","msg":"trace[694662358] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1554; }","duration":"489.910509ms","start":"2025-07-15T11:00:31.357542Z","end":"2025-07-15T11:00:31.847453Z","steps":["trace[694662358] 'agreement among raft nodes before linearized reading' (duration: 487.357639ms)"],"step_count":1} Jul 15 11:00:31.852647 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.847548Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:31.357501Z","time spent":"490.018758ms","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 11:00:31.852647 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.846974Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"487.701692ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/ipamblocks/\" range_end:\"/registry/crd.projectcalico.org/ipamblocks0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:31.853537 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.847699Z","caller":"traceutil/trace.go:171","msg":"trace[1297993455] range","detail":"{range_begin:/registry/crd.projectcalico.org/ipamblocks/; range_end:/registry/crd.projectcalico.org/ipamblocks0; response_count:0; response_revision:1554; }","duration":"490.440731ms","start":"2025-07-15T11:00:31.357237Z","end":"2025-07-15T11:00:31.847677Z","steps":["trace[1297993455] 'agreement among raft nodes before linearized reading' (duration: 487.765946ms)"],"step_count":1} Jul 15 11:00:31.853537 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.847720Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.164400723s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678d1663474b\" ","response":"range_response_count:1 size:904"} Jul 15 11:00:31.853537 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.847770Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:31.357223Z","time spent":"490.520485ms","remote":"172.24.4.149:54504","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":1,"response size":31,"request content":"key:\"/registry/crd.projectcalico.org/ipamblocks/\" range_end:\"/registry/crd.projectcalico.org/ipamblocks0\" count_only:true "} Jul 15 11:00:31.853537 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.847833Z","caller":"traceutil/trace.go:171","msg":"trace[1355122997] range","detail":"{range_begin:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678d1663474b; range_end:; response_count:1; response_revision:1554; }","duration":"3.164709328s","start":"2025-07-15T11:00:28.683057Z","end":"2025-07-15T11:00:31.847766Z","steps":["trace[1355122997] 'agreement among raft nodes before linearized reading' (duration: 3.164460348s)"],"step_count":1} Jul 15 11:00:31.854218 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.847925Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:28.683022Z","time spent":"3.164880718s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":97,"response count":1,"response size":928,"request content":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678d1663474b\" "} Jul 15 11:00:31.854218 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.848029Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.970740282s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/storageclasses/\" range_end:\"/registry/storageclasses0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:00:31.854218 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:31.848152Z","caller":"traceutil/trace.go:171","msg":"trace[1600474020] range","detail":"{range_begin:/registry/storageclasses/; range_end:/registry/storageclasses0; response_count:0; response_revision:1554; }","duration":"3.970862939s","start":"2025-07-15T11:00:27.877238Z","end":"2025-07-15T11:00:31.848100Z","steps":["trace[1600474020] 'agreement among raft nodes before linearized reading' (duration: 3.970753548s)"],"step_count":1} Jul 15 11:00:31.854218 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:31.848240Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:27.877216Z","time spent":"3.970979583s","remote":"172.24.4.149:33390","response type":"/etcdserverpb.KV/Range","request count":0,"request size":56,"response count":0,"response size":29,"request content":"key:\"/registry/storageclasses/\" range_end:\"/registry/storageclasses0\" count_only:true "} Jul 15 11:00:32.104886 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:32.103120Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"202.179737ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 11:00:32.104886 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:32.103545Z","caller":"traceutil/trace.go:171","msg":"trace[2086254056] linearizableReadLoop","detail":"{readStateIndex:1860; appliedIndex:1858; }","duration":"217.310816ms","start":"2025-07-15T11:00:31.886183Z","end":"2025-07-15T11:00:32.103494Z","steps":["trace[2086254056] 'read index received' (duration: 3.552737ms)","trace[2086254056] 'applied index is now lower than readState.Index' (duration: 213.756787ms)"],"step_count":2} Jul 15 11:00:32.104886 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:32.103565Z","caller":"traceutil/trace.go:171","msg":"trace[345093113] transaction","detail":"{read_only:false; response_revision:1557; number_of_response:1; }","duration":"222.290513ms","start":"2025-07-15T11:00:31.881214Z","end":"2025-07-15T11:00:32.103505Z","steps":["trace[345093113] 'process raft request' (duration: 19.563332ms)","trace[345093113] 'get key's previous created_revision and leaseID' {req_type:put; key:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678d1663474b; req_size:886; } (duration: 201.957217ms)"],"step_count":2} Jul 15 11:00:32.105657 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:32.103687Z","caller":"traceutil/trace.go:171","msg":"trace[1393335777] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1557; }","duration":"221.841398ms","start":"2025-07-15T11:00:31.881830Z","end":"2025-07-15T11:00:32.103672Z","steps":["trace[1393335777] 'process raft request' (duration: 221.43753ms)"],"step_count":1} Jul 15 11:00:32.114025 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:32.113481Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"225.196254ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:00:32.115036 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:32.114369Z","caller":"traceutil/trace.go:171","msg":"trace[918421331] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1557; }","duration":"228.191476ms","start":"2025-07-15T11:00:31.886146Z","end":"2025-07-15T11:00:32.114337Z","steps":["trace[918421331] 'agreement among raft nodes before linearized reading' (duration: 217.708011ms)"],"step_count":1} Jul 15 11:00:41.697807 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:33.842092Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075976,"retry-timeout":"500ms"} Jul 15 11:00:41.697807 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:34.343174Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075976,"retry-timeout":"500ms"} Jul 15 11:00:41.697807 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:34.844064Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075976,"retry-timeout":"500ms"} Jul 15 11:00:41.697807 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:35.344227Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075976,"retry-timeout":"500ms"} Jul 15 11:00:41.697807 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:35.844729Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075976,"retry-timeout":"500ms"} Jul 15 11:00:41.697807 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:36.344892Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075976,"retry-timeout":"500ms"} Jul 15 11:00:41.697807 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:36.846126Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075976,"retry-timeout":"500ms"} Jul 15 11:00:41.697807 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:37.347114Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075976,"retry-timeout":"500ms"} Jul 15 11:00:41.697807 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:37.847258Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075976,"retry-timeout":"500ms"} Jul 15 11:00:41.767270 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:38.347561Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075976,"retry-timeout":"500ms"} Jul 15 11:00:41.767270 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:38.848480Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075976,"retry-timeout":"500ms"} Jul 15 11:00:41.767270 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:39.142747Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:32.141443Z","time spent":"7.001278509s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:00:41.767270 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:39.349424Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075976,"retry-timeout":"500ms"} Jul 15 11:00:41.767270 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:39.849652Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075976,"retry-timeout":"500ms"} Jul 15 11:00:41.767270 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:40.342713Z","caller":"etcdserver/v3_server.go:932","msg":"timed out waiting for read index response (local node might have slow network)","timeout":"7s"} Jul 15 11:00:41.767270 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:40.342978Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.001097302s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"","error":"etcdserver: request timed out"} Jul 15 11:00:41.768358 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:40.343138Z","caller":"traceutil/trace.go:171","msg":"trace[1376862633] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; }","duration":"7.001245038s","start":"2025-07-15T11:00:33.341784Z","end":"2025-07-15T11:00:40.343029Z","steps":["trace[1376862633] 'agreement among raft nodes before linearized reading' (duration: 7.001141317s)"],"step_count":1} Jul 15 11:00:41.768358 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:40.343221Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:33.341747Z","time spent":"7.001451656s","remote":"172.24.4.149:54994","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":0,"response size":0,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" "} Jul 15 11:00:41.768358 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:40.428468Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.998481325s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 11:00:41.768358 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:40.428581Z","caller":"traceutil/trace.go:171","msg":"trace[1898893006] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"1.998694214s","start":"2025-07-15T11:00:38.429859Z","end":"2025-07-15T11:00:40.428553Z","steps":["trace[1898893006] 'agreement among raft nodes before linearized reading' (duration: 1.998513546s)"],"step_count":1} Jul 15 11:00:41.768358 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:40.428650Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:38.429837Z","time spent":"1.99879502s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:00:41.769007 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:40.843471Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075979,"retry-timeout":"500ms"} Jul 15 11:00:41.769007 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:41.339662Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:40.362455Z","time spent":"977.173274ms","remote":"172.24.4.149:54498","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:00:41.769007 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:41.344117Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075979,"retry-timeout":"500ms"} Jul 15 11:00:41.769007 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:41.696029Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:32.123577Z","time spent":"9.572423501s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:00:41.845742 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:41.845112Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075979,"retry-timeout":"500ms"} Jul 15 11:00:43.217795 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:42.345862Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075979,"retry-timeout":"500ms"} Jul 15 11:00:43.217795 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:42.429581Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000312985s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:00:43.217795 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:42.429731Z","caller":"traceutil/trace.go:171","msg":"trace[10200983] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.000530103s","start":"2025-07-15T11:00:40.429171Z","end":"2025-07-15T11:00:42.429701Z","steps":["trace[10200983] 'agreement among raft nodes before linearized reading' (duration: 2.000348774s)"],"step_count":1} Jul 15 11:00:43.217795 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:42.429838Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:40.429148Z","time spent":"2.000667637s","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:00:43.217795 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:42.846766Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075979,"retry-timeout":"500ms"} Jul 15 11:00:43.345859 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.345071Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000746056s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:00:43.345859 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:43.345182Z","caller":"traceutil/trace.go:171","msg":"trace[607210524] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.000911103s","start":"2025-07-15T11:00:41.344244Z","end":"2025-07-15T11:00:43.345155Z","steps":["trace[607210524] 'agreement among raft nodes before linearized reading' (duration: 2.000771955s)"],"step_count":1} Jul 15 11:00:43.345859 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.345246Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:41.344222Z","time spent":"2.001007761s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:00:43.347568 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.347302Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075979,"retry-timeout":"500ms"} Jul 15 11:00:43.365247 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.364935Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:38.381686Z","time spent":"4.983244777s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:00:43.552487 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.550929Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"11.410741057s","expected-duration":"1s"} Jul 15 11:00:43.631279 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:43.626421Z","caller":"traceutil/trace.go:171","msg":"trace[1076201302] linearizableReadLoop","detail":"{readStateIndex:1866; appliedIndex:1861; }","duration":"3.283486364s","start":"2025-07-15T11:00:40.342879Z","end":"2025-07-15T11:00:43.626365Z","steps":["trace[1076201302] 'read index received' (duration: 3.208996335s)","trace[1076201302] 'applied index is now lower than readState.Index' (duration: 74.488426ms)"],"step_count":2} Jul 15 11:00:43.631279 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.627310Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.260356541s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/priorityclasses/\" range_end:\"/registry/priorityclasses0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:43.631279 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:43.627437Z","caller":"traceutil/trace.go:171","msg":"trace[1739317535] range","detail":"{range_begin:/registry/priorityclasses/; range_end:/registry/priorityclasses0; response_count:0; response_revision:1561; }","duration":"5.260497423s","start":"2025-07-15T11:00:38.366880Z","end":"2025-07-15T11:00:43.627378Z","steps":["trace[1739317535] 'agreement among raft nodes before linearized reading' (duration: 5.259662502s)"],"step_count":1} Jul 15 11:00:43.631279 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.627498Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.260668081s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/kubecontrollersconfigurations/\" range_end:\"/registry/crd.projectcalico.org/kubecontrollersconfigurations0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:43.631873 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:43.627555Z","caller":"traceutil/trace.go:171","msg":"trace[1432300683] range","detail":"{range_begin:/registry/crd.projectcalico.org/kubecontrollersconfigurations/; range_end:/registry/crd.projectcalico.org/kubecontrollersconfigurations0; response_count:0; response_revision:1561; }","duration":"5.260816057s","start":"2025-07-15T11:00:38.366722Z","end":"2025-07-15T11:00:43.627538Z","steps":["trace[1432300683] 'agreement among raft nodes before linearized reading' (duration: 5.260694062s)"],"step_count":1} Jul 15 11:00:43.631873 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.627632Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:38.366693Z","time spent":"5.260917351s","remote":"172.24.4.149:54920","response type":"/etcdserverpb.KV/Range","request count":0,"request size":130,"response count":1,"response size":31,"request content":"key:\"/registry/crd.projectcalico.org/kubecontrollersconfigurations/\" range_end:\"/registry/crd.projectcalico.org/kubecontrollersconfigurations0\" count_only:true "} Jul 15 11:00:43.631873 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:43.627631Z","caller":"traceutil/trace.go:171","msg":"trace[1538143606] transaction","detail":"{read_only:false; response_revision:1561; number_of_response:1; }","duration":"1.793732156s","start":"2025-07-15T11:00:41.833879Z","end":"2025-07-15T11:00:43.627611Z","steps":["trace[1538143606] 'process raft request' (duration: 1.791361555s)"],"step_count":1} Jul 15 11:00:43.632203 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.627832Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:41.833811Z","time spent":"1.793941851s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":699,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:00:43.632203 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.627846Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.394731891s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/endpoints/\" range_end:\"/registry/services/endpoints0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:43.632203 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:43.627897Z","caller":"traceutil/trace.go:171","msg":"trace[555325800] range","detail":"{range_begin:/registry/services/endpoints/; range_end:/registry/services/endpoints0; response_count:0; response_revision:1561; }","duration":"7.394784422s","start":"2025-07-15T11:00:36.233100Z","end":"2025-07-15T11:00:43.627884Z","steps":["trace[555325800] 'agreement among raft nodes before linearized reading' (duration: 7.39469494s)"],"step_count":1} Jul 15 11:00:43.632203 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.627932Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:36.233027Z","time spent":"7.3948925s","remote":"172.24.4.149:33194","response type":"/etcdserverpb.KV/Range","request count":0,"request size":64,"response count":7,"response size":31,"request content":"key:\"/registry/services/endpoints/\" range_end:\"/registry/services/endpoints0\" count_only:true "} Jul 15 11:00:43.632693 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.627961Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.567155837s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/specs/\" range_end:\"/registry/services/specs0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:43.632693 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:43.628013Z","caller":"traceutil/trace.go:171","msg":"trace[1736944910] range","detail":"{range_begin:/registry/services/specs/; range_end:/registry/services/specs0; response_count:0; response_revision:1561; }","duration":"3.567505591s","start":"2025-07-15T11:00:40.060484Z","end":"2025-07-15T11:00:43.627989Z","steps":["trace[1736944910] 'agreement among raft nodes before linearized reading' (duration: 3.567440004s)"],"step_count":1} Jul 15 11:00:43.632693 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.628089Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:40.060460Z","time spent":"3.567591626s","remote":"172.24.4.149:33212","response type":"/etcdserverpb.KV/Range","request count":0,"request size":56,"response count":7,"response size":31,"request content":"key:\"/registry/services/specs/\" range_end:\"/registry/services/specs0\" count_only:true "} Jul 15 11:00:43.632693 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.628102Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.886243627s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/certificatesigningrequests/\" range_end:\"/registry/certificatesigningrequests0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:43.632995 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:43.628143Z","caller":"traceutil/trace.go:171","msg":"trace[1733891105] range","detail":"{range_begin:/registry/certificatesigningrequests/; range_end:/registry/certificatesigningrequests0; response_count:0; response_revision:1561; }","duration":"7.886329102s","start":"2025-07-15T11:00:35.741799Z","end":"2025-07-15T11:00:43.628128Z","steps":["trace[1733891105] 'agreement among raft nodes before linearized reading' (duration: 7.886268846s)"],"step_count":1} Jul 15 11:00:43.632995 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.628991Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:35.741769Z","time spent":"7.887195624s","remote":"172.24.4.149:33282","response type":"/etcdserverpb.KV/Range","request count":0,"request size":80,"response count":2,"response size":31,"request content":"key:\"/registry/certificatesigningrequests/\" range_end:\"/registry/certificatesigningrequests0\" count_only:true "} Jul 15 11:00:43.632995 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.627506Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:38.366869Z","time spent":"5.260616652s","remote":"172.24.4.149:33388","response type":"/etcdserverpb.KV/Range","request count":0,"request size":58,"response count":2,"response size":31,"request content":"key:\"/registry/priorityclasses/\" range_end:\"/registry/priorityclasses0\" count_only:true "} Jul 15 11:00:43.632995 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.628361Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.481331746s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 11:00:43.633244 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:43.629272Z","caller":"traceutil/trace.go:171","msg":"trace[1060857884] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1561; }","duration":"4.482288891s","start":"2025-07-15T11:00:39.146967Z","end":"2025-07-15T11:00:43.629255Z","steps":["trace[1060857884] 'agreement among raft nodes before linearized reading' (duration: 4.481345412s)"],"step_count":1} Jul 15 11:00:43.633244 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.629420Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:39.146949Z","time spent":"4.482411318s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 11:00:43.633244 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.628562Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.910591697s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/caliconodestatuses/\" range_end:\"/registry/crd.projectcalico.org/caliconodestatuses0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:00:43.633244 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:43.630162Z","caller":"traceutil/trace.go:171","msg":"trace[788994625] range","detail":"{range_begin:/registry/crd.projectcalico.org/caliconodestatuses/; range_end:/registry/crd.projectcalico.org/caliconodestatuses0; response_count:0; response_revision:1561; }","duration":"4.912223113s","start":"2025-07-15T11:00:38.717878Z","end":"2025-07-15T11:00:43.630101Z","steps":["trace[788994625] 'agreement among raft nodes before linearized reading' (duration: 4.910565627s)"],"step_count":1} Jul 15 11:00:43.633650 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.630282Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:38.717849Z","time spent":"4.912408931s","remote":"172.24.4.149:54648","response type":"/etcdserverpb.KV/Range","request count":0,"request size":108,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/caliconodestatuses/\" range_end:\"/registry/crd.projectcalico.org/caliconodestatuses0\" count_only:true "} Jul 15 11:00:43.633650 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.628801Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"8.39717269s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/hostendpoints/\" range_end:\"/registry/crd.projectcalico.org/hostendpoints0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:00:43.633650 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:43.630658Z","caller":"traceutil/trace.go:171","msg":"trace[2016159949] range","detail":"{range_begin:/registry/crd.projectcalico.org/hostendpoints/; range_end:/registry/crd.projectcalico.org/hostendpoints0; response_count:0; response_revision:1561; }","duration":"8.399026314s","start":"2025-07-15T11:00:35.231571Z","end":"2025-07-15T11:00:43.630597Z","steps":["trace[2016159949] 'agreement among raft nodes before linearized reading' (duration: 8.397187217s)"],"step_count":1} Jul 15 11:00:43.633650 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:43.630738Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:35.231547Z","time spent":"8.399165542s","remote":"172.24.4.149:54596","response type":"/etcdserverpb.KV/Range","request count":0,"request size":98,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/hostendpoints/\" range_end:\"/registry/crd.projectcalico.org/hostendpoints0\" count_only:true "} Jul 15 11:00:47.281588 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:45.618951Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.999476294s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:00:47.281588 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:45.619029Z","caller":"traceutil/trace.go:171","msg":"trace[1266093794] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.000003331s","start":"2025-07-15T11:00:43.619008Z","end":"2025-07-15T11:00:45.619011Z","steps":["trace[1266093794] 'agreement among raft nodes before linearized reading' (duration: 1.999898159s)"],"step_count":1} Jul 15 11:00:47.281588 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:45.619074Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:43.618974Z","time spent":"2.000088025s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:00:47.281588 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:45.761635Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075984,"retry-timeout":"500ms"} Jul 15 11:00:47.699990 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:47.698871Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075984,"retry-timeout":"500ms"} Jul 15 11:00:47.767024 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:47.766499Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.994669247s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 11:00:47.767603 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:47.767053Z","caller":"traceutil/trace.go:171","msg":"trace[1393174970] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"1.994955368s","start":"2025-07-15T11:00:45.771751Z","end":"2025-07-15T11:00:47.766706Z","steps":["trace[1393174970] 'agreement among raft nodes before linearized reading' (duration: 1.994693133s)"],"step_count":1} Jul 15 11:00:47.767882 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:47.767616Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:45.771731Z","time spent":"1.995850225s","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:00:48.200543 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:48.200032Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075984,"retry-timeout":"500ms"} Jul 15 11:00:48.582785 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:48.582002Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.99988507s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-controller-manager\" ","response":"","error":"context deadline exceeded"} Jul 15 11:00:48.582785 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:48.582111Z","caller":"traceutil/trace.go:171","msg":"trace[735840058] range","detail":"{range_begin:/registry/leases/kube-system/kube-controller-manager; range_end:; }","duration":"5.000055669s","start":"2025-07-15T11:00:43.582018Z","end":"2025-07-15T11:00:48.582073Z","steps":["trace[735840058] 'agreement among raft nodes before linearized reading' (duration: 4.999931109s)"],"step_count":1} Jul 15 11:00:48.582785 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:48.582178Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:43.581980Z","time spent":"5.000179167s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":54,"response count":0,"response size":0,"request content":"key:\"/registry/leases/kube-system/kube-controller-manager\" "} Jul 15 11:00:48.701683 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:48.701147Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075984,"retry-timeout":"500ms"} Jul 15 11:00:49.202886 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:49.202138Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075984,"retry-timeout":"500ms"} Jul 15 11:00:49.702869 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:49.702364Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075984,"retry-timeout":"500ms"} Jul 15 11:00:50.203762 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:50.203102Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075984,"retry-timeout":"500ms"} Jul 15 11:00:50.977860 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:50.975107Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885075984,"retry-timeout":"500ms"} Jul 15 11:00:50.977860 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:50.975238Z","caller":"etcdserver/v3_server.go:932","msg":"timed out waiting for read index response (local node might have slow network)","timeout":"7s"} Jul 15 11:00:50.977860 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:50.975349Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"9.261519796s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c645b0b83\" ","response":"","error":"etcdserver: request timed out"} Jul 15 11:00:50.977860 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:50.975667Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:43.585758Z","time spent":"7.389899017s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:00:50.977860 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:50.975824Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:43.631900Z","time spent":"7.343918283s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.Lease/LeaseGrant","request count":-1,"request size":-1,"response count":-1,"response size":-1,"request content":""} Jul 15 11:00:50.977860 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:50.975954Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.393606226s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/horizontalpodautoscalers/\" range_end:\"/registry/horizontalpodautoscalers0\" count_only:true ","response":"","error":"etcdserver: request timed out"} Jul 15 11:00:50.979075 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:50.975994Z","caller":"traceutil/trace.go:171","msg":"trace[836082668] range","detail":"{range_begin:/registry/horizontalpodautoscalers/; range_end:/registry/horizontalpodautoscalers0; }","duration":"7.393788227s","start":"2025-07-15T11:00:43.582192Z","end":"2025-07-15T11:00:50.975981Z","steps":["trace[836082668] 'agreement among raft nodes before linearized reading' (duration: 7.39373772s)"],"step_count":1} Jul 15 11:00:50.979075 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:50.976060Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:43.582178Z","time spent":"7.393842242s","remote":"172.24.4.149:33250","response type":"/etcdserverpb.KV/Range","request count":0,"request size":76,"response count":0,"response size":0,"request content":"key:\"/registry/horizontalpodautoscalers/\" range_end:\"/registry/horizontalpodautoscalers0\" count_only:true "} Jul 15 11:00:50.979075 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:50.976118Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.394255517s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/controllerrevisions/\" range_end:\"/registry/controllerrevisions0\" count_only:true ","response":"","error":"etcdserver: request timed out"} Jul 15 11:00:50.979075 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:50.976164Z","caller":"traceutil/trace.go:171","msg":"trace[437324809] range","detail":"{range_begin:/registry/controllerrevisions/; range_end:/registry/controllerrevisions0; }","duration":"7.39433971s","start":"2025-07-15T11:00:43.581813Z","end":"2025-07-15T11:00:50.976153Z","steps":["trace[437324809] 'agreement among raft nodes before linearized reading' (duration: 7.394292569s)"],"step_count":1} Jul 15 11:00:50.979450 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:50.976218Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:43.581788Z","time spent":"7.394398374s","remote":"172.24.4.149:33496","response type":"/etcdserverpb.KV/Range","request count":0,"request size":66,"response count":0,"response size":0,"request content":"key:\"/registry/controllerrevisions/\" range_end:\"/registry/controllerrevisions0\" count_only:true "} Jul 15 11:00:50.979450 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:50.976290Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"9.219010991s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal\" ","response":"","error":"etcdserver: request timed out"} Jul 15 11:00:50.979450 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:50.976331Z","caller":"traceutil/trace.go:171","msg":"trace[474961937] range","detail":"{range_begin:/registry/pods/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal; range_end:; }","duration":"9.219093701s","start":"2025-07-15T11:00:41.757226Z","end":"2025-07-15T11:00:50.976320Z","steps":["trace[474961937] 'agreement among raft nodes before linearized reading' (duration: 9.219052381s)"],"step_count":1} Jul 15 11:00:50.979450 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:50.976366Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:41.757177Z","time spent":"9.219177491s","remote":"172.24.4.149:33206","response type":"/etcdserverpb.KV/Range","request count":0,"request size":78,"response count":0,"response size":0,"request content":"key:\"/registry/pods/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal\" "} Jul 15 11:00:50.979450 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:50.976124Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:43.582218Z","time spent":"7.393742248s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:00:50.980809 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:50.975482Z","caller":"traceutil/trace.go:171","msg":"trace[2134297012] range","detail":"{range_begin:/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c645b0b83; range_end:; }","duration":"9.261722287s","start":"2025-07-15T11:00:41.713731Z","end":"2025-07-15T11:00:50.975453Z","steps":["trace[2134297012] 'agreement among raft nodes before linearized reading' (duration: 9.26156319s)"],"step_count":1} Jul 15 11:00:50.980809 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:50.976510Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:41.713700Z","time spent":"9.262790217s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":97,"response count":0,"response size":0,"request content":"key:\"/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c645b0b83\" "} Jul 15 11:00:51.339670 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.315780Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"7.690680785s","expected-duration":"1s"} Jul 15 11:00:51.373111 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.370500Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:50.144114Z","time spent":"1.226367033s","remote":"172.24.4.149:53274","response type":"/etcdserverpb.Maintenance/Status","request count":-1,"request size":-1,"response count":-1,"response size":-1,"request content":""} Jul 15 11:00:51.373111 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:51.372279Z","caller":"traceutil/trace.go:171","msg":"trace[1839437228] linearizableReadLoop","detail":"{readStateIndex:1870; appliedIndex:1866; }","duration":"396.964143ms","start":"2025-07-15T11:00:50.975282Z","end":"2025-07-15T11:00:51.372246Z","steps":["trace[1839437228] 'read index received' (duration: 341.305011ms)","trace[1839437228] 'applied index is now lower than readState.Index' (duration: 55.658ms)"],"step_count":2} Jul 15 11:00:51.377330 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.373718Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.59935516s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/namespaces/\" range_end:\"/registry/namespaces0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:51.377330 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:51.374577Z","caller":"traceutil/trace.go:171","msg":"trace[722343009] range","detail":"{range_begin:/registry/namespaces/; range_end:/registry/namespaces0; response_count:0; response_revision:1564; }","duration":"5.600253854s","start":"2025-07-15T11:00:45.774301Z","end":"2025-07-15T11:00:51.374555Z","steps":["trace[722343009] 'agreement among raft nodes before linearized reading' (duration: 5.598457209s)"],"step_count":1} Jul 15 11:00:51.377330 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.373727Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.673790098s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/\" range_end:\"/registry/operator.tigera.io/installations0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:51.377330 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:51.373801Z","caller":"traceutil/trace.go:171","msg":"trace[536840003] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1564; }","duration":"384.869795ms","start":"2025-07-15T11:00:50.988912Z","end":"2025-07-15T11:00:51.373782Z","steps":["trace[536840003] 'process raft request' (duration: 383.247004ms)"],"step_count":1} Jul 15 11:00:51.377330 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:51.373843Z","caller":"traceutil/trace.go:171","msg":"trace[2114665201] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1564; }","duration":"384.75362ms","start":"2025-07-15T11:00:50.989076Z","end":"2025-07-15T11:00:51.373829Z","steps":["trace[2114665201] 'process raft request' (duration: 383.127704ms)"],"step_count":1} Jul 15 11:00:51.377943 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.373891Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.237060198s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/daemonsets/\" range_end:\"/registry/daemonsets0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:51.377943 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.373979Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.23798896s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:00:51.377943 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.375018Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.596484405s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/felixconfigurations/\" range_end:\"/registry/crd.projectcalico.org/felixconfigurations0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:51.377943 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:51.375330Z","caller":"traceutil/trace.go:171","msg":"trace[37963070] range","detail":"{range_begin:/registry/crd.projectcalico.org/felixconfigurations/; range_end:/registry/crd.projectcalico.org/felixconfigurations0; response_count:0; response_revision:1564; }","duration":"5.596778302s","start":"2025-07-15T11:00:45.778487Z","end":"2025-07-15T11:00:51.375265Z","steps":["trace[37963070] 'agreement among raft nodes before linearized reading' (duration: 5.596444568s)"],"step_count":1} Jul 15 11:00:51.377943 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.375437Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:45.778478Z","time spent":"5.596930106s","remote":"172.24.4.149:54930","response type":"/etcdserverpb.KV/Range","request count":0,"request size":110,"response count":1,"response size":31,"request content":"key:\"/registry/crd.projectcalico.org/felixconfigurations/\" range_end:\"/registry/crd.projectcalico.org/felixconfigurations0\" count_only:true "} Jul 15 11:00:51.378696 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.375452Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:50.988897Z","time spent":"386.405907ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":71,"response count":0,"response size":675,"request content":"compare: success:> failure: >"} Jul 15 11:00:51.378696 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.375668Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:45.774291Z","time spent":"5.601348887s","remote":"172.24.4.149:33144","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":7,"response size":31,"request content":"key:\"/registry/namespaces/\" range_end:\"/registry/namespaces0\" count_only:true "} Jul 15 11:00:51.378696 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.375773Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:50.989066Z","time spent":"386.622886ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":48,"response count":0,"response size":583,"request content":"compare: success:> failure: >"} Jul 15 11:00:51.380106 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:51.375877Z","caller":"traceutil/trace.go:171","msg":"trace[658236268] range","detail":"{range_begin:/registry/daemonsets/; range_end:/registry/daemonsets0; response_count:0; response_revision:1564; }","duration":"1.239090615s","start":"2025-07-15T11:00:50.136756Z","end":"2025-07-15T11:00:51.375847Z","steps":["trace[658236268] 'agreement among raft nodes before linearized reading' (duration: 1.237106888s)"],"step_count":1} Jul 15 11:00:51.380106 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:51.375884Z","caller":"traceutil/trace.go:171","msg":"trace[1405916312] range","detail":"{range_begin:/registry/operator.tigera.io/installations/; range_end:/registry/operator.tigera.io/installations0; response_count:0; response_revision:1564; }","duration":"3.676002225s","start":"2025-07-15T11:00:47.699857Z","end":"2025-07-15T11:00:51.375859Z","steps":["trace[1405916312] 'agreement among raft nodes before linearized reading' (duration: 3.672782194s)"],"step_count":1} Jul 15 11:00:51.380106 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.375925Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:50.136716Z","time spent":"1.239191721s","remote":"172.24.4.149:33488","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":3,"response size":31,"request content":"key:\"/registry/daemonsets/\" range_end:\"/registry/daemonsets0\" count_only:true "} Jul 15 11:00:51.380106 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.375952Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:47.699835Z","time spent":"3.676096155s","remote":"172.24.4.149:54872","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":1,"response size":31,"request content":"key:\"/registry/operator.tigera.io/installations/\" range_end:\"/registry/operator.tigera.io/installations0\" count_only:true "} Jul 15 11:00:51.380549 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:51.376083Z","caller":"traceutil/trace.go:171","msg":"trace[988508120] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1564; }","duration":"1.240155349s","start":"2025-07-15T11:00:50.135914Z","end":"2025-07-15T11:00:51.376069Z","steps":["trace[988508120] 'agreement among raft nodes before linearized reading' (duration: 1.237997497s)"],"step_count":1} Jul 15 11:00:51.380549 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.376137Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:50.135885Z","time spent":"1.240237257s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 11:00:51.380549 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.375051Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.580070372s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"range_response_count:1 size:886"} Jul 15 11:00:51.380549 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:51.376317Z","caller":"traceutil/trace.go:171","msg":"trace[1692344349] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; response_count:1; response_revision:1564; }","duration":"5.58136531s","start":"2025-07-15T11:00:45.794935Z","end":"2025-07-15T11:00:51.376300Z","steps":["trace[1692344349] 'agreement among raft nodes before linearized reading' (duration: 5.579903861s)"],"step_count":1} Jul 15 11:00:51.380549 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.375083Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.594652974s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/networkpolicies/\" range_end:\"/registry/networkpolicies0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:51.380871 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:51.376698Z","caller":"traceutil/trace.go:171","msg":"trace[391886314] range","detail":"{range_begin:/registry/networkpolicies/; range_end:/registry/networkpolicies0; response_count:0; response_revision:1564; }","duration":"5.59598855s","start":"2025-07-15T11:00:45.780375Z","end":"2025-07-15T11:00:51.376364Z","steps":["trace[391886314] 'agreement among raft nodes before linearized reading' (duration: 5.594682601s)"],"step_count":1} Jul 15 11:00:51.380871 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.377147Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:45.780367Z","time spent":"5.596463295s","remote":"172.24.4.149:33316","response type":"/etcdserverpb.KV/Range","request count":0,"request size":58,"response count":3,"response size":31,"request content":"key:\"/registry/networkpolicies/\" range_end:\"/registry/networkpolicies0\" count_only:true "} Jul 15 11:00:51.380871 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:51.377757Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:45.794918Z","time spent":"5.581436828s","remote":"172.24.4.149:54994","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":1,"response size":910,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" "} Jul 15 11:00:51.488207 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:51.487665Z","caller":"traceutil/trace.go:171","msg":"trace[325303450] linearizableReadLoop","detail":"{readStateIndex:1873; appliedIndex:1872; }","duration":"113.42211ms","start":"2025-07-15T11:00:51.374148Z","end":"2025-07-15T11:00:51.487570Z","steps":["trace[325303450] 'read index received' (duration: 21.18298ms)","trace[325303450] 'applied index is now lower than readState.Index' (duration: 92.235323ms)"],"step_count":2} Jul 15 11:00:52.019307 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.016986Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"529.458487ms","expected-duration":"100ms","prefix":"","request":"header: lease_revoke:","response":"size:29"} Jul 15 11:00:52.019307 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:52.017371Z","caller":"traceutil/trace.go:171","msg":"trace[266823895] linearizableReadLoop","detail":"{readStateIndex:1875; appliedIndex:1873; }","duration":"529.214096ms","start":"2025-07-15T11:00:51.488125Z","end":"2025-07-15T11:00:52.017339Z","steps":["trace[266823895] 'read index received' (duration: 487.039µs)","trace[266823895] 'applied index is now lower than readState.Index' (duration: 528.725734ms)"],"step_count":2} Jul 15 11:00:52.019307 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.017435Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.028303559s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f\" ","response":"range_response_count:1 size:996"} Jul 15 11:00:52.019307 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.017594Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.036102422s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:0 size:5"} Jul 15 11:00:52.019307 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:52.017648Z","caller":"traceutil/trace.go:171","msg":"trace[1049917125] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:0; response_revision:1565; }","duration":"1.036269835s","start":"2025-07-15T11:00:50.981354Z","end":"2025-07-15T11:00:52.017624Z","steps":["trace[1049917125] 'agreement among raft nodes before linearized reading' (duration: 507.25299ms)","trace[1049917125] 'range keys from in-memory index tree' (duration: 528.943243ms)"],"step_count":2} Jul 15 11:00:52.023506 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.017736Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:50.981333Z","time spent":"1.036378314s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":0,"response size":29,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 11:00:52.023506 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:52.017717Z","caller":"traceutil/trace.go:171","msg":"trace[1973937565] range","detail":"{range_begin:/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f; range_end:; response_count:1; response_revision:1565; }","duration":"1.028563861s","start":"2025-07-15T11:00:50.989005Z","end":"2025-07-15T11:00:52.017569Z","steps":["trace[1973937565] 'agreement among raft nodes before linearized reading' (duration: 499.60082ms)","trace[1973937565] 'range keys from in-memory index tree' (duration: 528.382792ms)"],"step_count":2} Jul 15 11:00:52.023506 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.017970Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:50.988989Z","time spent":"1.028951179s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":97,"response count":1,"response size":1020,"request content":"key:\"/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f\" "} Jul 15 11:00:52.023506 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:52.018033Z","caller":"traceutil/trace.go:171","msg":"trace[964189461] transaction","detail":"{read_only:false; number_of_response:0; response_revision:1565; }","duration":"632.242058ms","start":"2025-07-15T11:00:51.385773Z","end":"2025-07-15T11:00:52.018015Z","steps":["trace[964189461] 'process raft request' (duration: 631.411997ms)"],"step_count":1} Jul 15 11:00:52.023926 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.018170Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:51.385752Z","time spent":"632.353282ms","remote":"172.24.4.149:54498","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":29,"request content":"compare: success:> failure:<>"} Jul 15 11:00:52.023926 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.019483Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"634.177471ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-node-lease/ci-4230-2-100-6ef5831daf.novalocal\" ","response":"range_response_count:1 size:640"} Jul 15 11:00:52.023926 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:52.019584Z","caller":"traceutil/trace.go:171","msg":"trace[89991117] range","detail":"{range_begin:/registry/leases/kube-node-lease/ci-4230-2-100-6ef5831daf.novalocal; range_end:; response_count:1; response_revision:1565; }","duration":"634.309425ms","start":"2025-07-15T11:00:51.385254Z","end":"2025-07-15T11:00:52.019564Z","steps":["trace[89991117] 'agreement among raft nodes before linearized reading' (duration: 634.035016ms)"],"step_count":1} Jul 15 11:00:52.023926 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.019697Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:51.385237Z","time spent":"634.381955ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":69,"response count":1,"response size":664,"request content":"key:\"/registry/leases/kube-node-lease/ci-4230-2-100-6ef5831daf.novalocal\" "} Jul 15 11:00:52.024286 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.019933Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"636.042788ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-node-lease/172.24.4.29\" ","response":"range_response_count:1 size:548"} Jul 15 11:00:52.024286 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:52.019983Z","caller":"traceutil/trace.go:171","msg":"trace[846586513] range","detail":"{range_begin:/registry/leases/kube-node-lease/172.24.4.29; range_end:; response_count:1; response_revision:1565; }","duration":"636.127432ms","start":"2025-07-15T11:00:51.383841Z","end":"2025-07-15T11:00:52.019968Z","steps":["trace[846586513] 'agreement among raft nodes before linearized reading' (duration: 635.968675ms)"],"step_count":1} Jul 15 11:00:52.024286 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.020024Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:51.383819Z","time spent":"636.191786ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":46,"response count":1,"response size":572,"request content":"key:\"/registry/leases/kube-node-lease/172.24.4.29\" "} Jul 15 11:00:52.024286 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.020125Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"639.778224ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:00:52.024286 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:52.020165Z","caller":"traceutil/trace.go:171","msg":"trace[1932854644] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1565; }","duration":"639.898766ms","start":"2025-07-15T11:00:51.380252Z","end":"2025-07-15T11:00:52.020151Z","steps":["trace[1932854644] 'agreement among raft nodes before linearized reading' (duration: 639.813782ms)"],"step_count":1} Jul 15 11:00:52.024717 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.020203Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:51.380226Z","time spent":"639.963671ms","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 11:00:52.024717 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.020983Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"424.702233ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowschemas/\" range_end:\"/registry/flowschemas0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:00:52.024717 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:52.021151Z","caller":"traceutil/trace.go:171","msg":"trace[1651076457] range","detail":"{range_begin:/registry/flowschemas/; range_end:/registry/flowschemas0; response_count:0; response_revision:1565; }","duration":"424.877131ms","start":"2025-07-15T11:00:51.596201Z","end":"2025-07-15T11:00:52.021078Z","steps":["trace[1651076457] 'agreement among raft nodes before linearized reading' (duration: 424.647177ms)"],"step_count":1} Jul 15 11:00:52.024717 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.021780Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:51.596168Z","time spent":"425.573514ms","remote":"172.24.4.149:33438","response type":"/etcdserverpb.KV/Range","request count":0,"request size":50,"response count":13,"response size":31,"request content":"key:\"/registry/flowschemas/\" range_end:\"/registry/flowschemas0\" count_only:true "} Jul 15 11:00:52.024717 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.022187Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.035549616s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal\" ","response":"range_response_count:1 size:4500"} Jul 15 11:00:52.025644 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:52.023020Z","caller":"traceutil/trace.go:171","msg":"trace[400564562] range","detail":"{range_begin:/registry/pods/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal; range_end:; response_count:1; response_revision:1565; }","duration":"1.036229418s","start":"2025-07-15T11:00:50.986567Z","end":"2025-07-15T11:00:52.022797Z","steps":["trace[400564562] 'agreement among raft nodes before linearized reading' (duration: 502.026879ms)","trace[400564562] 'range keys from in-memory index tree' (duration: 530.535574ms)"],"step_count":2} Jul 15 11:00:52.025644 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:52.024460Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:50.986548Z","time spent":"1.037878589s","remote":"172.24.4.149:33206","response type":"/etcdserverpb.KV/Range","request count":0,"request size":78,"response count":1,"response size":4524,"request content":"key:\"/registry/pods/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal\" "} Jul 15 11:00:59.284049 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:59.280844Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:52.056864Z","time spent":"7.223970389s","remote":"172.24.4.149:33206","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:00:59.284049 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:59.281734Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:52.044594Z","time spent":"7.237103513s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:00:59.285751 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:59.282493Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:52.044474Z","time spent":"7.238016154s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:00:59.285751 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:59.282845Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"7.234780553s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 11:00:59.285751 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:00:59.285288Z","caller":"traceutil/trace.go:171","msg":"trace[1951557287] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"7.237347115s","start":"2025-07-15T11:00:52.047823Z","end":"2025-07-15T11:00:59.285170Z","steps":["trace[1951557287] 'agreement among raft nodes before linearized reading' (duration: 7.234806884s)"],"step_count":1} Jul 15 11:00:59.285751 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:59.285425Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:52.047804Z","time spent":"7.237567431s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:00:59.285751 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:00:59.283927Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:52.024910Z","time spent":"7.259012186s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.Lease/LeaseGrant","request count":-1,"request size":-1,"response count":-1,"response size":-1,"request content":""} Jul 15 11:01:04.223102 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:03.825487Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.543532802s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:01:04.223102 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:03.825844Z","caller":"traceutil/trace.go:171","msg":"trace[1331817003] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"4.543993701s","start":"2025-07-15T11:00:59.281823Z","end":"2025-07-15T11:01:03.825816Z","steps":["trace[1331817003] 'agreement among raft nodes before linearized reading' (duration: 4.543542951s)"],"step_count":1} Jul 15 11:01:04.223102 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:03.826077Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.281817Z","time spent":"4.544232282s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:01:04.223102 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:03.826319Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.544366631s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:01:04.223102 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:03.826453Z","caller":"traceutil/trace.go:171","msg":"trace[1116926523] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"4.54451611s","start":"2025-07-15T11:00:59.281919Z","end":"2025-07-15T11:01:03.826435Z","steps":["trace[1116926523] 'agreement among raft nodes before linearized reading' (duration: 4.544379686s)"],"step_count":1} Jul 15 11:01:04.224175 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:03.826524Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.281912Z","time spent":"4.544587336s","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:01:04.286696 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.285920Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"12.260013336s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/tiers/default\" ","response":"","error":"context canceled"} Jul 15 11:01:04.286696 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.286023Z","caller":"traceutil/trace.go:171","msg":"trace[2067385608] range","detail":"{range_begin:/registry/crd.projectcalico.org/tiers/default; range_end:; }","duration":"12.260158856s","start":"2025-07-15T11:00:52.025840Z","end":"2025-07-15T11:01:04.285999Z","steps":["trace[2067385608] 'agreement among raft nodes before linearized reading' (duration: 12.260039526s)"],"step_count":1} Jul 15 11:01:04.286696 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.286086Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:52.025822Z","time spent":"12.260245414s","remote":"172.24.4.149:54498","response type":"/etcdserverpb.KV/Range","request count":0,"request size":47,"response count":0,"response size":0,"request content":"key:\"/registry/crd.projectcalico.org/tiers/default\" "} Jul 15 11:01:04.369821 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.369170Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"12.307107795s","expected-duration":"1s"} Jul 15 11:01:04.435427 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.434752Z","caller":"traceutil/trace.go:171","msg":"trace[2143644956] linearizableReadLoop","detail":"{readStateIndex:1877; appliedIndex:1876; }","duration":"12.408647645s","start":"2025-07-15T11:00:52.025876Z","end":"2025-07-15T11:01:04.434524Z","steps":["trace[2143644956] 'read index received' (duration: 36.481142ms)","trace[2143644956] 'applied index is now lower than readState.Index' (duration: 12.37216524s)"],"step_count":2} Jul 15 11:01:04.485840 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.484860Z","caller":"traceutil/trace.go:171","msg":"trace[1650267733] transaction","detail":"{read_only:false; response_revision:1570; number_of_response:1; }","duration":"5.201595156s","start":"2025-07-15T11:00:59.283224Z","end":"2025-07-15T11:01:04.484819Z","steps":["trace[1650267733] 'process raft request' (duration: 5.201260751s)"],"step_count":1} Jul 15 11:01:04.486353 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.485196Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.283208Z","time spent":"5.20178928s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":699,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:01:04.491438 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.488666Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.206206735s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/csistoragecapacities/\" range_end:\"/registry/csistoragecapacities0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:04.491438 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.489047Z","caller":"traceutil/trace.go:171","msg":"trace[857022522] range","detail":"{range_begin:/registry/csistoragecapacities/; range_end:/registry/csistoragecapacities0; response_count:0; response_revision:1570; }","duration":"5.206331886s","start":"2025-07-15T11:00:59.282406Z","end":"2025-07-15T11:01:04.488738Z","steps":["trace[857022522] 'agreement among raft nodes before linearized reading' (duration: 5.205672966s)"],"step_count":1} Jul 15 11:01:04.491438 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.489212Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.282401Z","time spent":"5.206754071s","remote":"172.24.4.149:33422","response type":"/etcdserverpb.KV/Range","request count":0,"request size":68,"response count":0,"response size":29,"request content":"key:\"/registry/csistoragecapacities/\" range_end:\"/registry/csistoragecapacities0\" count_only:true "} Jul 15 11:01:04.491438 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.489149Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"12.422311824s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/tigera-operator/tigera-operator-5bf8dfcb4-b2w9w.1852679c699bec52\" ","response":"range_response_count:1 size:953"} Jul 15 11:01:04.491438 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.489457Z","caller":"traceutil/trace.go:171","msg":"trace[1429761156] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1570; }","duration":"5.188221994s","start":"2025-07-15T11:00:59.301214Z","end":"2025-07-15T11:01:04.489436Z","steps":["trace[1429761156] 'process raft request' (duration: 5.183451469s)"],"step_count":1} Jul 15 11:01:04.492147 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.489464Z","caller":"traceutil/trace.go:171","msg":"trace[128360223] range","detail":"{range_begin:/registry/events/tigera-operator/tigera-operator-5bf8dfcb4-b2w9w.1852679c699bec52; range_end:; response_count:1; response_revision:1570; }","duration":"12.422725011s","start":"2025-07-15T11:00:52.066595Z","end":"2025-07-15T11:01:04.489320Z","steps":["trace[128360223] 'agreement among raft nodes before linearized reading' (duration: 12.420406797s)"],"step_count":1} Jul 15 11:01:04.492147 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.489585Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.301200Z","time spent":"5.188314754s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":71,"response count":0,"response size":674,"request content":"compare: success:> failure: >"} Jul 15 11:01:04.492147 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.489720Z","caller":"traceutil/trace.go:171","msg":"trace[939819545] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1570; }","duration":"5.186125169s","start":"2025-07-15T11:00:59.303555Z","end":"2025-07-15T11:01:04.489680Z","steps":["trace[939819545] 'process raft request' (duration: 5.183330336s)"],"step_count":1} Jul 15 11:01:04.492683 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.489825Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.303516Z","time spent":"5.186256453s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":48,"response count":0,"response size":582,"request content":"compare: success:> failure: >"} Jul 15 11:01:04.492683 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.489644Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:52.066573Z","time spent":"12.423031443s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":83,"response count":1,"response size":977,"request content":"key:\"/registry/events/tigera-operator/tigera-operator-5bf8dfcb4-b2w9w.1852679c699bec52\" "} Jul 15 11:01:04.492683 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.490113Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"167.917709ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:04.492683 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.490219Z","caller":"traceutil/trace.go:171","msg":"trace[17054693] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1570; }","duration":"168.060956ms","start":"2025-07-15T11:01:04.322138Z","end":"2025-07-15T11:01:04.490198Z","steps":["trace[17054693] 'agreement among raft nodes before linearized reading' (duration: 167.923089ms)"],"step_count":1} Jul 15 11:01:04.493182 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.490464Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.208006208s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/policy.networking.k8s.io/baselineadminnetworkpolicies/\" range_end:\"/registry/policy.networking.k8s.io/baselineadminnetworkpolicies0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:04.493182 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.490680Z","caller":"traceutil/trace.go:171","msg":"trace[123845156] range","detail":"{range_begin:/registry/policy.networking.k8s.io/baselineadminnetworkpolicies/; range_end:/registry/policy.networking.k8s.io/baselineadminnetworkpolicies0; response_count:0; response_revision:1570; }","duration":"5.208216444s","start":"2025-07-15T11:00:59.282365Z","end":"2025-07-15T11:01:04.490582Z","steps":["trace[123845156] 'agreement among raft nodes before linearized reading' (duration: 5.207464283s)"],"step_count":1} Jul 15 11:01:04.493182 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.491062Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.282358Z","time spent":"5.208670029s","remote":"172.24.4.149:54632","response type":"/etcdserverpb.KV/Range","request count":0,"request size":134,"response count":0,"response size":29,"request content":"key:\"/registry/policy.networking.k8s.io/baselineadminnetworkpolicies/\" range_end:\"/registry/policy.networking.k8s.io/baselineadminnetworkpolicies0\" count_only:true "} Jul 15 11:01:04.495813 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.491759Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.209530389s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/runtimeclasses/\" range_end:\"/registry/runtimeclasses0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:04.495813 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.491912Z","caller":"traceutil/trace.go:171","msg":"trace[667858722] range","detail":"{range_begin:/registry/runtimeclasses/; range_end:/registry/runtimeclasses0; response_count:0; response_revision:1570; }","duration":"5.209639781s","start":"2025-07-15T11:00:59.282160Z","end":"2025-07-15T11:01:04.491800Z","steps":["trace[667858722] 'agreement among raft nodes before linearized reading' (duration: 5.209527103s)"],"step_count":1} Jul 15 11:01:04.495813 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.492114Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.282155Z","time spent":"5.209807233s","remote":"172.24.4.149:33336","response type":"/etcdserverpb.KV/Range","request count":0,"request size":56,"response count":0,"response size":29,"request content":"key:\"/registry/runtimeclasses/\" range_end:\"/registry/runtimeclasses0\" count_only:true "} Jul 15 11:01:04.495813 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.492671Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.210539626s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiregistration.k8s.io/apiservices/\" range_end:\"/registry/apiregistration.k8s.io/apiservices0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:01:04.496315 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.492767Z","caller":"traceutil/trace.go:171","msg":"trace[930480197] range","detail":"{range_begin:/registry/apiregistration.k8s.io/apiservices/; range_end:/registry/apiregistration.k8s.io/apiservices0; response_count:0; response_revision:1570; }","duration":"5.210673435s","start":"2025-07-15T11:00:59.282079Z","end":"2025-07-15T11:01:04.492752Z","steps":["trace[930480197] 'agreement among raft nodes before linearized reading' (duration: 5.210487376s)"],"step_count":1} Jul 15 11:01:04.496315 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.492845Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.282074Z","time spent":"5.210720916s","remote":"172.24.4.149:33534","response type":"/etcdserverpb.KV/Range","request count":0,"request size":96,"response count":25,"response size":31,"request content":"key:\"/registry/apiregistration.k8s.io/apiservices/\" range_end:\"/registry/apiregistration.k8s.io/apiservices0\" count_only:true "} Jul 15 11:01:04.496315 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.493195Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.211144775s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/\" range_end:\"/registry/leases0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:01:04.496315 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.493299Z","caller":"traceutil/trace.go:171","msg":"trace[1656166843] range","detail":"{range_begin:/registry/leases/; range_end:/registry/leases0; response_count:0; response_revision:1570; }","duration":"5.211269254s","start":"2025-07-15T11:00:59.282020Z","end":"2025-07-15T11:01:04.493289Z","steps":["trace[1656166843] 'agreement among raft nodes before linearized reading' (duration: 5.211095568s)"],"step_count":1} Jul 15 11:01:04.497654 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.493375Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.282014Z","time spent":"5.211307198s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":40,"response count":6,"response size":31,"request content":"key:\"/registry/leases/\" range_end:\"/registry/leases0\" count_only:true "} Jul 15 11:01:04.497654 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.493507Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.211756304s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/resourcequotas/\" range_end:\"/registry/resourcequotas0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:04.497654 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.493547Z","caller":"traceutil/trace.go:171","msg":"trace[300898850] range","detail":"{range_begin:/registry/resourcequotas/; range_end:/registry/resourcequotas0; response_count:0; response_revision:1570; }","duration":"5.211783958s","start":"2025-07-15T11:00:59.281741Z","end":"2025-07-15T11:01:04.493525Z","steps":["trace[300898850] 'agreement among raft nodes before linearized reading' (duration: 5.211707751s)"],"step_count":1} Jul 15 11:01:04.497654 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.493722Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.281721Z","time spent":"5.21189377s","remote":"172.24.4.149:33112","response type":"/etcdserverpb.KV/Range","request count":0,"request size":56,"response count":0,"response size":29,"request content":"key:\"/registry/resourcequotas/\" range_end:\"/registry/resourcequotas0\" count_only:true "} Jul 15 11:01:04.497654 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.494038Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.212269495s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/\" range_end:\"/registry/clusterroles0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:01:04.499461 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.494122Z","caller":"traceutil/trace.go:171","msg":"trace[924897378] range","detail":"{range_begin:/registry/clusterroles/; range_end:/registry/clusterroles0; response_count:0; response_revision:1570; }","duration":"5.21241724s","start":"2025-07-15T11:00:59.281668Z","end":"2025-07-15T11:01:04.494086Z","steps":["trace[924897378] 'agreement among raft nodes before linearized reading' (duration: 5.212224047s)"],"step_count":1} Jul 15 11:01:04.499461 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.494177Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.281663Z","time spent":"5.21250062s","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":77,"response size":31,"request content":"key:\"/registry/clusterroles/\" range_end:\"/registry/clusterroles0\" count_only:true "} Jul 15 11:01:04.499461 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.494346Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.21270747s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions\" limit:1 ","response":"range_response_count:0 size:5"} Jul 15 11:01:04.499461 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.494369Z","caller":"traceutil/trace.go:171","msg":"trace[851575271] range","detail":"{range_begin:/registry/minions; range_end:; response_count:0; response_revision:1570; }","duration":"5.212746374s","start":"2025-07-15T11:00:59.281615Z","end":"2025-07-15T11:01:04.494362Z","steps":["trace[851575271] 'agreement among raft nodes before linearized reading' (duration: 5.212706357s)"],"step_count":1} Jul 15 11:01:04.499461 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.494406Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.281607Z","time spent":"5.21279045s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Range","request count":0,"request size":21,"response count":0,"response size":29,"request content":"key:\"/registry/minions\" limit:1 "} Jul 15 11:01:04.499978 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.494520Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.213277189s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/bgpfilters/\" range_end:\"/registry/crd.projectcalico.org/bgpfilters0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:04.499978 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.494541Z","caller":"traceutil/trace.go:171","msg":"trace[584698412] range","detail":"{range_begin:/registry/crd.projectcalico.org/bgpfilters/; range_end:/registry/crd.projectcalico.org/bgpfilters0; response_count:0; response_revision:1570; }","duration":"5.213299954s","start":"2025-07-15T11:00:59.281235Z","end":"2025-07-15T11:01:04.494535Z","steps":["trace[584698412] 'agreement among raft nodes before linearized reading' (duration: 5.213266529s)"],"step_count":1} Jul 15 11:01:04.499978 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.494568Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.281207Z","time spent":"5.213346534s","remote":"172.24.4.149:54664","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/bgpfilters/\" range_end:\"/registry/crd.projectcalico.org/bgpfilters0\" count_only:true "} Jul 15 11:01:04.499978 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.494600Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"216.905895ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/endpoints/calico-system/calico-typha\" ","response":"range_response_count:1 size:688"} Jul 15 11:01:04.500542 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.494759Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.213617475s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/tigerastatuses/\" range_end:\"/registry/operator.tigera.io/tigerastatuses0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:01:04.500542 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.494872Z","caller":"traceutil/trace.go:171","msg":"trace[1276914279] range","detail":"{range_begin:/registry/operator.tigera.io/tigerastatuses/; range_end:/registry/operator.tigera.io/tigerastatuses0; response_count:0; response_revision:1570; }","duration":"5.213700696s","start":"2025-07-15T11:00:59.281102Z","end":"2025-07-15T11:01:04.494802Z","steps":["trace[1276914279] 'agreement among raft nodes before linearized reading' (duration: 5.213615873s)"],"step_count":1} Jul 15 11:01:04.500542 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.494956Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.281087Z","time spent":"5.213834265s","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Range","request count":0,"request size":94,"response count":5,"response size":31,"request content":"key:\"/registry/operator.tigera.io/tigerastatuses/\" range_end:\"/registry/operator.tigera.io/tigerastatuses0\" count_only:true "} Jul 15 11:01:04.500542 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.494871Z","caller":"traceutil/trace.go:171","msg":"trace[555458321] range","detail":"{range_begin:/registry/services/endpoints/calico-system/calico-typha; range_end:; response_count:1; response_revision:1570; }","duration":"217.070402ms","start":"2025-07-15T11:01:04.277598Z","end":"2025-07-15T11:01:04.494669Z","steps":["trace[555458321] 'agreement among raft nodes before linearized reading' (duration: 212.705149ms)"],"step_count":1} Jul 15 11:01:04.500954 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.496369Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"223.776382ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/globalnetworkpolicies/\" range_end:\"/registry/crd.projectcalico.org/globalnetworkpolicies0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:04.500954 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.496465Z","caller":"traceutil/trace.go:171","msg":"trace[1177868275] range","detail":"{range_begin:/registry/crd.projectcalico.org/globalnetworkpolicies/; range_end:/registry/crd.projectcalico.org/globalnetworkpolicies0; response_count:0; response_revision:1570; }","duration":"223.92509ms","start":"2025-07-15T11:01:04.272526Z","end":"2025-07-15T11:01:04.496451Z","steps":["trace[1177868275] 'agreement among raft nodes before linearized reading' (duration: 223.806301ms)"],"step_count":1} Jul 15 11:01:04.500954 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.496802Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"230.905461ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:04.500954 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.496861Z","caller":"traceutil/trace.go:171","msg":"trace[1051796441] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:0; response_revision:1570; }","duration":"231.155273ms","start":"2025-07-15T11:01:04.265696Z","end":"2025-07-15T11:01:04.496852Z","steps":["trace[1051796441] 'agreement among raft nodes before linearized reading' (duration: 231.041283ms)"],"step_count":1} Jul 15 11:01:04.501359 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.496981Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"235.847719ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/mutatingwebhookconfigurations/\" range_end:\"/registry/mutatingwebhookconfigurations0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:04.501359 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.497198Z","caller":"traceutil/trace.go:171","msg":"trace[1104741832] range","detail":"{range_begin:/registry/mutatingwebhookconfigurations/; range_end:/registry/mutatingwebhookconfigurations0; response_count:0; response_revision:1570; }","duration":"236.100767ms","start":"2025-07-15T11:01:04.261087Z","end":"2025-07-15T11:01:04.497188Z","steps":["trace[1104741832] 'agreement among raft nodes before linearized reading' (duration: 235.873799ms)"],"step_count":1} Jul 15 11:01:04.501359 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.497332Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"237.728458ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/validatingwebhookconfigurations/\" range_end:\"/registry/validatingwebhookconfigurations0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:04.501359 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.497362Z","caller":"traceutil/trace.go:171","msg":"trace[2069522044] range","detail":"{range_begin:/registry/validatingwebhookconfigurations/; range_end:/registry/validatingwebhookconfigurations0; response_count:0; response_revision:1570; }","duration":"237.753626ms","start":"2025-07-15T11:01:04.259594Z","end":"2025-07-15T11:01:04.497347Z","steps":["trace[2069522044] 'agreement among raft nodes before linearized reading' (duration: 237.718499ms)"],"step_count":1} Jul 15 11:01:04.501869 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.497554Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.197437697s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/tigera-operator/tigera-operator-5bf8dfcb4-b2w9w\" ","response":"range_response_count:1 size:4117"} Jul 15 11:01:04.501869 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.497579Z","caller":"traceutil/trace.go:171","msg":"trace[19389869] range","detail":"{range_begin:/registry/pods/tigera-operator/tigera-operator-5bf8dfcb4-b2w9w; range_end:; response_count:1; response_revision:1570; }","duration":"5.197514346s","start":"2025-07-15T11:00:59.300057Z","end":"2025-07-15T11:01:04.497572Z","steps":["trace[19389869] 'agreement among raft nodes before linearized reading' (duration: 5.197408031s)"],"step_count":1} Jul 15 11:01:04.501869 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.497599Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:00:59.300022Z","time spent":"5.197569443s","remote":"172.24.4.149:33206","response type":"/etcdserverpb.KV/Range","request count":0,"request size":64,"response count":1,"response size":4141,"request content":"key:\"/registry/pods/tigera-operator/tigera-operator-5bf8dfcb4-b2w9w\" "} Jul 15 11:01:04.710117 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.707581Z","caller":"traceutil/trace.go:171","msg":"trace[1820658646] linearizableReadLoop","detail":"{readStateIndex:1886; appliedIndex:1884; }","duration":"186.688133ms","start":"2025-07-15T11:01:04.520856Z","end":"2025-07-15T11:01:04.707544Z","steps":["trace[1820658646] 'read index received' (duration: 132.390402ms)","trace[1820658646] 'applied index is now lower than readState.Index' (duration: 54.296318ms)"],"step_count":2} Jul 15 11:01:04.710117 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.707827Z","caller":"traceutil/trace.go:171","msg":"trace[1378252302] transaction","detail":"{read_only:false; response_revision:1572; number_of_response:1; }","duration":"191.469199ms","start":"2025-07-15T11:01:04.516329Z","end":"2025-07-15T11:01:04.707798Z","steps":["trace[1378252302] 'process raft request' (duration: 190.768126ms)"],"step_count":1} Jul 15 11:01:04.710117 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.708958Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"181.825889ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76\" ","response":"range_response_count:1 size:905"} Jul 15 11:01:04.710117 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.709079Z","caller":"traceutil/trace.go:171","msg":"trace[338215056] range","detail":"{range_begin:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76; range_end:; response_count:1; response_revision:1572; }","duration":"181.996638ms","start":"2025-07-15T11:01:04.527042Z","end":"2025-07-15T11:01:04.709038Z","steps":["trace[338215056] 'agreement among raft nodes before linearized reading' (duration: 181.574914ms)"],"step_count":1} Jul 15 11:01:04.711266 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.709093Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"204.287555ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-node-lease/172.24.4.29\" ","response":"range_response_count:1 size:547"} Jul 15 11:01:04.711266 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.709142Z","caller":"traceutil/trace.go:171","msg":"trace[1665136717] range","detail":"{range_begin:/registry/leases/kube-node-lease/172.24.4.29; range_end:; response_count:1; response_revision:1572; }","duration":"204.350687ms","start":"2025-07-15T11:01:04.504776Z","end":"2025-07-15T11:01:04.709126Z","steps":["trace[1665136717] 'agreement among raft nodes before linearized reading' (duration: 203.195447ms)"],"step_count":1} Jul 15 11:01:04.711266 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:04.709325Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"204.541445ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-node-lease/ci-4230-2-100-6ef5831daf.novalocal\" ","response":"range_response_count:1 size:639"} Jul 15 11:01:04.711266 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:04.710352Z","caller":"traceutil/trace.go:171","msg":"trace[1961063212] range","detail":"{range_begin:/registry/leases/kube-node-lease/ci-4230-2-100-6ef5831daf.novalocal; range_end:; response_count:1; response_revision:1572; }","duration":"205.587413ms","start":"2025-07-15T11:01:04.504739Z","end":"2025-07-15T11:01:04.710326Z","steps":["trace[1961063212] 'agreement among raft nodes before linearized reading' (duration: 204.466961ms)"],"step_count":1} Jul 15 11:01:10.348422 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:10.326016Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000062344s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:01:10.348422 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:10.326987Z","caller":"traceutil/trace.go:171","msg":"trace[1793766057] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.00140259s","start":"2025-07-15T11:01:08.325424Z","end":"2025-07-15T11:01:10.326827Z","steps":["trace[1793766057] 'agreement among raft nodes before linearized reading' (duration: 2.000110417s)"],"step_count":1} Jul 15 11:01:10.348422 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:10.327512Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:08.325363Z","time spent":"2.001908467s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:01:10.638174 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:10.635182Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.810466879s","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 11:01:10.641331 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:10.638232Z","caller":"traceutil/trace.go:171","msg":"trace[489779648] linearizableReadLoop","detail":"{readStateIndex:1889; appliedIndex:1887; }","duration":"5.906715867s","start":"2025-07-15T11:01:04.731433Z","end":"2025-07-15T11:01:10.638149Z","steps":["trace[489779648] 'read index received' (duration: 57.357696ms)","trace[489779648] 'applied index is now lower than readState.Index' (duration: 5.849355065s)"],"step_count":2} Jul 15 11:01:10.641331 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:10.638822Z","caller":"traceutil/trace.go:171","msg":"trace[632389505] transaction","detail":"{read_only:false; response_revision:1575; number_of_response:1; }","duration":"5.90975919s","start":"2025-07-15T11:01:04.729038Z","end":"2025-07-15T11:01:10.638797Z","steps":["trace[632389505] 'process raft request' (duration: 94.692645ms)","trace[632389505] 'compare' (duration: 5.809429335s)"],"step_count":2} Jul 15 11:01:10.641331 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:10.639014Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:04.729015Z","time spent":"5.909917416s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":533,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:01:10.641331 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:10.639408Z","caller":"traceutil/trace.go:171","msg":"trace[269061812] transaction","detail":"{read_only:false; response_revision:1576; number_of_response:1; }","duration":"5.898201947s","start":"2025-07-15T11:01:04.738986Z","end":"2025-07-15T11:01:10.637188Z","steps":["trace[269061812] 'process raft request' (duration: 5.896734245s)"],"step_count":1} Jul 15 11:01:10.642293 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:10.639917Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:04.738964Z","time spent":"5.900756888s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":625,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:01:11.667980 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.666480Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.030336097s","expected-duration":"100ms","prefix":"","request":"header: lease_revoke:","response":"size:29"} Jul 15 11:01:11.667980 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.666631Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"6.93522522s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/endpoints/default/kubernetes\" ","response":"range_response_count:1 size:420"} Jul 15 11:01:11.667980 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:11.666710Z","caller":"traceutil/trace.go:171","msg":"trace[216984801] range","detail":"{range_begin:/registry/services/endpoints/default/kubernetes; range_end:; response_count:1; response_revision:1576; }","duration":"6.935340311s","start":"2025-07-15T11:01:04.731342Z","end":"2025-07-15T11:01:11.666682Z","steps":["trace[216984801] 'agreement among raft nodes before linearized reading' (duration: 5.907137951s)","trace[216984801] 'range keys from in-memory index tree' (duration: 1.028016941s)"],"step_count":2} Jul 15 11:01:11.667980 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.666787Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:04.731325Z","time spent":"6.935443199s","remote":"172.24.4.149:33194","response type":"/etcdserverpb.KV/Range","request count":0,"request size":49,"response count":1,"response size":444,"request content":"key:\"/registry/services/endpoints/default/kubernetes\" "} Jul 15 11:01:11.667980 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:11.667190Z","caller":"traceutil/trace.go:171","msg":"trace[1652935045] transaction","detail":"{read_only:false; response_revision:1577; number_of_response:1; }","duration":"144.8945ms","start":"2025-07-15T11:01:11.522277Z","end":"2025-07-15T11:01:11.667172Z","steps":["trace[1652935045] 'process raft request' (duration: 144.353376ms)"],"step_count":1} Jul 15 11:01:11.670713 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:11.667713Z","caller":"traceutil/trace.go:171","msg":"trace[1953807735] linearizableReadLoop","detail":"{readStateIndex:1891; appliedIndex:1890; }","duration":"1.029356878s","start":"2025-07-15T11:01:10.638335Z","end":"2025-07-15T11:01:11.667692Z","steps":["trace[1953807735] 'read index received' (duration: 178.155µs)","trace[1953807735] 'applied index is now lower than readState.Index' (duration: 1.02917669s)"],"step_count":2} Jul 15 11:01:11.721631 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.717852Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.409299291s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/ippools/\" range_end:\"/registry/crd.projectcalico.org/ippools0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:01:11.721631 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:11.718014Z","caller":"traceutil/trace.go:171","msg":"trace[342190056] range","detail":"{range_begin:/registry/crd.projectcalico.org/ippools/; range_end:/registry/crd.projectcalico.org/ippools0; response_count:0; response_revision:1577; }","duration":"3.409580174s","start":"2025-07-15T11:01:08.308367Z","end":"2025-07-15T11:01:11.717947Z","steps":["trace[342190056] 'agreement among raft nodes before linearized reading' (duration: 3.360429104s)","trace[342190056] 'count revisions from in-memory index tree' (duration: 48.85097ms)"],"step_count":2} Jul 15 11:01:11.721631 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.718100Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:08.307668Z","time spent":"3.41040675s","remote":"172.24.4.149:54896","response type":"/etcdserverpb.KV/Range","request count":0,"request size":86,"response count":1,"response size":31,"request content":"key:\"/registry/crd.projectcalico.org/ippools/\" range_end:\"/registry/crd.projectcalico.org/ippools0\" count_only:true "} Jul 15 11:01:11.721631 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.718589Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.665645558s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/horizontalpodautoscalers/\" range_end:\"/registry/horizontalpodautoscalers0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:11.724866 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.718749Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"6.987143149s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/kube-system/kube-controller-manager-ci-4230-2-100-6ef5831daf.novalocal\" ","response":"range_response_count:1 size:6582"} Jul 15 11:01:11.724866 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:11.718770Z","caller":"traceutil/trace.go:171","msg":"trace[277840988] range","detail":"{range_begin:/registry/horizontalpodautoscalers/; range_end:/registry/horizontalpodautoscalers0; response_count:0; response_revision:1577; }","duration":"1.666068584s","start":"2025-07-15T11:01:10.052653Z","end":"2025-07-15T11:01:11.718722Z","steps":["trace[277840988] 'agreement among raft nodes before linearized reading' (duration: 1.616182977s)","trace[277840988] 'count revisions from in-memory index tree' (duration: 49.685521ms)"],"step_count":2} Jul 15 11:01:11.724866 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.718854Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:10.052634Z","time spent":"1.666194477s","remote":"172.24.4.149:33242","response type":"/etcdserverpb.KV/Range","request count":0,"request size":76,"response count":0,"response size":29,"request content":"key:\"/registry/horizontalpodautoscalers/\" range_end:\"/registry/horizontalpodautoscalers0\" count_only:true "} Jul 15 11:01:11.724866 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:11.718871Z","caller":"traceutil/trace.go:171","msg":"trace[818712389] range","detail":"{range_begin:/registry/pods/kube-system/kube-controller-manager-ci-4230-2-100-6ef5831daf.novalocal; range_end:; response_count:1; response_revision:1577; }","duration":"6.987231057s","start":"2025-07-15T11:01:04.731563Z","end":"2025-07-15T11:01:11.718794Z","steps":["trace[818712389] 'agreement among raft nodes before linearized reading' (duration: 6.937128872s)","trace[818712389] 'range keys from in-memory index tree' (duration: 49.775434ms)"],"step_count":2} Jul 15 11:01:11.726495 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.718946Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:04.731551Z","time spent":"6.987365418s","remote":"172.24.4.149:33206","response type":"/etcdserverpb.KV/Range","request count":0,"request size":87,"response count":1,"response size":6606,"request content":"key:\"/registry/pods/kube-system/kube-controller-manager-ci-4230-2-100-6ef5831daf.novalocal\" "} Jul 15 11:01:11.726495 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.719266Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.389372743s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/ci-4230-2-100-6ef5831daf.novalocal\" ","response":"range_response_count:1 size:6690"} Jul 15 11:01:11.726495 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:11.719326Z","caller":"traceutil/trace.go:171","msg":"trace[1086780684] range","detail":"{range_begin:/registry/minions/ci-4230-2-100-6ef5831daf.novalocal; range_end:; response_count:1; response_revision:1577; }","duration":"3.389463158s","start":"2025-07-15T11:01:08.329842Z","end":"2025-07-15T11:01:11.719305Z","steps":["trace[1086780684] 'agreement among raft nodes before linearized reading' (duration: 3.339000137s)","trace[1086780684] 'range keys from in-memory index tree' (duration: 50.243447ms)"],"step_count":2} Jul 15 11:01:11.726495 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.719786Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.123512932s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/calico-system/calico-cni-plugin\" ","response":"range_response_count:1 size:537"} Jul 15 11:01:11.726982 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:11.720014Z","caller":"traceutil/trace.go:171","msg":"trace[2033022051] range","detail":"{range_begin:/registry/serviceaccounts/calico-system/calico-cni-plugin; range_end:; response_count:1; response_revision:1577; }","duration":"1.123799365s","start":"2025-07-15T11:01:10.596189Z","end":"2025-07-15T11:01:11.719988Z","steps":["trace[2033022051] 'agreement among raft nodes before linearized reading' (duration: 1.072539244s)","trace[2033022051] 'range keys from in-memory index tree' (duration: 50.929292ms)"],"step_count":2} Jul 15 11:01:11.726982 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.720142Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:10.596079Z","time spent":"1.124038156s","remote":"172.24.4.149:33226","response type":"/etcdserverpb.KV/Range","request count":0,"request size":59,"response count":1,"response size":561,"request content":"key:\"/registry/serviceaccounts/calico-system/calico-cni-plugin\" "} Jul 15 11:01:11.726982 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.719506Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:08.329829Z","time spent":"3.389600713s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Range","request count":0,"request size":54,"response count":1,"response size":6714,"request content":"key:\"/registry/minions/ci-4230-2-100-6ef5831daf.novalocal\" "} Jul 15 11:01:11.726982 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.721012Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.400866938s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"range_response_count:1 size:886"} Jul 15 11:01:11.727460 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:11.721082Z","caller":"traceutil/trace.go:171","msg":"trace[106038867] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; response_count:1; response_revision:1577; }","duration":"3.401034811s","start":"2025-07-15T11:01:08.320027Z","end":"2025-07-15T11:01:11.721062Z","steps":["trace[106038867] 'agreement among raft nodes before linearized reading' (duration: 3.348819407s)","trace[106038867] 'range keys from in-memory index tree' (duration: 52.02695ms)"],"step_count":2} Jul 15 11:01:11.727460 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.721164Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:08.319981Z","time spent":"3.401143091s","remote":"172.24.4.149:54994","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":1,"response size":910,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" "} Jul 15 11:01:11.727460 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.722350Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"824.170989ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/\" range_end:\"/registry/clusterrolebindings0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:01:11.727460 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:11.722542Z","caller":"traceutil/trace.go:171","msg":"trace[1107099231] range","detail":"{range_begin:/registry/clusterrolebindings/; range_end:/registry/clusterrolebindings0; response_count:0; response_revision:1577; }","duration":"824.42063ms","start":"2025-07-15T11:01:10.898097Z","end":"2025-07-15T11:01:11.722518Z","steps":["trace[1107099231] 'agreement among raft nodes before linearized reading' (duration: 770.638091ms)","trace[1107099231] 'count revisions from in-memory index tree' (duration: 53.532878ms)"],"step_count":2} Jul 15 11:01:11.728237 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.722668Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:10.898064Z","time spent":"824.557685ms","remote":"172.24.4.149:33376","response type":"/etcdserverpb.KV/Range","request count":0,"request size":66,"response count":63,"response size":31,"request content":"key:\"/registry/clusterrolebindings/\" range_end:\"/registry/clusterrolebindings0\" count_only:true "} Jul 15 11:01:11.728237 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.723165Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"6.816280778s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/\" range_end:\"/registry/serviceaccounts0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:01:11.728237 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:11.723817Z","caller":"traceutil/trace.go:171","msg":"trace[1510070975] range","detail":"{range_begin:/registry/serviceaccounts/; range_end:/registry/serviceaccounts0; response_count:0; response_revision:1577; }","duration":"6.816423203s","start":"2025-07-15T11:01:04.906811Z","end":"2025-07-15T11:01:11.723234Z","steps":["trace[1510070975] 'agreement among raft nodes before linearized reading' (duration: 6.76204245s)","trace[1510070975] 'count revisions from in-memory index tree' (duration: 54.270721ms)"],"step_count":2} Jul 15 11:01:11.728237 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.723518Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.368288316s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:11.729801 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:11.724046Z","caller":"traceutil/trace.go:171","msg":"trace[1088752613] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1577; }","duration":"1.368871411s","start":"2025-07-15T11:01:10.355152Z","end":"2025-07-15T11:01:11.724024Z","steps":["trace[1088752613] 'agreement among raft nodes before linearized reading' (duration: 1.313640687s)","trace[1088752613] 'range keys from in-memory index tree' (duration: 54.695581ms)"],"step_count":2} Jul 15 11:01:11.729801 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.724217Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:10.355115Z","time spent":"1.36901548s","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 11:01:11.729801 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.724703Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"6.791734421s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/csinodes/\" range_end:\"/registry/csinodes0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:01:11.729801 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:11.725574Z","caller":"traceutil/trace.go:171","msg":"trace[535807910] range","detail":"{range_begin:/registry/csinodes/; range_end:/registry/csinodes0; response_count:0; response_revision:1577; }","duration":"6.792600392s","start":"2025-07-15T11:01:04.932909Z","end":"2025-07-15T11:01:11.725509Z","steps":["trace[535807910] 'agreement among raft nodes before linearized reading' (duration: 6.735908237s)","trace[535807910] 'count revisions from in-memory index tree' (duration: 55.851443ms)"],"step_count":2} Jul 15 11:01:11.730261 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.725681Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:04.932887Z","time spent":"6.792768266s","remote":"172.24.4.149:33400","response type":"/etcdserverpb.KV/Range","request count":0,"request size":44,"response count":2,"response size":31,"request content":"key:\"/registry/csinodes/\" range_end:\"/registry/csinodes0\" count_only:true "} Jul 15 11:01:11.730261 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.723913Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:04.906784Z","time spent":"6.817101652s","remote":"172.24.4.149:33226","response type":"/etcdserverpb.KV/Range","request count":0,"request size":58,"response count":51,"response size":31,"request content":"key:\"/registry/serviceaccounts/\" range_end:\"/registry/serviceaccounts0\" count_only:true "} Jul 15 11:01:11.730261 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.725041Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"6.929739096s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f\" ","response":"range_response_count:1 size:996"} Jul 15 11:01:11.730261 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:11.726743Z","caller":"traceutil/trace.go:171","msg":"trace[1209615756] range","detail":"{range_begin:/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f; range_end:; response_count:1; response_revision:1577; }","duration":"6.931478233s","start":"2025-07-15T11:01:04.795227Z","end":"2025-07-15T11:01:11.726706Z","steps":["trace[1209615756] 'agreement among raft nodes before linearized reading' (duration: 6.873657108s)","trace[1209615756] 'range keys from in-memory index tree' (duration: 55.999198ms)"],"step_count":2} Jul 15 11:01:11.730744 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:11.726834Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:04.795208Z","time spent":"6.931581481s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":97,"response count":1,"response size":1020,"request content":"key:\"/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f\" "} Jul 15 11:01:12.010716 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:12.009277Z","caller":"traceutil/trace.go:171","msg":"trace[1757293553] linearizableReadLoop","detail":"{readStateIndex:1898; appliedIndex:1897; }","duration":"110.411093ms","start":"2025-07-15T11:01:11.898831Z","end":"2025-07-15T11:01:12.009242Z","steps":["trace[1757293553] 'read index received' (duration: 59.503813ms)","trace[1757293553] 'applied index is now lower than readState.Index' (duration: 50.906148ms)"],"step_count":2} Jul 15 11:01:12.010716 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:12.009488Z","caller":"traceutil/trace.go:171","msg":"trace[1008356217] transaction","detail":"{read_only:false; response_revision:1581; number_of_response:1; }","duration":"113.310879ms","start":"2025-07-15T11:01:11.896152Z","end":"2025-07-15T11:01:12.009463Z","steps":["trace[1008356217] 'process raft request' (duration: 62.352651ms)","trace[1008356217] 'compare' (duration: 50.118658ms)"],"step_count":2} Jul 15 11:01:12.010716 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:12.009906Z","caller":"traceutil/trace.go:171","msg":"trace[1977262861] transaction","detail":"{read_only:false; response_revision:1582; number_of_response:1; }","duration":"107.019646ms","start":"2025-07-15T11:01:11.902866Z","end":"2025-07-15T11:01:12.009886Z","steps":["trace[1977262861] 'process raft request' (duration: 106.274367ms)"],"step_count":1} Jul 15 11:01:12.011825 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:12.011049Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"112.181049ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/tiers/adminnetworkpolicy\" ","response":"range_response_count:1 size:721"} Jul 15 11:01:12.011825 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:12.011173Z","caller":"traceutil/trace.go:171","msg":"trace[1407317053] range","detail":"{range_begin:/registry/crd.projectcalico.org/tiers/adminnetworkpolicy; range_end:; response_count:1; response_revision:1582; }","duration":"112.353462ms","start":"2025-07-15T11:01:11.898791Z","end":"2025-07-15T11:01:12.011144Z","steps":["trace[1407317053] 'agreement among raft nodes before linearized reading' (duration: 110.635816ms)"],"step_count":1} Jul 15 11:01:27.121080 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:25.950457Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885076232,"retry-timeout":"500ms"} Jul 15 11:01:27.121080 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:26.451567Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885076232,"retry-timeout":"500ms"} Jul 15 11:01:27.121080 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:26.952188Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885076232,"retry-timeout":"500ms"} Jul 15 11:01:27.454880 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:27.453546Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885076232,"retry-timeout":"500ms"} Jul 15 11:01:27.562490 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:27.561565Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"2.84866781s","expected-duration":"1s"} Jul 15 11:01:27.564021 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:27.562554Z","caller":"traceutil/trace.go:171","msg":"trace[1184378504] transaction","detail":"{read_only:false; response_revision:1620; number_of_response:1; }","duration":"2.849764969s","start":"2025-07-15T11:01:24.712729Z","end":"2025-07-15T11:01:27.562494Z","steps":["trace[1184378504] 'process raft request' (duration: 2.849260335s)"],"step_count":1} Jul 15 11:01:27.564021 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:27.563110Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:24.712693Z","time spent":"2.849997768s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":699,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:01:27.581249 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:27.577754Z","caller":"traceutil/trace.go:171","msg":"trace[103019159] linearizableReadLoop","detail":"{readStateIndex:1978; appliedIndex:1977; }","duration":"2.128127658s","start":"2025-07-15T11:01:25.449564Z","end":"2025-07-15T11:01:27.577692Z","steps":["trace[103019159] 'read index received' (duration: 2.113660693s)","trace[103019159] 'applied index is now lower than readState.Index' (duration: 14.465392ms)"],"step_count":2} Jul 15 11:01:27.581249 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:27.577961Z","caller":"traceutil/trace.go:171","msg":"trace[1656163562] transaction","detail":"{read_only:false; response_revision:1621; number_of_response:1; }","duration":"457.697307ms","start":"2025-07-15T11:01:27.120241Z","end":"2025-07-15T11:01:27.577939Z","steps":["trace[1656163562] 'process raft request' (duration: 457.286795ms)"],"step_count":1} Jul 15 11:01:27.581249 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:27.578152Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:27.120215Z","time spent":"457.796609ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":488,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:01:27.581249 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:27.578238Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.613518642s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:27.582433 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:27.578367Z","caller":"traceutil/trace.go:171","msg":"trace[746166190] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1621; }","duration":"1.613647321s","start":"2025-07-15T11:01:25.964685Z","end":"2025-07-15T11:01:27.578332Z","steps":["trace[746166190] 'agreement among raft nodes before linearized reading' (duration: 1.613483144s)"],"step_count":1} Jul 15 11:01:27.582433 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:27.578585Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:25.964598Z","time spent":"1.613960636s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 11:01:27.582433 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:27.578592Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.129026043s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/tigera-operator/operator-lock\" ","response":"range_response_count:1 size:505"} Jul 15 11:01:27.582433 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:27.578807Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"617.652648ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/statefulsets/\" range_end:\"/registry/statefulsets0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:27.582433 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:27.578932Z","caller":"traceutil/trace.go:171","msg":"trace[353134650] range","detail":"{range_begin:/registry/statefulsets/; range_end:/registry/statefulsets0; response_count:0; response_revision:1621; }","duration":"617.796205ms","start":"2025-07-15T11:01:26.961067Z","end":"2025-07-15T11:01:27.578862Z","steps":["trace[353134650] 'agreement among raft nodes before linearized reading' (duration: 617.673979ms)"],"step_count":1} Jul 15 11:01:27.582911 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:27.579011Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:26.961037Z","time spent":"617.941215ms","remote":"172.24.4.149:33472","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":0,"response size":29,"request content":"key:\"/registry/statefulsets/\" range_end:\"/registry/statefulsets0\" count_only:true "} Jul 15 11:01:27.582911 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:27.578687Z","caller":"traceutil/trace.go:171","msg":"trace[1842861295] range","detail":"{range_begin:/registry/leases/tigera-operator/operator-lock; range_end:; response_count:1; response_revision:1621; }","duration":"2.129147998s","start":"2025-07-15T11:01:25.449498Z","end":"2025-07-15T11:01:27.578646Z","steps":["trace[1842861295] 'agreement among raft nodes before linearized reading' (duration: 2.128944484s)"],"step_count":1} Jul 15 11:01:27.582911 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:27.580261Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:25.449462Z","time spent":"2.130765221s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":1,"response size":529,"request content":"key:\"/registry/leases/tigera-operator/operator-lock\" "} Jul 15 11:01:32.630547 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:32.627611Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885076248,"retry-timeout":"500ms"} Jul 15 11:01:33.130496 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:33.129192Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885076248,"retry-timeout":"500ms"} Jul 15 11:01:33.630293 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:33.629707Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885076248,"retry-timeout":"500ms"} Jul 15 11:01:34.131604 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:34.130767Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885076248,"retry-timeout":"500ms"} Jul 15 11:01:35.792342 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.776192Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.796658308s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:01:35.792342 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.779698Z","caller":"traceutil/trace.go:171","msg":"trace[114753886] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.800301895s","start":"2025-07-15T11:01:32.979162Z","end":"2025-07-15T11:01:35.779464Z","steps":["trace[114753886] 'agreement among raft nodes before linearized reading' (duration: 2.796735097s)"],"step_count":1} Jul 15 11:01:35.792342 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.780091Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:32.979046Z","time spent":"2.80098788s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:01:35.792342 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.776999Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885076248,"retry-timeout":"500ms"} Jul 15 11:01:35.837958 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.815709Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"4.575558626s","expected-duration":"1s"} Jul 15 11:01:35.837958 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.818574Z","caller":"traceutil/trace.go:171","msg":"trace[1943040970] transaction","detail":"{read_only:false; response_revision:1623; number_of_response:1; }","duration":"4.578509574s","start":"2025-07-15T11:01:31.239969Z","end":"2025-07-15T11:01:35.818479Z","steps":["trace[1943040970] 'process raft request' (duration: 4.576835151s)"],"step_count":1} Jul 15 11:01:35.837958 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.819842Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:31.239942Z","time spent":"4.578913213s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":624,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:01:35.977972 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.975174Z","caller":"traceutil/trace.go:171","msg":"trace[412110865] transaction","detail":"{read_only:false; response_revision:1624; number_of_response:1; }","duration":"188.219662ms","start":"2025-07-15T11:01:35.786928Z","end":"2025-07-15T11:01:35.975147Z","steps":["trace[412110865] 'process raft request' (duration: 120.038367ms)","trace[412110865] 'compare' (duration: 67.813374ms)"],"step_count":2} Jul 15 11:01:35.977972 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.975339Z","caller":"traceutil/trace.go:171","msg":"trace[1139065508] linearizableReadLoop","detail":"{readStateIndex:1983; appliedIndex:1980; }","duration":"3.848161675s","start":"2025-07-15T11:01:32.127167Z","end":"2025-07-15T11:01:35.975329Z","steps":["trace[1139065508] 'read index received' (duration: 3.689295512s)","trace[1139065508] 'applied index is now lower than readState.Index' (duration: 158.865542ms)"],"step_count":2} Jul 15 11:01:35.977972 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.975472Z","caller":"traceutil/trace.go:171","msg":"trace[1569809665] transaction","detail":"{read_only:false; response_revision:1625; number_of_response:1; }","duration":"188.136239ms","start":"2025-07-15T11:01:35.787327Z","end":"2025-07-15T11:01:35.975463Z","steps":["trace[1569809665] 'process raft request' (duration: 187.722671ms)"],"step_count":1} Jul 15 11:01:35.977972 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.975801Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.84861969s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"range_response_count:1 size:886"} Jul 15 11:01:35.978696 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.975865Z","caller":"traceutil/trace.go:171","msg":"trace[1976268751] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; response_count:1; response_revision:1625; }","duration":"3.848707359s","start":"2025-07-15T11:01:32.127115Z","end":"2025-07-15T11:01:35.975822Z","steps":["trace[1976268751] 'agreement among raft nodes before linearized reading' (duration: 3.848425846s)"],"step_count":1} Jul 15 11:01:35.978696 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.975913Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:32.127086Z","time spent":"3.848803514s","remote":"172.24.4.149:54994","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":1,"response size":910,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" "} Jul 15 11:01:35.978696 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.976036Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"194.667493ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/tigera-operator/operator-lock\" ","response":"range_response_count:1 size:505"} Jul 15 11:01:35.978696 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.976103Z","caller":"traceutil/trace.go:171","msg":"trace[1201729594] range","detail":"{range_begin:/registry/leases/tigera-operator/operator-lock; range_end:; response_count:1; response_revision:1625; }","duration":"194.788156ms","start":"2025-07-15T11:01:35.781298Z","end":"2025-07-15T11:01:35.976086Z","steps":["trace[1201729594] 'agreement among raft nodes before linearized reading' (duration: 194.507734ms)"],"step_count":1} Jul 15 11:01:35.978696 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.976264Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"189.015929ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-controller-manager\" ","response":"range_response_count:1 size:532"} Jul 15 11:01:35.978900 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.976283Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"191.397227ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/managementclusterconnections/\" range_end:\"/registry/operator.tigera.io/managementclusterconnections0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:35.978900 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.976290Z","caller":"traceutil/trace.go:171","msg":"trace[425032310] range","detail":"{range_begin:/registry/leases/kube-system/kube-controller-manager; range_end:; response_count:1; response_revision:1625; }","duration":"189.121432ms","start":"2025-07-15T11:01:35.787161Z","end":"2025-07-15T11:01:35.976282Z","steps":["trace[425032310] 'agreement among raft nodes before linearized reading' (duration: 188.987784ms)"],"step_count":1} Jul 15 11:01:35.978900 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.976329Z","caller":"traceutil/trace.go:171","msg":"trace[1323422759] range","detail":"{range_begin:/registry/operator.tigera.io/managementclusterconnections/; range_end:/registry/operator.tigera.io/managementclusterconnections0; response_count:0; response_revision:1625; }","duration":"191.520705ms","start":"2025-07-15T11:01:35.784782Z","end":"2025-07-15T11:01:35.976303Z","steps":["trace[1323422759] 'agreement among raft nodes before linearized reading' (duration: 191.432365ms)"],"step_count":1} Jul 15 11:01:35.978900 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.976376Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"128.574165ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/\" range_end:\"/registry/minions0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:01:35.979121 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.976621Z","caller":"traceutil/trace.go:171","msg":"trace[781678120] range","detail":"{range_begin:/registry/minions/; range_end:/registry/minions0; response_count:0; response_revision:1625; }","duration":"128.828024ms","start":"2025-07-15T11:01:35.847753Z","end":"2025-07-15T11:01:35.976581Z","steps":["trace[781678120] 'agreement among raft nodes before linearized reading' (duration: 128.575847ms)"],"step_count":1} Jul 15 11:01:35.979121 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.976932Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"129.554768ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/endpoints/calico-system/calico-typha\" ","response":"range_response_count:1 size:688"} Jul 15 11:01:35.979121 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.976867Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"192.730361ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:35.979121 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.976996Z","caller":"traceutil/trace.go:171","msg":"trace[2143878273] range","detail":"{range_begin:/registry/services/endpoints/calico-system/calico-typha; range_end:; response_count:1; response_revision:1625; }","duration":"129.693956ms","start":"2025-07-15T11:01:35.847293Z","end":"2025-07-15T11:01:35.976987Z","steps":["trace[2143878273] 'agreement among raft nodes before linearized reading' (duration: 129.496194ms)"],"step_count":1} Jul 15 11:01:35.979121 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.977200Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"188.965982ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 11:01:35.979356 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.977392Z","caller":"traceutil/trace.go:171","msg":"trace[1030792350] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1625; }","duration":"189.223259ms","start":"2025-07-15T11:01:35.788145Z","end":"2025-07-15T11:01:35.977369Z","steps":["trace[1030792350] 'agreement among raft nodes before linearized reading' (duration: 189.004376ms)"],"step_count":1} Jul 15 11:01:35.979356 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.977513Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"193.729521ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/tiers/\" range_end:\"/registry/crd.projectcalico.org/tiers0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:01:35.979356 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.977033Z","caller":"traceutil/trace.go:171","msg":"trace[389534840] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1625; }","duration":"192.939404ms","start":"2025-07-15T11:01:35.784082Z","end":"2025-07-15T11:01:35.977022Z","steps":["trace[389534840] 'agreement among raft nodes before linearized reading' (duration: 192.731203ms)"],"step_count":1} Jul 15 11:01:35.979356 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.977811Z","caller":"traceutil/trace.go:171","msg":"trace[703882807] range","detail":"{range_begin:/registry/crd.projectcalico.org/tiers/; range_end:/registry/crd.projectcalico.org/tiers0; response_count:0; response_revision:1625; }","duration":"193.996867ms","start":"2025-07-15T11:01:35.783723Z","end":"2025-07-15T11:01:35.977720Z","steps":["trace[703882807] 'agreement among raft nodes before linearized reading' (duration: 193.687469ms)"],"step_count":1} Jul 15 11:01:35.979614 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:35.978245Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"195.487256ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/stagednetworkpolicies/\" range_end:\"/registry/crd.projectcalico.org/stagednetworkpolicies0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:35.979614 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:35.978981Z","caller":"traceutil/trace.go:171","msg":"trace[2131560670] range","detail":"{range_begin:/registry/crd.projectcalico.org/stagednetworkpolicies/; range_end:/registry/crd.projectcalico.org/stagednetworkpolicies0; response_count:0; response_revision:1625; }","duration":"196.16807ms","start":"2025-07-15T11:01:35.782702Z","end":"2025-07-15T11:01:35.978870Z","steps":["trace[2131560670] 'agreement among raft nodes before linearized reading' (duration: 195.517794ms)"],"step_count":1} Jul 15 11:01:36.157470 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:36.155175Z","caller":"traceutil/trace.go:171","msg":"trace[1844600943] linearizableReadLoop","detail":"{readStateIndex:1984; appliedIndex:1983; }","duration":"115.155297ms","start":"2025-07-15T11:01:36.039995Z","end":"2025-07-15T11:01:36.155150Z","steps":["trace[1844600943] 'read index received' (duration: 71.025901ms)","trace[1844600943] 'applied index is now lower than readState.Index' (duration: 44.128755ms)"],"step_count":2} Jul 15 11:01:36.157470 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:36.155502Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"115.470184ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/kube-system/kube-controller-manager-ci-4230-2-100-6ef5831daf.novalocal\" ","response":"range_response_count:1 size:6792"} Jul 15 11:01:36.157470 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:36.155552Z","caller":"traceutil/trace.go:171","msg":"trace[1506740489] range","detail":"{range_begin:/registry/pods/kube-system/kube-controller-manager-ci-4230-2-100-6ef5831daf.novalocal; range_end:; response_count:1; response_revision:1625; }","duration":"115.564847ms","start":"2025-07-15T11:01:36.039958Z","end":"2025-07-15T11:01:36.155523Z","steps":["trace[1506740489] 'agreement among raft nodes before linearized reading' (duration: 115.313202ms)"],"step_count":1} Jul 15 11:01:36.205164 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:36.203864Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"134.106688ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:36.205164 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:36.203992Z","caller":"traceutil/trace.go:171","msg":"trace[26364655] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1626; }","duration":"134.278027ms","start":"2025-07-15T11:01:36.069685Z","end":"2025-07-15T11:01:36.203963Z","steps":["trace[26364655] 'agreement among raft nodes before linearized reading' (duration: 133.799013ms)"],"step_count":1} Jul 15 11:01:36.205164 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:36.204027Z","caller":"traceutil/trace.go:171","msg":"trace[2139035036] transaction","detail":"{read_only:false; response_revision:1626; number_of_response:1; }","duration":"131.149937ms","start":"2025-07-15T11:01:36.072844Z","end":"2025-07-15T11:01:36.203994Z","steps":["trace[2139035036] 'process raft request' (duration: 130.237785ms)"],"step_count":1} Jul 15 11:01:36.917266 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:36.914870Z","caller":"traceutil/trace.go:171","msg":"trace[1531927847] linearizableReadLoop","detail":"{readStateIndex:1993; appliedIndex:1992; }","duration":"201.534889ms","start":"2025-07-15T11:01:36.713304Z","end":"2025-07-15T11:01:36.914839Z","steps":["trace[1531927847] 'read index received' (duration: 200.979085ms)","trace[1531927847] 'applied index is now lower than readState.Index' (duration: 554.571µs)"],"step_count":2} Jul 15 11:01:36.917266 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:36.915053Z","caller":"traceutil/trace.go:171","msg":"trace[488040369] transaction","detail":"{read_only:false; response_revision:1632; number_of_response:1; }","duration":"202.800483ms","start":"2025-07-15T11:01:36.712236Z","end":"2025-07-15T11:01:36.915037Z","steps":["trace[488040369] 'process raft request' (duration: 202.25545ms)"],"step_count":1} Jul 15 11:01:36.917266 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:36.915377Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"202.046547ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/calico-system/whisker-backend-key-pair\" ","response":"range_response_count:1 size:4802"} Jul 15 11:01:36.917266 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:36.915478Z","caller":"traceutil/trace.go:171","msg":"trace[1550562639] range","detail":"{range_begin:/registry/secrets/calico-system/whisker-backend-key-pair; range_end:; response_count:1; response_revision:1632; }","duration":"202.192278ms","start":"2025-07-15T11:01:36.713270Z","end":"2025-07-15T11:01:36.915462Z","steps":["trace[1550562639] 'agreement among raft nodes before linearized reading' (duration: 201.916005ms)"],"step_count":1} Jul 15 11:01:36.917266 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:36.915598Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"200.592698ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/whisker:csr-creator\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:36.918944 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:36.915723Z","caller":"traceutil/trace.go:171","msg":"trace[12176658] range","detail":"{range_begin:/registry/clusterrolebindings/whisker:csr-creator; range_end:; response_count:0; response_revision:1632; }","duration":"200.778408ms","start":"2025-07-15T11:01:36.714913Z","end":"2025-07-15T11:01:36.915692Z","steps":["trace[12176658] 'agreement among raft nodes before linearized reading' (duration: 200.583211ms)"],"step_count":1} Jul 15 11:01:36.918944 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:36.915896Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"196.923413ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/ippools/default-ipv4-ippool\" ","response":"range_response_count:1 size:1224"} Jul 15 11:01:36.918944 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:36.915970Z","caller":"traceutil/trace.go:171","msg":"trace[941888953] range","detail":"{range_begin:/registry/crd.projectcalico.org/ippools/default-ipv4-ippool; range_end:; response_count:1; response_revision:1632; }","duration":"197.019418ms","start":"2025-07-15T11:01:36.718917Z","end":"2025-07-15T11:01:36.915937Z","steps":["trace[941888953] 'agreement among raft nodes before linearized reading' (duration: 196.816005ms)"],"step_count":1} Jul 15 11:01:36.918944 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:36.916133Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"199.410306ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/calico-tier-getter\" ","response":"range_response_count:1 size:647"} Jul 15 11:01:36.919466 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:36.916189Z","caller":"traceutil/trace.go:171","msg":"trace[1520478087] range","detail":"{range_begin:/registry/clusterrolebindings/calico-tier-getter; range_end:; response_count:1; response_revision:1632; }","duration":"199.492534ms","start":"2025-07-15T11:01:36.716679Z","end":"2025-07-15T11:01:36.916172Z","steps":["trace[1520478087] 'agreement among raft nodes before linearized reading' (duration: 199.243373ms)"],"step_count":1} Jul 15 11:01:36.919466 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:36.916335Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"131.732592ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/deployments\" limit:1 ","response":"range_response_count:0 size:5"} Jul 15 11:01:36.919466 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:36.916441Z","caller":"traceutil/trace.go:171","msg":"trace[2069153971] range","detail":"{range_begin:/registry/deployments; range_end:; response_count:0; response_revision:1632; }","duration":"131.879163ms","start":"2025-07-15T11:01:36.784497Z","end":"2025-07-15T11:01:36.916376Z","steps":["trace[2069153971] 'agreement among raft nodes before linearized reading' (duration: 131.756146ms)"],"step_count":1} Jul 15 11:01:37.145571 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:37.143313Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"152.458847ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 11:01:37.145571 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:37.143541Z","caller":"traceutil/trace.go:171","msg":"trace[2033703287] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1632; }","duration":"152.688058ms","start":"2025-07-15T11:01:36.990774Z","end":"2025-07-15T11:01:37.143462Z","steps":["trace[2033703287] 'range keys from in-memory index tree' (duration: 152.29014ms)"],"step_count":1} Jul 15 11:01:37.145571 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:37.143915Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"216.561985ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/whisker:csr-creator\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:37.145571 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:37.143978Z","caller":"traceutil/trace.go:171","msg":"trace[1191399312] range","detail":"{range_begin:/registry/clusterrolebindings/whisker:csr-creator; range_end:; response_count:0; response_revision:1632; }","duration":"216.669473ms","start":"2025-07-15T11:01:36.927292Z","end":"2025-07-15T11:01:37.143962Z","steps":["trace[1191399312] 'range keys from in-memory index tree' (duration: 216.359213ms)"],"step_count":1} Jul 15 11:01:37.145571 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:37.144352Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"171.536009ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/ippools/default-ipv4-ippool\" ","response":"range_response_count:1 size:1224"} Jul 15 11:01:37.146560 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:37.144561Z","caller":"traceutil/trace.go:171","msg":"trace[782416560] range","detail":"{range_begin:/registry/crd.projectcalico.org/ippools/default-ipv4-ippool; range_end:; response_count:1; response_revision:1632; }","duration":"171.736755ms","start":"2025-07-15T11:01:36.972773Z","end":"2025-07-15T11:01:37.144510Z","steps":["trace[782416560] 'range keys from in-memory index tree' (duration: 171.255576ms)"],"step_count":1} Jul 15 11:01:37.146560 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:37.144473Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"173.815741ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/calico-tiered-policy-passthrough\" ","response":"range_response_count:1 size:638"} Jul 15 11:01:37.146560 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:37.144871Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"171.084506ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/tigera-operator/tigera-ca-private\" ","response":"range_response_count:1 size:3369"} Jul 15 11:01:37.146560 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:37.144932Z","caller":"traceutil/trace.go:171","msg":"trace[483219805] range","detail":"{range_begin:/registry/secrets/tigera-operator/tigera-ca-private; range_end:; response_count:1; response_revision:1632; }","duration":"171.173267ms","start":"2025-07-15T11:01:36.973744Z","end":"2025-07-15T11:01:37.144917Z","steps":["trace[483219805] 'range keys from in-memory index tree' (duration: 170.913445ms)"],"step_count":1} Jul 15 11:01:37.146560 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:37.144905Z","caller":"traceutil/trace.go:171","msg":"trace[1663272484] range","detail":"{range_begin:/registry/clusterroles/calico-tiered-policy-passthrough; range_end:; response_count:1; response_revision:1632; }","duration":"174.096061ms","start":"2025-07-15T11:01:36.970593Z","end":"2025-07-15T11:01:37.144689Z","steps":["trace[1663272484] 'range keys from in-memory index tree' (duration: 173.082966ms)"],"step_count":1} Jul 15 11:01:37.147156 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:37.145322Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"196.025143ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/tigera-csr-creator\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:37.147156 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:37.145364Z","caller":"traceutil/trace.go:171","msg":"trace[616320515] range","detail":"{range_begin:/registry/clusterroles/tigera-csr-creator; range_end:; response_count:0; response_revision:1632; }","duration":"196.118834ms","start":"2025-07-15T11:01:36.949237Z","end":"2025-07-15T11:01:37.145356Z","steps":["trace[616320515] 'range keys from in-memory index tree' (duration: 195.939889ms)"],"step_count":1} Jul 15 11:01:37.147156 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:37.145440Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"184.671285ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 11:01:37.147156 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:37.145478Z","caller":"traceutil/trace.go:171","msg":"trace[1553354365] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1632; }","duration":"184.721332ms","start":"2025-07-15T11:01:36.960734Z","end":"2025-07-15T11:01:37.145455Z","steps":["trace[1553354365] 'range keys from in-memory index tree' (duration: 184.588345ms)"],"step_count":1} Jul 15 11:01:37.147156 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:37.145287Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"167.256012ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/calico-system/goldmane\" ","response":"range_response_count:1 size:429"} Jul 15 11:01:37.147749 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:37.146197Z","caller":"traceutil/trace.go:171","msg":"trace[102622055] range","detail":"{range_begin:/registry/serviceaccounts/calico-system/goldmane; range_end:; response_count:1; response_revision:1632; }","duration":"168.19219ms","start":"2025-07-15T11:01:36.977990Z","end":"2025-07-15T11:01:37.146182Z","steps":["trace[102622055] 'range keys from in-memory index tree' (duration: 167.042591ms)"],"step_count":1} Jul 15 11:01:38.384088 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.383410Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885076382,"retry-timeout":"500ms"} Jul 15 11:01:38.433444 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:38.431964Z","caller":"traceutil/trace.go:171","msg":"trace[1384040826] transaction","detail":"{read_only:false; response_revision:1633; number_of_response:1; }","duration":"558.175473ms","start":"2025-07-15T11:01:37.873752Z","end":"2025-07-15T11:01:38.431928Z","steps":["trace[1384040826] 'process raft request' (duration: 557.888229ms)"],"step_count":1} Jul 15 11:01:38.433444 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.432263Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:37.873731Z","time spent":"558.36046ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":699,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:01:38.547606 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:38.543656Z","caller":"traceutil/trace.go:171","msg":"trace[398808741] transaction","detail":"{read_only:false; response_revision:1634; number_of_response:1; }","duration":"489.419337ms","start":"2025-07-15T11:01:38.054181Z","end":"2025-07-15T11:01:38.543601Z","steps":["trace[398808741] 'process raft request' (duration: 481.735376ms)"],"step_count":1} Jul 15 11:01:38.547606 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.543928Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:38.054139Z","time spent":"489.64352ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:01:38.547606 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:38.544109Z","caller":"traceutil/trace.go:171","msg":"trace[1511387243] linearizableReadLoop","detail":"{readStateIndex:1996; appliedIndex:1994; }","duration":"661.766313ms","start":"2025-07-15T11:01:37.882320Z","end":"2025-07-15T11:01:38.544087Z","steps":["trace[1511387243] 'read index received' (duration: 549.966159ms)","trace[1511387243] 'applied index is now lower than readState.Index' (duration: 111.798431ms)"],"step_count":2} Jul 15 11:01:38.547606 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:38.544317Z","caller":"traceutil/trace.go:171","msg":"trace[669787647] transaction","detail":"{read_only:false; response_revision:1635; number_of_response:1; }","duration":"203.43901ms","start":"2025-07-15T11:01:38.340827Z","end":"2025-07-15T11:01:38.544266Z","steps":["trace[669787647] 'process raft request' (duration: 202.667991ms)"],"step_count":1} Jul 15 11:01:38.550519 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.544897Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"644.913296ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/networkpolicies/calico-system/goldmane\" ","response":"range_response_count:1 size:597"} Jul 15 11:01:38.550519 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:38.544957Z","caller":"traceutil/trace.go:171","msg":"trace[1141334226] range","detail":"{range_begin:/registry/networkpolicies/calico-system/goldmane; range_end:; response_count:1; response_revision:1635; }","duration":"645.020772ms","start":"2025-07-15T11:01:37.899920Z","end":"2025-07-15T11:01:38.544940Z","steps":["trace[1141334226] 'agreement among raft nodes before linearized reading' (duration: 644.822179ms)"],"step_count":1} Jul 15 11:01:38.550519 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.545006Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:37.899872Z","time spent":"645.116779ms","remote":"172.24.4.149:33316","response type":"/etcdserverpb.KV/Range","request count":0,"request size":50,"response count":1,"response size":621,"request content":"key:\"/registry/networkpolicies/calico-system/goldmane\" "} Jul 15 11:01:38.550519 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.545262Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"662.885404ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/stagedkubernetesnetworkpolicies.crd.projectcalico.org\" ","response":"range_response_count:1 size:20393"} Jul 15 11:01:38.552328 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:38.545478Z","caller":"traceutil/trace.go:171","msg":"trace[406508384] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/stagedkubernetesnetworkpolicies.crd.projectcalico.org; range_end:; response_count:1; response_revision:1635; }","duration":"663.161338ms","start":"2025-07-15T11:01:37.882282Z","end":"2025-07-15T11:01:38.545443Z","steps":["trace[406508384] 'agreement among raft nodes before linearized reading' (duration: 661.951432ms)"],"step_count":1} Jul 15 11:01:38.552328 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.545634Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:37.882262Z","time spent":"663.307289ms","remote":"172.24.4.149:33078","response type":"/etcdserverpb.KV/Range","request count":0,"request size":112,"response count":1,"response size":20417,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/stagedkubernetesnetworkpolicies.crd.projectcalico.org\" "} Jul 15 11:01:38.552328 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.545675Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"377.74236ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"range_response_count:1 size:886"} Jul 15 11:01:38.552328 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:38.545860Z","caller":"traceutil/trace.go:171","msg":"trace[1674030853] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; response_count:1; response_revision:1635; }","duration":"377.969638ms","start":"2025-07-15T11:01:38.167869Z","end":"2025-07-15T11:01:38.545838Z","steps":["trace[1674030853] 'agreement among raft nodes before linearized reading' (duration: 377.622068ms)"],"step_count":1} Jul 15 11:01:38.553026 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.545988Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:38.167843Z","time spent":"378.119268ms","remote":"172.24.4.149:54994","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":1,"response size":910,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" "} Jul 15 11:01:38.553026 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.546237Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"588.097386ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:38.553026 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:38.546353Z","caller":"traceutil/trace.go:171","msg":"trace[380476126] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1635; }","duration":"588.289808ms","start":"2025-07-15T11:01:37.958045Z","end":"2025-07-15T11:01:38.546335Z","steps":["trace[380476126] 'agreement among raft nodes before linearized reading' (duration: 588.154187ms)"],"step_count":1} Jul 15 11:01:38.553026 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.546602Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:37.958016Z","time spent":"588.482349ms","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 11:01:38.553026 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.547457Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"637.25759ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/poddisruptionbudgets/calico-apiserver/calico-apiserver\" ","response":"range_response_count:1 size:1016"} Jul 15 11:01:38.553863 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:38.548056Z","caller":"traceutil/trace.go:171","msg":"trace[149395395] range","detail":"{range_begin:/registry/poddisruptionbudgets/calico-apiserver/calico-apiserver; range_end:; response_count:1; response_revision:1635; }","duration":"637.818023ms","start":"2025-07-15T11:01:37.910151Z","end":"2025-07-15T11:01:38.547969Z","steps":["trace[149395395] 'agreement among raft nodes before linearized reading' (duration: 637.137007ms)"],"step_count":1} Jul 15 11:01:38.553863 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.548533Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:37.910135Z","time spent":"638.36581ms","remote":"172.24.4.149:33346","response type":"/etcdserverpb.KV/Range","request count":0,"request size":66,"response count":1,"response size":1040,"request content":"key:\"/registry/poddisruptionbudgets/calico-apiserver/calico-apiserver\" "} Jul 15 11:01:38.553863 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.549537Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"649.300018ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/configmaps/calico-system/whisker-ca-bundle\" ","response":"range_response_count:1 size:1956"} Jul 15 11:01:38.553863 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:38.549913Z","caller":"traceutil/trace.go:171","msg":"trace[1176300546] range","detail":"{range_begin:/registry/configmaps/calico-system/whisker-ca-bundle; range_end:; response_count:1; response_revision:1635; }","duration":"649.634644ms","start":"2025-07-15T11:01:37.900173Z","end":"2025-07-15T11:01:38.549807Z","steps":["trace[1176300546] 'agreement among raft nodes before linearized reading' (duration: 649.042751ms)"],"step_count":1} Jul 15 11:01:38.554295 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.550289Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:37.900159Z","time spent":"650.006913ms","remote":"172.24.4.149:33128","response type":"/etcdserverpb.KV/Range","request count":0,"request size":54,"response count":1,"response size":1980,"request content":"key:\"/registry/configmaps/calico-system/whisker-ca-bundle\" "} Jul 15 11:01:38.855171 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:38.853164Z","caller":"traceutil/trace.go:171","msg":"trace[1228402026] linearizableReadLoop","detail":"{readStateIndex:1999; appliedIndex:1998; }","duration":"105.153066ms","start":"2025-07-15T11:01:38.747976Z","end":"2025-07-15T11:01:38.853129Z","steps":["trace[1228402026] 'read index received' (duration: 104.907582ms)","trace[1228402026] 'applied index is now lower than readState.Index' (duration: 244.041µs)"],"step_count":2} Jul 15 11:01:38.855171 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:38.853432Z","caller":"traceutil/trace.go:171","msg":"trace[1180798310] transaction","detail":"{read_only:false; number_of_response:0; response_revision:1635; }","duration":"122.895574ms","start":"2025-07-15T11:01:38.730275Z","end":"2025-07-15T11:01:38.853171Z","steps":["trace[1180798310] 'process raft request' (duration: 122.69665ms)"],"step_count":1} Jul 15 11:01:38.855171 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.853577Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"105.562537ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/deployments/calico-system/whisker\" ","response":"range_response_count:1 size:5108"} Jul 15 11:01:38.855171 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:38.853663Z","caller":"traceutil/trace.go:171","msg":"trace[1856672604] range","detail":"{range_begin:/registry/deployments/calico-system/whisker; range_end:; response_count:1; response_revision:1635; }","duration":"105.677038ms","start":"2025-07-15T11:01:38.747941Z","end":"2025-07-15T11:01:38.853618Z","steps":["trace[1856672604] 'agreement among raft nodes before linearized reading' (duration: 105.386056ms)"],"step_count":1} Jul 15 11:01:38.855171 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:38.853938Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"103.002744ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rolebindings/kube-system/tigera-auth-reader\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:38.856288 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:38.854083Z","caller":"traceutil/trace.go:171","msg":"trace[655195448] range","detail":"{range_begin:/registry/rolebindings/kube-system/tigera-auth-reader; range_end:; response_count:0; response_revision:1635; }","duration":"103.189084ms","start":"2025-07-15T11:01:38.750863Z","end":"2025-07-15T11:01:38.854052Z","steps":["trace[655195448] 'agreement among raft nodes before linearized reading' (duration: 102.896409ms)"],"step_count":1} Jul 15 11:01:39.215974 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:39.214138Z","caller":"traceutil/trace.go:171","msg":"trace[457456363] linearizableReadLoop","detail":"{readStateIndex:2000; appliedIndex:1999; }","duration":"116.527337ms","start":"2025-07-15T11:01:39.097575Z","end":"2025-07-15T11:01:39.214102Z","steps":["trace[457456363] 'read index received' (duration: 115.871059ms)","trace[457456363] 'applied index is now lower than readState.Index' (duration: 580.602µs)"],"step_count":2} Jul 15 11:01:39.215974 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:39.214347Z","caller":"traceutil/trace.go:171","msg":"trace[589959603] transaction","detail":"{read_only:false; response_revision:1636; number_of_response:1; }","duration":"117.959342ms","start":"2025-07-15T11:01:39.096370Z","end":"2025-07-15T11:01:39.214329Z","steps":["trace[589959603] 'process raft request' (duration: 117.132284ms)"],"step_count":1} Jul 15 11:01:39.215974 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:39.214458Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"115.910896ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/whisker:csr-creator\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:39.215974 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:39.214563Z","caller":"traceutil/trace.go:171","msg":"trace[1882549489] range","detail":"{range_begin:/registry/clusterrolebindings/whisker:csr-creator; range_end:; response_count:0; response_revision:1636; }","duration":"116.048552ms","start":"2025-07-15T11:01:39.098491Z","end":"2025-07-15T11:01:39.214540Z","steps":["trace[1882549489] 'agreement among raft nodes before linearized reading' (duration: 115.870759ms)"],"step_count":1} Jul 15 11:01:39.215974 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:39.214829Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"113.713752ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/networkpolicies/calico-system/goldmane\" ","response":"range_response_count:1 size:597"} Jul 15 11:01:39.217655 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:39.214908Z","caller":"traceutil/trace.go:171","msg":"trace[218714210] range","detail":"{range_begin:/registry/networkpolicies/calico-system/goldmane; range_end:; response_count:1; response_revision:1636; }","duration":"113.818365ms","start":"2025-07-15T11:01:39.101075Z","end":"2025-07-15T11:01:39.214893Z","steps":["trace[218714210] 'agreement among raft nodes before linearized reading' (duration: 113.610814ms)"],"step_count":1} Jul 15 11:01:39.217655 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:39.214958Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"111.955106ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/imagesets.operator.tigera.io\" ","response":"range_response_count:1 size:4490"} Jul 15 11:01:39.217655 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:39.215026Z","caller":"traceutil/trace.go:171","msg":"trace[783266477] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/imagesets.operator.tigera.io; range_end:; response_count:1; response_revision:1636; }","duration":"112.066561ms","start":"2025-07-15T11:01:39.102941Z","end":"2025-07-15T11:01:39.215007Z","steps":["trace[783266477] 'agreement among raft nodes before linearized reading' (duration: 111.741713ms)"],"step_count":1} Jul 15 11:01:39.217655 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:39.215096Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"117.517719ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/tigera-crds\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:39.218252 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:39.215140Z","caller":"traceutil/trace.go:171","msg":"trace[1865035918] range","detail":"{range_begin:/registry/clusterroles/tigera-crds; range_end:; response_count:0; response_revision:1636; }","duration":"117.608094ms","start":"2025-07-15T11:01:39.097519Z","end":"2025-07-15T11:01:39.215127Z","steps":["trace[1865035918] 'agreement among raft nodes before linearized reading' (duration: 117.545594ms)"],"step_count":1} Jul 15 11:01:39.380785 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:39.379792Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"152.481952ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/tigera-apiserver-access-tigera-crds\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:39.380785 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:39.379910Z","caller":"traceutil/trace.go:171","msg":"trace[1289464340] range","detail":"{range_begin:/registry/clusterrolebindings/tigera-apiserver-access-tigera-crds; range_end:; response_count:0; response_revision:1636; }","duration":"152.660235ms","start":"2025-07-15T11:01:39.227223Z","end":"2025-07-15T11:01:39.379883Z","steps":["trace[1289464340] 'range keys from in-memory index tree' (duration: 152.358072ms)"],"step_count":1} Jul 15 11:01:39.387081 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:39.386179Z","caller":"traceutil/trace.go:171","msg":"trace[1811902143] transaction","detail":"{read_only:false; response_revision:1637; number_of_response:1; }","duration":"155.108945ms","start":"2025-07-15T11:01:39.231040Z","end":"2025-07-15T11:01:39.386149Z","steps":["trace[1811902143] 'process raft request' (duration: 154.826951ms)"],"step_count":1} Jul 15 11:01:39.403454 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:39.402375Z","caller":"traceutil/trace.go:171","msg":"trace[233235201] linearizableReadLoop","detail":"{readStateIndex:2002; appliedIndex:2001; }","duration":"164.536262ms","start":"2025-07-15T11:01:39.237804Z","end":"2025-07-15T11:01:39.402341Z","steps":["trace[233235201] 'read index received' (duration: 148.481825ms)","trace[233235201] 'applied index is now lower than readState.Index' (duration: 16.053265ms)"],"step_count":2} Jul 15 11:01:39.403454 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:39.402587Z","caller":"traceutil/trace.go:171","msg":"trace[1165283829] transaction","detail":"{read_only:false; response_revision:1638; number_of_response:1; }","duration":"167.851486ms","start":"2025-07-15T11:01:39.234720Z","end":"2025-07-15T11:01:39.402571Z","steps":["trace[1165283829] 'process raft request' (duration: 167.281364ms)"],"step_count":1} Jul 15 11:01:39.404302 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:39.403275Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"161.133711ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:39.404302 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:39.403349Z","caller":"traceutil/trace.go:171","msg":"trace[1783548994] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1638; }","duration":"161.248622ms","start":"2025-07-15T11:01:39.242084Z","end":"2025-07-15T11:01:39.403333Z","steps":["trace[1783548994] 'agreement among raft nodes before linearized reading' (duration: 160.955156ms)"],"step_count":1} Jul 15 11:01:39.406971 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:39.404359Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"166.545143ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/calico-system/whisker\" ","response":"range_response_count:1 size:426"} Jul 15 11:01:39.406971 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:39.404576Z","caller":"traceutil/trace.go:171","msg":"trace[1548086648] range","detail":"{range_begin:/registry/serviceaccounts/calico-system/whisker; range_end:; response_count:1; response_revision:1638; }","duration":"166.795265ms","start":"2025-07-15T11:01:39.237765Z","end":"2025-07-15T11:01:39.404560Z","steps":["trace[1548086648] 'agreement among raft nodes before linearized reading' (duration: 164.919683ms)"],"step_count":1} Jul 15 11:01:39.406971 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:39.405280Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"144.681765ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 11:01:39.406971 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:39.405422Z","caller":"traceutil/trace.go:171","msg":"trace[289219018] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1638; }","duration":"144.856442ms","start":"2025-07-15T11:01:39.260471Z","end":"2025-07-15T11:01:39.405328Z","steps":["trace[289219018] 'agreement among raft nodes before linearized reading' (duration: 144.216536ms)"],"step_count":1} Jul 15 11:01:57.840984 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:53.636243Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:48.636943Z","time spent":"4.999262202s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:01:57.840984 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:54.939293Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.998342876s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"","error":"context canceled"} Jul 15 11:01:57.840984 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:54.939633Z","caller":"traceutil/trace.go:171","msg":"trace[2128660078] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; }","duration":"5.99886317s","start":"2025-07-15T11:01:48.940656Z","end":"2025-07-15T11:01:54.939519Z","steps":["trace[2128660078] 'agreement among raft nodes before linearized reading' (duration: 5.998358786s)"],"step_count":1} Jul 15 11:01:57.840984 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:54.939846Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:48.940631Z","time spent":"5.999189209s","remote":"172.24.4.149:54994","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":0,"response size":0,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" "} Jul 15 11:01:57.840984 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:54.945113Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000261069s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:01:57.847637 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:54.945178Z","caller":"traceutil/trace.go:171","msg":"trace[1171326186] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.000383025s","start":"2025-07-15T11:01:52.944779Z","end":"2025-07-15T11:01:54.945162Z","steps":["trace[1171326186] 'agreement among raft nodes before linearized reading' (duration: 2.000301828s)"],"step_count":1} Jul 15 11:01:57.847637 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:54.945224Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:52.944755Z","time spent":"2.000458321s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:01:57.847637 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:54.967940Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"6.268616018s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"} Jul 15 11:01:57.847637 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:54.968044Z","caller":"traceutil/trace.go:171","msg":"trace[410621200] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"6.268748875s","start":"2025-07-15T11:01:48.699270Z","end":"2025-07-15T11:01:54.968019Z","steps":["trace[410621200] 'agreement among raft nodes before linearized reading' (duration: 6.268633882s)"],"step_count":1} Jul 15 11:01:57.847637 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:54.968101Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:48.699259Z","time spent":"6.268824631s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:01:57.848108 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:55.738271Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:48.737277Z","time spent":"7.000979302s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:01:57.848108 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:57.042063Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.999895136s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:01:57.848108 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:57.042184Z","caller":"traceutil/trace.go:171","msg":"trace[225762350] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.000084031s","start":"2025-07-15T11:01:55.042078Z","end":"2025-07-15T11:01:57.042162Z","steps":["trace[225762350] 'agreement among raft nodes before linearized reading' (duration: 1.999927348s)"],"step_count":1} Jul 15 11:01:57.848108 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:57.042236Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:55.042057Z","time spent":"2.000167082s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:01:57.849610 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:57.848766Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"9.111389183s","expected-duration":"1s"} Jul 15 11:01:58.088898 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.084618Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"9.388597776s","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 11:01:58.088898 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.084973Z","caller":"traceutil/trace.go:171","msg":"trace[189503376] linearizableReadLoop","detail":"{readStateIndex:2066; appliedIndex:2065; }","duration":"9.39153999s","start":"2025-07-15T11:01:48.693374Z","end":"2025-07-15T11:01:58.084914Z","steps":["trace[189503376] 'read index received' (duration: 2.238374ms)","trace[189503376] 'applied index is now lower than readState.Index' (duration: 9.389298229s)"],"step_count":2} Jul 15 11:01:58.088898 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.085376Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"9.391958717s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/validatingadmissionpolicybindings/\" range_end:\"/registry/validatingadmissionpolicybindings0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:58.088898 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.085476Z","caller":"traceutil/trace.go:171","msg":"trace[744326870] range","detail":"{range_begin:/registry/validatingadmissionpolicybindings/; range_end:/registry/validatingadmissionpolicybindings0; response_count:0; response_revision:1698; }","duration":"9.39211477s","start":"2025-07-15T11:01:48.693340Z","end":"2025-07-15T11:01:58.085455Z","steps":["trace[744326870] 'agreement among raft nodes before linearized reading' (duration: 9.391769392s)"],"step_count":1} Jul 15 11:01:58.093682 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.085531Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:48.693325Z","time spent":"9.392184715s","remote":"172.24.4.149:33530","response type":"/etcdserverpb.KV/Range","request count":0,"request size":94,"response count":0,"response size":29,"request content":"key:\"/registry/validatingadmissionpolicybindings/\" range_end:\"/registry/validatingadmissionpolicybindings0\" count_only:true "} Jul 15 11:01:58.093682 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.085843Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"6.967243478s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/\" range_end:\"/registry/pods0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:01:58.093682 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.085886Z","caller":"traceutil/trace.go:171","msg":"trace[1685377715] range","detail":"{range_begin:/registry/pods/; range_end:/registry/pods0; response_count:0; response_revision:1700; }","duration":"6.967321739s","start":"2025-07-15T11:01:51.118551Z","end":"2025-07-15T11:01:58.085873Z","steps":["trace[1685377715] 'agreement among raft nodes before linearized reading' (duration: 6.967123436s)"],"step_count":1} Jul 15 11:01:58.093682 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.085927Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:51.118530Z","time spent":"6.967382577s","remote":"172.24.4.149:33206","response type":"/etcdserverpb.KV/Range","request count":0,"request size":36,"response count":19,"response size":31,"request content":"key:\"/registry/pods/\" range_end:\"/registry/pods0\" count_only:true "} Jul 15 11:01:58.093682 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.086038Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.04319072s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:01:58.094649 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.086074Z","caller":"traceutil/trace.go:171","msg":"trace[1207080570] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1700; }","duration":"1.043254324s","start":"2025-07-15T11:01:57.042807Z","end":"2025-07-15T11:01:58.086061Z","steps":["trace[1207080570] 'agreement among raft nodes before linearized reading' (duration: 1.043204638s)"],"step_count":1} Jul 15 11:01:58.094649 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.086065Z","caller":"traceutil/trace.go:171","msg":"trace[230019576] transaction","detail":"{read_only:false; response_revision:1697; number_of_response:1; }","duration":"3.116456558s","start":"2025-07-15T11:01:54.969583Z","end":"2025-07-15T11:01:58.086040Z","steps":["trace[230019576] 'process raft request' (duration: 3.115318158s)"],"step_count":1} Jul 15 11:01:58.094649 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.086118Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:57.042790Z","time spent":"1.043315381s","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 11:01:58.094649 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.086215Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.365337686s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/roles/\" range_end:\"/registry/roles0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:01:58.095270 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.086209Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:54.969539Z","time spent":"3.116587471s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":516,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:01:58.095270 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.086252Z","caller":"traceutil/trace.go:171","msg":"trace[92291483] range","detail":"{range_begin:/registry/roles/; range_end:/registry/roles0; response_count:0; response_revision:1700; }","duration":"1.36542801s","start":"2025-07-15T11:01:56.720812Z","end":"2025-07-15T11:01:58.086240Z","steps":["trace[92291483] 'agreement among raft nodes before linearized reading' (duration: 1.365361701s)"],"step_count":1} Jul 15 11:01:58.095270 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.086294Z","caller":"traceutil/trace.go:171","msg":"trace[1136287602] transaction","detail":"{read_only:false; response_revision:1698; number_of_response:1; }","duration":"3.074634304s","start":"2025-07-15T11:01:55.011645Z","end":"2025-07-15T11:01:58.086279Z","steps":["trace[1136287602] 'process raft request' (duration: 3.073304575s)"],"step_count":1} Jul 15 11:01:58.095270 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.086327Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:56.720784Z","time spent":"1.365493906s","remote":"172.24.4.149:33348","response type":"/etcdserverpb.KV/Range","request count":0,"request size":38,"response count":12,"response size":31,"request content":"key:\"/registry/roles/\" range_end:\"/registry/roles0\" count_only:true "} Jul 15 11:01:58.098769 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.086366Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:55.011618Z","time spent":"3.074705792s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":490,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:01:58.098769 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.086669Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.266601024s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/hostendpoints/\" range_end:\"/registry/crd.projectcalico.org/hostendpoints0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:58.098769 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.086753Z","caller":"traceutil/trace.go:171","msg":"trace[1322902757] transaction","detail":"{read_only:false; response_revision:1699; number_of_response:1; }","duration":"1.483157109s","start":"2025-07-15T11:01:56.603578Z","end":"2025-07-15T11:01:58.086735Z","steps":["trace[1322902757] 'process raft request' (duration: 1.481561487s)"],"step_count":1} Jul 15 11:01:58.098769 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.086896Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:56.603539Z","time spent":"1.483299865s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":533,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:01:58.099365 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.086920Z","caller":"traceutil/trace.go:171","msg":"trace[307093170] range","detail":"{range_begin:/registry/crd.projectcalico.org/hostendpoints/; range_end:/registry/crd.projectcalico.org/hostendpoints0; response_count:0; response_revision:1700; }","duration":"2.266855846s","start":"2025-07-15T11:01:55.819956Z","end":"2025-07-15T11:01:58.086812Z","steps":["trace[307093170] 'agreement among raft nodes before linearized reading' (duration: 2.266623247s)"],"step_count":1} Jul 15 11:01:58.099365 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.087281Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:55.819926Z","time spent":"2.267308439s","remote":"172.24.4.149:54596","response type":"/etcdserverpb.KV/Range","request count":0,"request size":98,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/hostendpoints/\" range_end:\"/registry/crd.projectcalico.org/hostendpoints0\" count_only:true "} Jul 15 11:01:58.099365 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.087801Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.119927724s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/priorityclasses/\" range_end:\"/registry/priorityclasses0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:01:58.099365 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.089627Z","caller":"traceutil/trace.go:171","msg":"trace[926717327] range","detail":"{range_begin:/registry/priorityclasses/; range_end:/registry/priorityclasses0; response_count:0; response_revision:1700; }","duration":"3.121755606s","start":"2025-07-15T11:01:54.967803Z","end":"2025-07-15T11:01:58.089558Z","steps":["trace[926717327] 'agreement among raft nodes before linearized reading' (duration: 3.119902866s)"],"step_count":1} Jul 15 11:01:58.100843 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.089721Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:54.967771Z","time spent":"3.121922817s","remote":"172.24.4.149:33388","response type":"/etcdserverpb.KV/Range","request count":0,"request size":58,"response count":2,"response size":31,"request content":"key:\"/registry/priorityclasses/\" range_end:\"/registry/priorityclasses0\" count_only:true "} Jul 15 11:01:58.100843 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.088187Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.123456s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 11:01:58.100843 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.089967Z","caller":"traceutil/trace.go:171","msg":"trace[1596644682] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1700; }","duration":"3.12525713s","start":"2025-07-15T11:01:54.964682Z","end":"2025-07-15T11:01:58.089939Z","steps":["trace[1596644682] 'agreement among raft nodes before linearized reading' (duration: 3.123321881s)"],"step_count":1} Jul 15 11:01:58.100843 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.090187Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:54.964660Z","time spent":"3.125434472s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 11:01:58.100843 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.088638Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.112695092s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-scheduler\" ","response":"range_response_count:1 size:505"} Jul 15 11:01:58.101259 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.090480Z","caller":"traceutil/trace.go:171","msg":"trace[1389990833] range","detail":"{range_begin:/registry/leases/kube-system/kube-scheduler; range_end:; response_count:1; response_revision:1700; }","duration":"3.114583731s","start":"2025-07-15T11:01:54.975877Z","end":"2025-07-15T11:01:58.090461Z","steps":["trace[1389990833] 'agreement among raft nodes before linearized reading' (duration: 3.11147265s)"],"step_count":1} Jul 15 11:01:58.101259 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.090779Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:54.975859Z","time spent":"3.114689405s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":45,"response count":1,"response size":529,"request content":"key:\"/registry/leases/kube-system/kube-scheduler\" "} Jul 15 11:01:58.101259 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.088982Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.959284063s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/podtemplates/\" range_end:\"/registry/podtemplates0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:01:58.101259 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.091920Z","caller":"traceutil/trace.go:171","msg":"trace[941596309] range","detail":"{range_begin:/registry/podtemplates/; range_end:/registry/podtemplates0; response_count:0; response_revision:1700; }","duration":"2.962073753s","start":"2025-07-15T11:01:55.129399Z","end":"2025-07-15T11:01:58.091472Z","steps":["trace[941596309] 'agreement among raft nodes before linearized reading' (duration: 2.959291037s)"],"step_count":1} Jul 15 11:01:58.101688 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.092307Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:55.129358Z","time spent":"2.962708449s","remote":"172.24.4.149:33174","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":0,"response size":29,"request content":"key:\"/registry/podtemplates/\" range_end:\"/registry/podtemplates0\" count_only:true "} Jul 15 11:01:58.101688 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.089273Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.106506507s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/daemonsets/calico-system/calico-node\" ","response":"range_response_count:1 size:11440"} Jul 15 11:01:58.101688 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.092636Z","caller":"traceutil/trace.go:171","msg":"trace[2144516052] range","detail":"{range_begin:/registry/daemonsets/calico-system/calico-node; range_end:; response_count:1; response_revision:1700; }","duration":"3.109870466s","start":"2025-07-15T11:01:54.982747Z","end":"2025-07-15T11:01:58.092618Z","steps":["trace[2144516052] 'agreement among raft nodes before linearized reading' (duration: 3.106321619s)"],"step_count":1} Jul 15 11:01:58.101688 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.092691Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:54.982698Z","time spent":"3.109973184s","remote":"172.24.4.149:33488","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":1,"response size":11464,"request content":"key:\"/registry/daemonsets/calico-system/calico-node\" "} Jul 15 11:01:58.101688 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.086681Z","caller":"traceutil/trace.go:171","msg":"trace[1626554357] transaction","detail":"{read_only:false; number_of_response:1; response_revision:1698; }","duration":"2.340427576s","start":"2025-07-15T11:01:55.746232Z","end":"2025-07-15T11:01:58.086660Z","steps":["trace[1626554357] 'process raft request' (duration: 2.338817926s)"],"step_count":1} Jul 15 11:01:58.102093 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.092863Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:01:55.746197Z","time spent":"2.346595583s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":69,"response count":0,"response size":748,"request content":"compare: success:> failure: >"} Jul 15 11:01:58.238763 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.237946Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"139.792031ms","expected-duration":"100ms","prefix":"","request":"header: lease_revoke:","response":"size:29"} Jul 15 11:01:58.239761 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.238812Z","caller":"traceutil/trace.go:171","msg":"trace[337021294] linearizableReadLoop","detail":"{readStateIndex:2074; appliedIndex:2072; }","duration":"129.73444ms","start":"2025-07-15T11:01:58.109053Z","end":"2025-07-15T11:01:58.238787Z","steps":["trace[337021294] 'read index received' (duration: 20.505296ms)","trace[337021294] 'applied index is now lower than readState.Index' (duration: 109.227431ms)"],"step_count":2} Jul 15 11:01:58.239761 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.239170Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"130.104434ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/apiserver-jtty2bqg5eqwaymygkxixvrno4\" ","response":"range_response_count:1 size:713"} Jul 15 11:01:58.239761 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.239226Z","caller":"traceutil/trace.go:171","msg":"trace[669577139] range","detail":"{range_begin:/registry/leases/kube-system/apiserver-jtty2bqg5eqwaymygkxixvrno4; range_end:; response_count:1; response_revision:1700; }","duration":"130.222714ms","start":"2025-07-15T11:01:58.108987Z","end":"2025-07-15T11:01:58.239209Z","steps":["trace[669577139] 'agreement among raft nodes before linearized reading' (duration: 130.009291ms)"],"step_count":1} Jul 15 11:01:58.240670 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.239368Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"127.638841ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-scheduler\" ","response":"range_response_count:1 size:505"} Jul 15 11:01:58.240670 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.239637Z","caller":"traceutil/trace.go:171","msg":"trace[1796077422] range","detail":"{range_begin:/registry/leases/kube-system/kube-scheduler; range_end:; response_count:1; response_revision:1700; }","duration":"127.952127ms","start":"2025-07-15T11:01:58.111676Z","end":"2025-07-15T11:01:58.239628Z","steps":["trace[1796077422] 'agreement among raft nodes before linearized reading' (duration: 127.628732ms)"],"step_count":1} Jul 15 11:01:58.240670 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.239806Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"120.296927ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/daemonsets/calico-system/calico-node\" ","response":"range_response_count:1 size:11440"} Jul 15 11:01:58.240670 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.239829Z","caller":"traceutil/trace.go:171","msg":"trace[1071873308] range","detail":"{range_begin:/registry/daemonsets/calico-system/calico-node; range_end:; response_count:1; response_revision:1700; }","duration":"120.507533ms","start":"2025-07-15T11:01:58.119315Z","end":"2025-07-15T11:01:58.239823Z","steps":["trace[1071873308] 'agreement among raft nodes before linearized reading' (duration: 120.409204ms)"],"step_count":1} Jul 15 11:01:58.550702 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:01:58.550031Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"209.854477ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure:<>>","response":"size:16"} Jul 15 11:01:58.550702 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:01:58.550152Z","caller":"traceutil/trace.go:171","msg":"trace[1103781451] transaction","detail":"{read_only:false; response_revision:1703; number_of_response:1; }","duration":"241.855511ms","start":"2025-07-15T11:01:58.308279Z","end":"2025-07-15T11:01:58.550134Z","steps":["trace[1103781451] 'process raft request' (duration: 31.829382ms)","trace[1103781451] 'compare' (duration: 209.495313ms)"],"step_count":2} Jul 15 11:02:02.666485 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:02.664900Z","caller":"traceutil/trace.go:171","msg":"trace[301873990] transaction","detail":"{read_only:false; response_revision:1707; number_of_response:1; }","duration":"528.699526ms","start":"2025-07-15T11:02:02.136002Z","end":"2025-07-15T11:02:02.664701Z","steps":["trace[301873990] 'process raft request' (duration: 527.719179ms)"],"step_count":1} Jul 15 11:02:02.668538 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:02.666917Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:02.135854Z","time spent":"529.825604ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:02.675893 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:02.674496Z","caller":"traceutil/trace.go:171","msg":"trace[213766746] transaction","detail":"{read_only:false; response_revision:1708; number_of_response:1; }","duration":"414.248609ms","start":"2025-07-15T11:02:02.260124Z","end":"2025-07-15T11:02:02.674373Z","steps":["trace[213766746] 'process raft request' (duration: 413.723126ms)"],"step_count":1} Jul 15 11:02:02.675893 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:02.674625Z","caller":"traceutil/trace.go:171","msg":"trace[955529339] linearizableReadLoop","detail":"{readStateIndex:2087; appliedIndex:2085; }","duration":"342.172975ms","start":"2025-07-15T11:02:02.332429Z","end":"2025-07-15T11:02:02.674602Z","steps":["trace[955529339] 'read index received' (duration: 335.562525ms)","trace[955529339] 'applied index is now lower than readState.Index' (duration: 6.609177ms)"],"step_count":2} Jul 15 11:02:02.675893 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:02.675272Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"342.844209ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:02:02.675893 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:02.675805Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:02.260080Z","time spent":"414.608193ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:02.678111 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:02.675771Z","caller":"traceutil/trace.go:171","msg":"trace[1873528058] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1709; }","duration":"343.399604ms","start":"2025-07-15T11:02:02.332333Z","end":"2025-07-15T11:02:02.675733Z","steps":["trace[1873528058] 'agreement among raft nodes before linearized reading' (duration: 342.628121ms)"],"step_count":1} Jul 15 11:02:02.678111 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:02.676316Z","caller":"traceutil/trace.go:171","msg":"trace[664323476] transaction","detail":"{read_only:false; response_revision:1709; number_of_response:1; }","duration":"353.688466ms","start":"2025-07-15T11:02:02.322604Z","end":"2025-07-15T11:02:02.676292Z","steps":["trace[664323476] 'process raft request' (duration: 351.598731ms)"],"step_count":1} Jul 15 11:02:02.678111 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:02.676552Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:02.332312Z","time spent":"344.146272ms","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 11:02:02.678111 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:02.676739Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:02.322575Z","time spent":"354.012258ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":516,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:03.670904 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:03.669853Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"587.746306ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/\" range_end:\"/registry/events0\" count_only:true ","response":"range_response_count:0 size:8"} Jul 15 11:02:03.670904 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:03.670007Z","caller":"traceutil/trace.go:171","msg":"trace[1139485850] range","detail":"{range_begin:/registry/events/; range_end:/registry/events0; response_count:0; response_revision:1709; }","duration":"587.93305ms","start":"2025-07-15T11:02:03.082020Z","end":"2025-07-15T11:02:03.669953Z","steps":["trace[1139485850] 'count revisions from in-memory index tree' (duration: 587.651897ms)"],"step_count":1} Jul 15 11:02:03.670904 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:03.670076Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:03.082000Z","time spent":"588.054039ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":40,"response count":255,"response size":32,"request content":"key:\"/registry/events/\" range_end:\"/registry/events0\" count_only:true "} Jul 15 11:02:06.716301 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:06.715116Z","caller":"traceutil/trace.go:171","msg":"trace[1579998257] transaction","detail":"{read_only:false; response_revision:1722; number_of_response:1; }","duration":"278.325946ms","start":"2025-07-15T11:02:06.436753Z","end":"2025-07-15T11:02:06.715078Z","steps":["trace[1579998257] 'process raft request' (duration: 277.913645ms)"],"step_count":1} Jul 15 11:02:07.024435 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:07.023368Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"105.995049ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:3794"} Jul 15 11:02:07.024435 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:07.023552Z","caller":"traceutil/trace.go:171","msg":"trace[1243335326] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1726; }","duration":"106.242905ms","start":"2025-07-15T11:02:06.917285Z","end":"2025-07-15T11:02:07.023528Z","steps":["trace[1243335326] 'range keys from in-memory index tree' (duration: 105.558846ms)"],"step_count":1} Jul 15 11:02:14.118096 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:14.117567Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"130.76935ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"","error":"context canceled"} Jul 15 11:02:14.118096 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:14.117658Z","caller":"traceutil/trace.go:171","msg":"trace[233892087] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; }","duration":"130.948164ms","start":"2025-07-15T11:02:13.986690Z","end":"2025-07-15T11:02:14.117638Z","steps":["trace[233892087] 'agreement among raft nodes before linearized reading' (duration: 130.806869ms)"],"step_count":1} Jul 15 11:02:14.323797 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:14.323249Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885077464,"retry-timeout":"500ms"} Jul 15 11:02:14.386852 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:14.384464Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"5.978335341s","expected-duration":"1s"} Jul 15 11:02:14.386852 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:14.385602Z","caller":"traceutil/trace.go:171","msg":"trace[419589551] transaction","detail":"{read_only:false; response_revision:1727; number_of_response:1; }","duration":"5.979742298s","start":"2025-07-15T11:02:08.405826Z","end":"2025-07-15T11:02:14.385568Z","steps":["trace[419589551] 'process raft request' (duration: 5.979264225s)"],"step_count":1} Jul 15 11:02:14.386852 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:14.385945Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:08.405790Z","time spent":"5.979914817s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":625,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:19.477241 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:16.013284Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000913492s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:02:19.477241 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:16.013530Z","caller":"traceutil/trace.go:171","msg":"trace[147613610] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.00117813s","start":"2025-07-15T11:02:14.012250Z","end":"2025-07-15T11:02:16.013428Z","steps":["trace[147613610] 'agreement among raft nodes before linearized reading' (duration: 2.00095673s)"],"step_count":1} Jul 15 11:02:19.477241 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:16.013652Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:14.012233Z","time spent":"2.001395642s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:02:19.477241 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:16.013782Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.001386317s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:02:19.477241 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:16.013923Z","caller":"traceutil/trace.go:171","msg":"trace[1654290263] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.001607645s","start":"2025-07-15T11:02:14.012279Z","end":"2025-07-15T11:02:16.013886Z","steps":["trace[1654290263] 'agreement among raft nodes before linearized reading' (duration: 2.001379663s)"],"step_count":1} Jul 15 11:02:19.478640 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:16.014058Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:14.012262Z","time spent":"2.001735647s","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:02:19.478640 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.192018Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:14.011632Z","time spent":"5.180344357s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:02:19.478640 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.219138Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:13.986850Z","time spent":"5.232271888s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:02:19.478640 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.219471Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.232640836s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-scheduler\" ","response":"","error":"context canceled"} Jul 15 11:02:19.478640 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.219545Z","caller":"traceutil/trace.go:171","msg":"trace[577239413] range","detail":"{range_begin:/registry/leases/kube-system/kube-scheduler; range_end:; }","duration":"5.232935367s","start":"2025-07-15T11:02:13.986581Z","end":"2025-07-15T11:02:19.219516Z","steps":["trace[577239413] 'agreement among raft nodes before linearized reading' (duration: 5.232847297s)"],"step_count":1} Jul 15 11:02:19.478950 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.219641Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:13.986578Z","time spent":"5.233047299s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":45,"response count":0,"response size":0,"request content":"key:\"/registry/leases/kube-system/kube-scheduler\" "} Jul 15 11:02:19.478950 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.462315Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.4302483s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:02:19.478950 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.462444Z","caller":"traceutil/trace.go:171","msg":"trace[348897554] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"3.430477435s","start":"2025-07-15T11:02:16.031930Z","end":"2025-07-15T11:02:19.462408Z","steps":["trace[348897554] 'agreement among raft nodes before linearized reading' (duration: 3.430301407s)"],"step_count":1} Jul 15 11:02:19.478950 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.462508Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:16.031851Z","time spent":"3.430642432s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:02:19.542205 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.541611Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"5.156000615s","expected-duration":"1s"} Jul 15 11:02:19.557158 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.555695Z","caller":"traceutil/trace.go:171","msg":"trace[1480639630] transaction","detail":"{read_only:false; response_revision:1728; number_of_response:1; }","duration":"5.569312247s","start":"2025-07-15T11:02:13.986364Z","end":"2025-07-15T11:02:19.555676Z","steps":["trace[1480639630] 'process raft request' (duration: 5.556136172s)","trace[1480639630] 'compare' (duration: 12.623566ms)"],"step_count":2} Jul 15 11:02:19.557158 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.555810Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:13.986328Z","time spent":"5.569421856s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":699,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:19.557158 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.555975Z","caller":"traceutil/trace.go:171","msg":"trace[1987639672] linearizableReadLoop","detail":"{readStateIndex:2112; appliedIndex:2105; }","duration":"5.733684134s","start":"2025-07-15T11:02:13.822261Z","end":"2025-07-15T11:02:19.555945Z","steps":["trace[1987639672] 'read index received' (duration: 562.870215ms)","trace[1987639672] 'applied index is now lower than readState.Index' (duration: 5.170812777s)"],"step_count":2} Jul 15 11:02:19.557158 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.556159Z","caller":"traceutil/trace.go:171","msg":"trace[884934848] transaction","detail":"{read_only:false; response_revision:1733; number_of_response:1; }","duration":"5.476861339s","start":"2025-07-15T11:02:14.079282Z","end":"2025-07-15T11:02:19.556144Z","steps":["trace[884934848] 'process raft request' (duration: 5.476571526s)"],"step_count":1} Jul 15 11:02:19.558543 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.556244Z","caller":"traceutil/trace.go:171","msg":"trace[471550462] transaction","detail":"{read_only:false; response_revision:1731; number_of_response:1; }","duration":"5.544701923s","start":"2025-07-15T11:02:14.011533Z","end":"2025-07-15T11:02:19.556235Z","steps":["trace[471550462] 'process raft request' (duration: 5.543977423s)"],"step_count":1} Jul 15 11:02:19.558543 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.556302Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:14.011510Z","time spent":"5.544765187s","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":943,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:19.558543 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.556318Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:14.079260Z","time spent":"5.476936806s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":533,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:19.558543 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.556068Z","caller":"traceutil/trace.go:171","msg":"trace[1903487256] transaction","detail":"{read_only:false; response_revision:1730; number_of_response:1; }","duration":"5.546438956s","start":"2025-07-15T11:02:14.009620Z","end":"2025-07-15T11:02:19.556059Z","steps":["trace[1903487256] 'process raft request' (duration: 5.545640553s)"],"step_count":1} Jul 15 11:02:19.559076 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.556501Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:14.009596Z","time spent":"5.546879443s","remote":"172.24.4.149:54882","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1521,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:19.562909 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.557624Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.5764522s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/\" range_end:\"/registry/apiextensions.k8s.io/customresourcedefinitions0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:19.562909 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.557828Z","caller":"traceutil/trace.go:171","msg":"trace[1429710207] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/; range_end:/registry/apiextensions.k8s.io/customresourcedefinitions0; response_count:0; response_revision:1734; }","duration":"5.576722729s","start":"2025-07-15T11:02:13.981094Z","end":"2025-07-15T11:02:19.557817Z","steps":["trace[1429710207] 'agreement among raft nodes before linearized reading' (duration: 5.576464232s)"],"step_count":1} Jul 15 11:02:19.562909 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.558016Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:13.981055Z","time spent":"5.576945171s","remote":"172.24.4.149:33078","response type":"/etcdserverpb.KV/Range","request count":0,"request size":120,"response count":32,"response size":31,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/\" range_end:\"/registry/apiextensions.k8s.io/customresourcedefinitions0\" count_only:true "} Jul 15 11:02:19.562909 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.558714Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"321.889205ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/tigera-operator/operator-lock\" ","response":"range_response_count:1 size:505"} Jul 15 11:02:19.564697 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.558876Z","caller":"traceutil/trace.go:171","msg":"trace[1927078677] range","detail":"{range_begin:/registry/leases/tigera-operator/operator-lock; range_end:; response_count:1; response_revision:1734; }","duration":"322.083868ms","start":"2025-07-15T11:02:19.236783Z","end":"2025-07-15T11:02:19.558866Z","steps":["trace[1927078677] 'agreement among raft nodes before linearized reading' (duration: 321.884227ms)"],"step_count":1} Jul 15 11:02:19.564697 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.559080Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:19.236727Z","time spent":"322.313183ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":1,"response size":529,"request content":"key:\"/registry/leases/tigera-operator/operator-lock\" "} Jul 15 11:02:19.564697 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.559287Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"336.138852ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/kube-system/kube-controller-manager\" ","response":"range_response_count:1 size:531"} Jul 15 11:02:19.564697 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.559457Z","caller":"traceutil/trace.go:171","msg":"trace[1216731115] range","detail":"{range_begin:/registry/leases/kube-system/kube-controller-manager; range_end:; response_count:1; response_revision:1734; }","duration":"336.308828ms","start":"2025-07-15T11:02:19.223139Z","end":"2025-07-15T11:02:19.559448Z","steps":["trace[1216731115] 'agreement among raft nodes before linearized reading' (duration: 336.125287ms)"],"step_count":1} Jul 15 11:02:19.566147 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.559677Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:19.223014Z","time spent":"336.651149ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Range","request count":0,"request size":54,"response count":1,"response size":555,"request content":"key:\"/registry/leases/kube-system/kube-controller-manager\" "} Jul 15 11:02:19.566147 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.559900Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"341.701439ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/networksets/\" range_end:\"/registry/crd.projectcalico.org/networksets0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:19.566147 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.559949Z","caller":"traceutil/trace.go:171","msg":"trace[1695135936] range","detail":"{range_begin:/registry/crd.projectcalico.org/networksets/; range_end:/registry/crd.projectcalico.org/networksets0; response_count:0; response_revision:1734; }","duration":"341.862652ms","start":"2025-07-15T11:02:19.218078Z","end":"2025-07-15T11:02:19.559940Z","steps":["trace[1695135936] 'agreement among raft nodes before linearized reading' (duration: 341.774292ms)"],"step_count":1} Jul 15 11:02:19.566147 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.559990Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:19.218066Z","time spent":"341.902814ms","remote":"172.24.4.149:54570","response type":"/etcdserverpb.KV/Range","request count":0,"request size":94,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/networksets/\" range_end:\"/registry/crd.projectcalico.org/networksets0\" count_only:true "} Jul 15 11:02:19.566770 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.560128Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"342.245694ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/\" range_end:\"/registry/clusterroles0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:19.566770 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.560181Z","caller":"traceutil/trace.go:171","msg":"trace[1844587764] range","detail":"{range_begin:/registry/clusterroles/; range_end:/registry/clusterroles0; response_count:0; response_revision:1734; }","duration":"342.302277ms","start":"2025-07-15T11:02:19.217840Z","end":"2025-07-15T11:02:19.560143Z","steps":["trace[1844587764] 'agreement among raft nodes before linearized reading' (duration: 342.204419ms)"],"step_count":1} Jul 15 11:02:19.566770 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.560223Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:19.217819Z","time spent":"342.376881ms","remote":"172.24.4.149:33368","response type":"/etcdserverpb.KV/Range","request count":0,"request size":52,"response count":77,"response size":31,"request content":"key:\"/registry/clusterroles/\" range_end:\"/registry/clusterroles0\" count_only:true "} Jul 15 11:02:19.566770 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.560313Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"353.573928ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/configmaps/\" range_end:\"/registry/configmaps0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:19.566770 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.560332Z","caller":"traceutil/trace.go:171","msg":"trace[1163789562] range","detail":"{range_begin:/registry/configmaps/; range_end:/registry/configmaps0; response_count:0; response_revision:1734; }","duration":"353.654704ms","start":"2025-07-15T11:02:19.206672Z","end":"2025-07-15T11:02:19.560326Z","steps":["trace[1163789562] 'agreement among raft nodes before linearized reading' (duration: 353.612678ms)"],"step_count":1} Jul 15 11:02:19.567265 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.560351Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:19.206637Z","time spent":"353.70774ms","remote":"172.24.4.149:33128","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":20,"response size":31,"request content":"key:\"/registry/configmaps/\" range_end:\"/registry/configmaps0\" count_only:true "} Jul 15 11:02:19.567265 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.560775Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.399703374s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"range_response_count:1 size:886"} Jul 15 11:02:19.567265 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.560933Z","caller":"traceutil/trace.go:171","msg":"trace[1657701974] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; response_count:1; response_revision:1734; }","duration":"3.399898546s","start":"2025-07-15T11:02:16.160988Z","end":"2025-07-15T11:02:19.560886Z","steps":["trace[1657701974] 'agreement among raft nodes before linearized reading' (duration: 3.399662299s)"],"step_count":1} Jul 15 11:02:19.567265 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.561183Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:16.160953Z","time spent":"3.400165218s","remote":"172.24.4.149:54994","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":1,"response size":910,"request content":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" "} Jul 15 11:02:19.567265 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.561491Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.562339037s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/gatewayapis/\" range_end:\"/registry/operator.tigera.io/gatewayapis0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:19.568765 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.561482Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.561743612s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowschemas/\" range_end:\"/registry/flowschemas0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:19.568765 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.561645Z","caller":"traceutil/trace.go:171","msg":"trace[672836337] range","detail":"{range_begin:/registry/flowschemas/; range_end:/registry/flowschemas0; response_count:0; response_revision:1734; }","duration":"5.562253523s","start":"2025-07-15T11:02:13.999355Z","end":"2025-07-15T11:02:19.561608Z","steps":["trace[672836337] 'agreement among raft nodes before linearized reading' (duration: 5.561783943s)"],"step_count":1} Jul 15 11:02:19.568765 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.561586Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.575011394s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/persistentvolumes/\" range_end:\"/registry/persistentvolumes0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:19.568765 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.561770Z","caller":"traceutil/trace.go:171","msg":"trace[243144829] range","detail":"{range_begin:/registry/persistentvolumes/; range_end:/registry/persistentvolumes0; response_count:0; response_revision:1734; }","duration":"5.57520794s","start":"2025-07-15T11:02:13.986553Z","end":"2025-07-15T11:02:19.561761Z","steps":["trace[243144829] 'agreement among raft nodes before linearized reading' (duration: 5.575014771s)"],"step_count":1} Jul 15 11:02:19.569059 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.561948Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:13.986546Z","time spent":"5.575387525s","remote":"172.24.4.149:33186","response type":"/etcdserverpb.KV/Range","request count":0,"request size":62,"response count":0,"response size":29,"request content":"key:\"/registry/persistentvolumes/\" range_end:\"/registry/persistentvolumes0\" count_only:true "} Jul 15 11:02:19.569059 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.562118Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.575572148s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/ci-4230-2-100-6ef5831daf.novalocal\" ","response":"range_response_count:1 size:6690"} Jul 15 11:02:19.569059 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.562147Z","caller":"traceutil/trace.go:171","msg":"trace[1419822274] range","detail":"{range_begin:/registry/minions/ci-4230-2-100-6ef5831daf.novalocal; range_end:; response_count:1; response_revision:1734; }","duration":"5.575595469s","start":"2025-07-15T11:02:13.986538Z","end":"2025-07-15T11:02:19.562134Z","steps":["trace[1419822274] 'agreement among raft nodes before linearized reading' (duration: 5.575492734s)"],"step_count":1} Jul 15 11:02:19.569059 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.562143Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.525934502s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76\" ","response":"range_response_count:1 size:905"} Jul 15 11:02:19.569059 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.562205Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.578471726s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/ranges/servicenodeports\" ","response":"range_response_count:1 size:118"} Jul 15 11:02:19.569605 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.562202Z","caller":"traceutil/trace.go:171","msg":"trace[1743115431] range","detail":"{range_begin:/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76; range_end:; response_count:1; response_revision:1734; }","duration":"3.525992537s","start":"2025-07-15T11:02:16.036191Z","end":"2025-07-15T11:02:19.562184Z","steps":["trace[1743115431] 'agreement among raft nodes before linearized reading' (duration: 3.525718431s)"],"step_count":1} Jul 15 11:02:19.569605 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.562229Z","caller":"traceutil/trace.go:171","msg":"trace[44735380] range","detail":"{range_begin:/registry/ranges/servicenodeports; range_end:; response_count:1; response_revision:1734; }","duration":"5.578510576s","start":"2025-07-15T11:02:13.983713Z","end":"2025-07-15T11:02:19.562223Z","steps":["trace[44735380] 'agreement among raft nodes before linearized reading' (duration: 5.578465324s)"],"step_count":1} Jul 15 11:02:19.569605 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.562249Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:13.983705Z","time spent":"5.578538075s","remote":"172.24.4.149:33092","response type":"/etcdserverpb.KV/Range","request count":0,"request size":35,"response count":1,"response size":142,"request content":"key:\"/registry/ranges/servicenodeports\" "} Jul 15 11:02:19.569605 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.562252Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:16.036159Z","time spent":"3.526077961s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":97,"response count":1,"response size":929,"request content":"key:\"/registry/events/kube-system/kube-apiserver-ci-4230-2-100-6ef5831daf.novalocal.1852678572039b76\" "} Jul 15 11:02:19.570015 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.562339Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.578700149s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/specs/\" range_end:\"/registry/services/specs0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:19.570015 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.562369Z","caller":"traceutil/trace.go:171","msg":"trace[939952402] range","detail":"{range_begin:/registry/services/specs/; range_end:/registry/services/specs0; response_count:0; response_revision:1734; }","duration":"5.578740671s","start":"2025-07-15T11:02:13.983614Z","end":"2025-07-15T11:02:19.562354Z","steps":["trace[939952402] 'agreement among raft nodes before linearized reading' (duration: 5.578694929s)"],"step_count":1} Jul 15 11:02:19.570015 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.562423Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:13.983605Z","time spent":"5.578807864s","remote":"172.24.4.149:33212","response type":"/etcdserverpb.KV/Range","request count":0,"request size":56,"response count":7,"response size":31,"request content":"key:\"/registry/services/specs/\" range_end:\"/registry/services/specs0\" count_only:true "} Jul 15 11:02:19.570015 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.562511Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.174330774s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 11:02:19.570015 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.562563Z","caller":"traceutil/trace.go:171","msg":"trace[12569677] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1734; }","duration":"4.174415057s","start":"2025-07-15T11:02:15.388133Z","end":"2025-07-15T11:02:19.562548Z","steps":["trace[12569677] 'agreement among raft nodes before linearized reading' (duration: 4.174246392s)"],"step_count":1} Jul 15 11:02:19.570360 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.562607Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:15.388112Z","time spent":"4.174480925s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 11:02:19.570360 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.562682Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.579156803s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/ranges/serviceips\" ","response":"range_response_count:1 size:108196"} Jul 15 11:02:19.570360 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.562709Z","caller":"traceutil/trace.go:171","msg":"trace[1774807040] range","detail":"{range_begin:/registry/ranges/serviceips; range_end:; response_count:1; response_revision:1734; }","duration":"5.579220399s","start":"2025-07-15T11:02:13.983481Z","end":"2025-07-15T11:02:19.562701Z","steps":["trace[1774807040] 'agreement among raft nodes before linearized reading' (duration: 5.579043579s)"],"step_count":1} Jul 15 11:02:19.570360 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.562737Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:13.983456Z","time spent":"5.579269877s","remote":"172.24.4.149:33090","response type":"/etcdserverpb.KV/Range","request count":0,"request size":29,"response count":1,"response size":108220,"request content":"key:\"/registry/ranges/serviceips\" "} Jul 15 11:02:19.570360 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.561865Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:13.999348Z","time spent":"5.562363843s","remote":"172.24.4.149:33426","response type":"/etcdserverpb.KV/Range","request count":0,"request size":50,"response count":13,"response size":31,"request content":"key:\"/registry/flowschemas/\" range_end:\"/registry/flowschemas0\" count_only:true "} Jul 15 11:02:19.570984 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.561546Z","caller":"traceutil/trace.go:171","msg":"trace[1270122485] range","detail":"{range_begin:/registry/operator.tigera.io/gatewayapis/; range_end:/registry/operator.tigera.io/gatewayapis0; response_count:0; response_revision:1734; }","duration":"5.562431874s","start":"2025-07-15T11:02:13.999106Z","end":"2025-07-15T11:02:19.561538Z","steps":["trace[1270122485] 'agreement among raft nodes before linearized reading' (duration: 5.562363129s)"],"step_count":1} Jul 15 11:02:19.570984 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.564768Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:13.999082Z","time spent":"5.565663917s","remote":"172.24.4.149:54960","response type":"/etcdserverpb.KV/Range","request count":0,"request size":88,"response count":0,"response size":29,"request content":"key:\"/registry/operator.tigera.io/gatewayapis/\" range_end:\"/registry/operator.tigera.io/gatewayapis0\" count_only:true "} Jul 15 11:02:19.570984 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.566171Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.566601465s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/namespaces/\" range_end:\"/registry/namespaces0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:19.570984 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.567069Z","caller":"traceutil/trace.go:171","msg":"trace[1613258442] range","detail":"{range_begin:/registry/namespaces/; range_end:/registry/namespaces0; response_count:0; response_revision:1734; }","duration":"5.5674772s","start":"2025-07-15T11:02:13.999520Z","end":"2025-07-15T11:02:19.566997Z","steps":["trace[1613258442] 'agreement among raft nodes before linearized reading' (duration: 5.566555493s)"],"step_count":1} Jul 15 11:02:19.571164 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.567575Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:13.999431Z","time spent":"5.568108781s","remote":"172.24.4.149:33144","response type":"/etcdserverpb.KV/Range","request count":0,"request size":48,"response count":7,"response size":31,"request content":"key:\"/registry/namespaces/\" range_end:\"/registry/namespaces0\" count_only:true "} Jul 15 11:02:19.571164 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.567919Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.400842949s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/horizontalpodautoscalers/\" range_end:\"/registry/horizontalpodautoscalers0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:19.571164 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.568172Z","caller":"traceutil/trace.go:171","msg":"trace[999866499] range","detail":"{range_begin:/registry/horizontalpodautoscalers/; range_end:/registry/horizontalpodautoscalers0; response_count:0; response_revision:1734; }","duration":"5.40124715s","start":"2025-07-15T11:02:14.166903Z","end":"2025-07-15T11:02:19.568150Z","steps":["trace[999866499] 'agreement among raft nodes before linearized reading' (duration: 5.400914307s)"],"step_count":1} Jul 15 11:02:19.571164 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.568246Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:14.166873Z","time spent":"5.401349315s","remote":"172.24.4.149:33242","response type":"/etcdserverpb.KV/Range","request count":0,"request size":76,"response count":0,"response size":29,"request content":"key:\"/registry/horizontalpodautoscalers/\" range_end:\"/registry/horizontalpodautoscalers0\" count_only:true "} Jul 15 11:02:19.571329 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.562168Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:13.986509Z","time spent":"5.575652283s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Range","request count":0,"request size":54,"response count":1,"response size":6714,"request content":"key:\"/registry/minions/ci-4230-2-100-6ef5831daf.novalocal\" "} Jul 15 11:02:19.572338 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.572078Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.320441995s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/installations.operator.tigera.io\" ","response":"range_response_count:1 size:630504"} Jul 15 11:02:19.572613 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:19.572154Z","caller":"traceutil/trace.go:171","msg":"trace[1149749092] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/installations.operator.tigera.io; range_end:; response_count:1; response_revision:1734; }","duration":"5.32057776s","start":"2025-07-15T11:02:14.251559Z","end":"2025-07-15T11:02:19.572136Z","steps":["trace[1149749092] 'agreement among raft nodes before linearized reading' (duration: 5.311146336s)"],"step_count":1} Jul 15 11:02:19.572613 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:19.572202Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:14.251256Z","time spent":"5.320930978s","remote":"172.24.4.149:33078","response type":"/etcdserverpb.KV/Range","request count":0,"request size":91,"response count":1,"response size":630528,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/installations.operator.tigera.io\" "} Jul 15 11:02:20.224254 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:20.222571Z","caller":"traceutil/trace.go:171","msg":"trace[1308382678] transaction","detail":"{read_only:false; number_of_response:0; response_revision:1738; }","duration":"442.792965ms","start":"2025-07-15T11:02:19.779748Z","end":"2025-07-15T11:02:20.222541Z","steps":["trace[1308382678] 'process raft request' (duration: 442.733117ms)"],"step_count":1} Jul 15 11:02:20.224254 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:20.222597Z","caller":"traceutil/trace.go:171","msg":"trace[1347019537] transaction","detail":"{read_only:false; response_revision:1738; number_of_response:1; }","duration":"448.185827ms","start":"2025-07-15T11:02:19.774372Z","end":"2025-07-15T11:02:20.222558Z","steps":["trace[1347019537] 'process raft request' (duration: 407.797151ms)","trace[1347019537] 'compare' (duration: 40.00433ms)"],"step_count":2} Jul 15 11:02:20.224254 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:20.222783Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:19.779730Z","time spent":"442.950752ms","remote":"172.24.4.149:54498","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":29,"request content":"compare: success:> failure:<>"} Jul 15 11:02:20.224254 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:20.222923Z","caller":"traceutil/trace.go:171","msg":"trace[72571669] linearizableReadLoop","detail":"{readStateIndex:2121; appliedIndex:2120; }","duration":"447.128947ms","start":"2025-07-15T11:02:19.775776Z","end":"2025-07-15T11:02:20.222905Z","steps":["trace[72571669] 'read index received' (duration: 406.42435ms)","trace[72571669] 'applied index is now lower than readState.Index' (duration: 40.703135ms)"],"step_count":2} Jul 15 11:02:20.225096 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:20.222956Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:19.774342Z","time spent":"448.495087ms","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1184,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:20.225096 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:20.223183Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"447.39538ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/calico-apiserver-access-calico-crds\" ","response":"range_response_count:1 size:661"} Jul 15 11:02:20.225096 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:20.223232Z","caller":"traceutil/trace.go:171","msg":"trace[401062048] range","detail":"{range_begin:/registry/clusterrolebindings/calico-apiserver-access-calico-crds; range_end:; response_count:1; response_revision:1738; }","duration":"447.510307ms","start":"2025-07-15T11:02:19.775707Z","end":"2025-07-15T11:02:20.223217Z","steps":["trace[401062048] 'agreement among raft nodes before linearized reading' (duration: 447.304886ms)"],"step_count":1} Jul 15 11:02:20.225096 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:20.223278Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:19.775672Z","time spent":"447.591374ms","remote":"172.24.4.149:33376","response type":"/etcdserverpb.KV/Range","request count":0,"request size":67,"response count":1,"response size":685,"request content":"key:\"/registry/clusterrolebindings/calico-apiserver-access-calico-crds\" "} Jul 15 11:02:20.225740 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:20.223306Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"443.687484ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/ippools/default-ipv4-ippool\" ","response":"range_response_count:1 size:1224"} Jul 15 11:02:20.225740 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:20.223341Z","caller":"traceutil/trace.go:171","msg":"trace[79851601] range","detail":"{range_begin:/registry/crd.projectcalico.org/ippools/default-ipv4-ippool; range_end:; response_count:1; response_revision:1738; }","duration":"443.747112ms","start":"2025-07-15T11:02:19.779580Z","end":"2025-07-15T11:02:20.223327Z","steps":["trace[79851601] 'agreement among raft nodes before linearized reading' (duration: 443.638215ms)"],"step_count":1} Jul 15 11:02:20.225740 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:20.223457Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:19.779563Z","time spent":"443.884551ms","remote":"172.24.4.149:54896","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":1,"response size":1248,"request content":"key:\"/registry/crd.projectcalico.org/ippools/default-ipv4-ippool\" "} Jul 15 11:02:20.757307 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:20.755795Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"101.908188ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 11:02:20.757307 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:20.755910Z","caller":"traceutil/trace.go:171","msg":"trace[1933440134] linearizableReadLoop","detail":"{readStateIndex:2125; appliedIndex:2124; }","duration":"184.545376ms","start":"2025-07-15T11:02:20.571346Z","end":"2025-07-15T11:02:20.755891Z","steps":["trace[1933440134] 'read index received' (duration: 45.843404ms)","trace[1933440134] 'applied index is now lower than readState.Index' (duration: 138.700609ms)"],"step_count":2} Jul 15 11:02:20.757307 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:20.755967Z","caller":"traceutil/trace.go:171","msg":"trace[2081090785] transaction","detail":"{read_only:false; response_revision:1740; number_of_response:1; }","duration":"194.978852ms","start":"2025-07-15T11:02:20.560924Z","end":"2025-07-15T11:02:20.755903Z","steps":["trace[2081090785] 'process raft request' (duration: 92.855302ms)","trace[2081090785] 'compare' (duration: 101.746878ms)"],"step_count":2} Jul 15 11:02:20.757307 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:20.756468Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"185.108794ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/tigera-apiserver\" ","response":"range_response_count:0 size:5"} Jul 15 11:02:20.758716 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:20.756543Z","caller":"traceutil/trace.go:171","msg":"trace[1118711736] range","detail":"{range_begin:/registry/clusterrolebindings/tigera-apiserver; range_end:; response_count:0; response_revision:1740; }","duration":"185.205309ms","start":"2025-07-15T11:02:20.571302Z","end":"2025-07-15T11:02:20.756507Z","steps":["trace[1118711736] 'agreement among raft nodes before linearized reading' (duration: 184.681402ms)"],"step_count":1} Jul 15 11:02:20.759330 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:20.758850Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"154.363986ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/apiservers.operator.tigera.io\" ","response":"range_response_count:1 size:61144"} Jul 15 11:02:20.760058 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:20.758954Z","caller":"traceutil/trace.go:171","msg":"trace[1876564096] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/apiservers.operator.tigera.io; range_end:; response_count:1; response_revision:1740; }","duration":"154.513797ms","start":"2025-07-15T11:02:20.604423Z","end":"2025-07-15T11:02:20.758937Z","steps":["trace[1876564096] 'agreement among raft nodes before linearized reading' (duration: 151.780453ms)"],"step_count":1} Jul 15 11:02:21.835886 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:21.831196Z","caller":"traceutil/trace.go:171","msg":"trace[455524714] transaction","detail":"{read_only:false; response_revision:1741; number_of_response:1; }","duration":"199.688198ms","start":"2025-07-15T11:02:21.631472Z","end":"2025-07-15T11:02:21.831161Z","steps":["trace[455524714] 'process raft request' (duration: 199.230671ms)"],"step_count":1} Jul 15 11:02:21.835886 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:21.832244Z","caller":"traceutil/trace.go:171","msg":"trace[146685906] linearizableReadLoop","detail":"{readStateIndex:2126; appliedIndex:2126; }","duration":"198.0091ms","start":"2025-07-15T11:02:21.634216Z","end":"2025-07-15T11:02:21.832225Z","steps":["trace[146685906] 'read index received' (duration: 198.002257ms)","trace[146685906] 'applied index is now lower than readState.Index' (duration: 5.711µs)"],"step_count":2} Jul 15 11:02:21.835886 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:21.834265Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"198.161356ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/tigera-apiserver\" ","response":"range_response_count:0 size:5"} Jul 15 11:02:21.835886 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:21.834356Z","caller":"traceutil/trace.go:171","msg":"trace[2100697049] range","detail":"{range_begin:/registry/clusterrolebindings/tigera-apiserver; range_end:; response_count:0; response_revision:1741; }","duration":"200.146578ms","start":"2025-07-15T11:02:21.634185Z","end":"2025-07-15T11:02:21.834332Z","steps":["trace[2100697049] 'agreement among raft nodes before linearized reading' (duration: 198.124799ms)"],"step_count":1} Jul 15 11:02:22.135086 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.131638Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"275.990496ms","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 11:02:22.135086 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.131949Z","caller":"traceutil/trace.go:171","msg":"trace[1462153727] transaction","detail":"{read_only:false; response_revision:1742; number_of_response:1; }","duration":"494.41648ms","start":"2025-07-15T11:02:21.637503Z","end":"2025-07-15T11:02:22.131919Z","steps":["trace[1462153727] 'process raft request' (duration: 218.016802ms)","trace[1462153727] 'compare' (duration: 275.696635ms)"],"step_count":2} Jul 15 11:02:22.135086 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.132119Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:21.637475Z","time spent":"494.552736ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":516,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:22.140530 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.132238Z","caller":"traceutil/trace.go:171","msg":"trace[1880510885] linearizableReadLoop","detail":"{readStateIndex:2128; appliedIndex:2126; }","duration":"299.924435ms","start":"2025-07-15T11:02:21.832286Z","end":"2025-07-15T11:02:22.132210Z","steps":["trace[1880510885] 'read index received' (duration: 23.18905ms)","trace[1880510885] 'applied index is now lower than readState.Index' (duration: 276.734154ms)"],"step_count":2} Jul 15 11:02:22.140530 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.132888Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"426.047418ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/apiservers/\" range_end:\"/registry/operator.tigera.io/apiservers0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:22.140530 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.132965Z","caller":"traceutil/trace.go:171","msg":"trace[4031456] range","detail":"{range_begin:/registry/operator.tigera.io/apiservers/; range_end:/registry/operator.tigera.io/apiservers0; response_count:0; response_revision:1743; }","duration":"426.162858ms","start":"2025-07-15T11:02:21.706767Z","end":"2025-07-15T11:02:22.132930Z","steps":["trace[4031456] 'agreement among raft nodes before linearized reading' (duration: 425.689461ms)"],"step_count":1} Jul 15 11:02:22.140530 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.133038Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:21.706735Z","time spent":"426.267697ms","remote":"172.24.4.149:54882","response type":"/etcdserverpb.KV/Range","request count":0,"request size":86,"response count":1,"response size":31,"request content":"key:\"/registry/operator.tigera.io/apiservers/\" range_end:\"/registry/operator.tigera.io/apiservers0\" count_only:true "} Jul 15 11:02:22.141161 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.133119Z","caller":"traceutil/trace.go:171","msg":"trace[1902574853] transaction","detail":"{read_only:false; response_revision:1743; number_of_response:1; }","duration":"390.54383ms","start":"2025-07-15T11:02:21.742562Z","end":"2025-07-15T11:02:22.133106Z","steps":["trace[1902574853] 'process raft request' (duration: 389.205497ms)"],"step_count":1} Jul 15 11:02:22.141161 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.133126Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"288.37304ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/\" range_end:\"/registry/serviceaccounts0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:22.141161 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.133200Z","caller":"traceutil/trace.go:171","msg":"trace[263975007] range","detail":"{range_begin:/registry/serviceaccounts/; range_end:/registry/serviceaccounts0; response_count:0; response_revision:1743; }","duration":"288.513523ms","start":"2025-07-15T11:02:21.844662Z","end":"2025-07-15T11:02:22.133175Z","steps":["trace[263975007] 'agreement among raft nodes before linearized reading' (duration: 288.357871ms)"],"step_count":1} Jul 15 11:02:22.141161 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.133314Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:21.742523Z","time spent":"390.624015ms","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":490,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:22.141880 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.133590Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"228.665865ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/tigera-csr-creator\" ","response":"range_response_count:0 size:5"} Jul 15 11:02:22.141880 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.133650Z","caller":"traceutil/trace.go:171","msg":"trace[1303841290] range","detail":"{range_begin:/registry/clusterroles/tigera-csr-creator; range_end:; response_count:0; response_revision:1743; }","duration":"228.754194ms","start":"2025-07-15T11:02:21.904881Z","end":"2025-07-15T11:02:22.133636Z","steps":["trace[1303841290] 'agreement among raft nodes before linearized reading' (duration: 228.643443ms)"],"step_count":1} Jul 15 11:02:22.141880 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.133863Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"214.986627ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:5590"} Jul 15 11:02:22.141880 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.133916Z","caller":"traceutil/trace.go:171","msg":"trace[620022705] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1743; }","duration":"215.207517ms","start":"2025-07-15T11:02:21.918696Z","end":"2025-07-15T11:02:22.133903Z","steps":["trace[620022705] 'agreement among raft nodes before linearized reading' (duration: 215.023484ms)"],"step_count":1} Jul 15 11:02:22.141880 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.134542Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"216.164109ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/pods/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal\" ","response":"range_response_count:1 size:4794"} Jul 15 11:02:22.142423 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.134613Z","caller":"traceutil/trace.go:171","msg":"trace[2092488423] range","detail":"{range_begin:/registry/pods/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal; range_end:; response_count:1; response_revision:1743; }","duration":"216.267947ms","start":"2025-07-15T11:02:21.918330Z","end":"2025-07-15T11:02:22.134598Z","steps":["trace[2092488423] 'agreement among raft nodes before linearized reading' (duration: 215.953136ms)"],"step_count":1} Jul 15 11:02:22.142423 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.135059Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"218.3908ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:5590"} Jul 15 11:02:22.142423 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.135120Z","caller":"traceutil/trace.go:171","msg":"trace[1614808848] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1743; }","duration":"218.515305ms","start":"2025-07-15T11:02:21.916591Z","end":"2025-07-15T11:02:22.135106Z","steps":["trace[1614808848] 'agreement among raft nodes before linearized reading' (duration: 218.218537ms)"],"step_count":1} Jul 15 11:02:22.142423 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.135472Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"230.305323ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/whiskers/default\" ","response":"range_response_count:1 size:948"} Jul 15 11:02:22.142876 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.135595Z","caller":"traceutil/trace.go:171","msg":"trace[1070311092] range","detail":"{range_begin:/registry/operator.tigera.io/whiskers/default; range_end:; response_count:1; response_revision:1743; }","duration":"230.385769ms","start":"2025-07-15T11:02:21.905123Z","end":"2025-07-15T11:02:22.135509Z","steps":["trace[1070311092] 'agreement among raft nodes before linearized reading' (duration: 230.147718ms)"],"step_count":1} Jul 15 11:02:22.142876 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.136559Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"233.657723ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterroles/tigera-uisettingsgroup-getter\" ","response":"range_response_count:0 size:5"} Jul 15 11:02:22.142876 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.136625Z","caller":"traceutil/trace.go:171","msg":"trace[691527101] range","detail":"{range_begin:/registry/clusterroles/tigera-uisettingsgroup-getter; range_end:; response_count:0; response_revision:1743; }","duration":"233.808275ms","start":"2025-07-15T11:02:21.902801Z","end":"2025-07-15T11:02:22.136609Z","steps":["trace[691527101] 'agreement among raft nodes before linearized reading' (duration: 233.006012ms)"],"step_count":1} Jul 15 11:02:22.142876 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.136874Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"216.733981ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f\" ","response":"range_response_count:1 size:996"} Jul 15 11:02:22.143247 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.138507Z","caller":"traceutil/trace.go:171","msg":"trace[694276135] range","detail":"{range_begin:/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f; range_end:; response_count:1; response_revision:1743; }","duration":"218.746867ms","start":"2025-07-15T11:02:21.919719Z","end":"2025-07-15T11:02:22.138466Z","steps":["trace[694276135] 'agreement among raft nodes before linearized reading' (duration: 213.616377ms)"],"step_count":1} Jul 15 11:02:22.143247 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.137696Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"234.726026ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:5590"} Jul 15 11:02:22.143247 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.138799Z","caller":"traceutil/trace.go:171","msg":"trace[57893511] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1743; }","duration":"235.859569ms","start":"2025-07-15T11:02:21.902924Z","end":"2025-07-15T11:02:22.138784Z","steps":["trace[57893511] 'agreement among raft nodes before linearized reading' (duration: 233.883691ms)"],"step_count":1} Jul 15 11:02:22.143247 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.137733Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"242.068638ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/installations/default\" ","response":"range_response_count:1 size:5590"} Jul 15 11:02:22.143692 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.139078Z","caller":"traceutil/trace.go:171","msg":"trace[1300322002] range","detail":"{range_begin:/registry/operator.tigera.io/installations/default; range_end:; response_count:1; response_revision:1743; }","duration":"243.461699ms","start":"2025-07-15T11:02:21.895601Z","end":"2025-07-15T11:02:22.139063Z","steps":["trace[1300322002] 'agreement among raft nodes before linearized reading' (duration: 241.827279ms)"],"step_count":1} Jul 15 11:02:22.143692 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.137884Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"260.266885ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/tigera-operator/tigera-ca-private\" ","response":"range_response_count:1 size:3369"} Jul 15 11:02:22.143692 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.139516Z","caller":"traceutil/trace.go:171","msg":"trace[137250749] range","detail":"{range_begin:/registry/secrets/tigera-operator/tigera-ca-private; range_end:; response_count:1; response_revision:1743; }","duration":"261.931288ms","start":"2025-07-15T11:02:21.877561Z","end":"2025-07-15T11:02:22.139492Z","steps":["trace[137250749] 'agreement among raft nodes before linearized reading' (duration: 260.206695ms)"],"step_count":1} Jul 15 11:02:22.769595 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.761527Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"385.536371ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/tigera-apiserver-access-tigera-crds\" ","response":"range_response_count:0 size:5"} Jul 15 11:02:22.769595 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.766242Z","caller":"traceutil/trace.go:171","msg":"trace[1286520350] range","detail":"{range_begin:/registry/clusterrolebindings/tigera-apiserver-access-tigera-crds; range_end:; response_count:0; response_revision:1745; }","duration":"390.638739ms","start":"2025-07-15T11:02:22.375477Z","end":"2025-07-15T11:02:22.766116Z","steps":["trace[1286520350] 'range keys from in-memory index tree' (duration: 385.296325ms)"],"step_count":1} Jul 15 11:02:22.769595 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.766937Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:22.375458Z","time spent":"391.317509ms","remote":"172.24.4.149:33376","response type":"/etcdserverpb.KV/Range","request count":0,"request size":67,"response count":0,"response size":29,"request content":"key:\"/registry/clusterrolebindings/tigera-apiserver-access-tigera-crds\" "} Jul 15 11:02:22.769595 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.767352Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"390.063118ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/serviceaccounts/calico-system/whisker\" ","response":"range_response_count:1 size:426"} Jul 15 11:02:22.769595 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.767999Z","caller":"traceutil/trace.go:171","msg":"trace[1692831979] range","detail":"{range_begin:/registry/serviceaccounts/calico-system/whisker; range_end:; response_count:1; response_revision:1745; }","duration":"391.088775ms","start":"2025-07-15T11:02:22.376854Z","end":"2025-07-15T11:02:22.767943Z","steps":["trace[1692831979] 'range keys from in-memory index tree' (duration: 389.769615ms)"],"step_count":1} Jul 15 11:02:22.773447 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.768319Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:22.376833Z","time spent":"391.389899ms","remote":"172.24.4.149:33226","response type":"/etcdserverpb.KV/Range","request count":0,"request size":49,"response count":1,"response size":450,"request content":"key:\"/registry/serviceaccounts/calico-system/whisker\" "} Jul 15 11:02:22.774462 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.772325Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"400.052326ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/configmaps/calico-system/goldmane\" ","response":"range_response_count:1 size:494"} Jul 15 11:02:22.776609 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.772924Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"390.605218ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/tiers.crd.projectcalico.org\" ","response":"range_response_count:1 size:3593"} Jul 15 11:02:22.776609 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.773082Z","caller":"traceutil/trace.go:171","msg":"trace[1423331772] range","detail":"{range_begin:/registry/configmaps/calico-system/goldmane; range_end:; response_count:1; response_revision:1745; }","duration":"400.349313ms","start":"2025-07-15T11:02:22.372198Z","end":"2025-07-15T11:02:22.772547Z","steps":["trace[1423331772] 'range keys from in-memory index tree' (duration: 399.669973ms)"],"step_count":1} Jul 15 11:02:22.776609 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:22.773093Z","caller":"traceutil/trace.go:171","msg":"trace[1297947831] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/tiers.crd.projectcalico.org; range_end:; response_count:1; response_revision:1745; }","duration":"390.806252ms","start":"2025-07-15T11:02:22.382262Z","end":"2025-07-15T11:02:22.773069Z","steps":["trace[1297947831] 'range keys from in-memory index tree' (duration: 378.443468ms)"],"step_count":1} Jul 15 11:02:22.779426 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.774896Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:22.382245Z","time spent":"392.593609ms","remote":"172.24.4.149:33078","response type":"/etcdserverpb.KV/Range","request count":0,"request size":86,"response count":1,"response size":3617,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/tiers.crd.projectcalico.org\" "} Jul 15 11:02:22.779426 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:22.773556Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:22.372181Z","time spent":"401.34168ms","remote":"172.24.4.149:33128","response type":"/etcdserverpb.KV/Range","request count":0,"request size":45,"response count":1,"response size":518,"request content":"key:\"/registry/configmaps/calico-system/goldmane\" "} Jul 15 11:02:25.443052 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:25.442305Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885077793,"retry-timeout":"500ms"} Jul 15 11:02:28.277129 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.275168Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885077793,"retry-timeout":"500ms"} Jul 15 11:02:28.277129 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.275599Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.333973251s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:02:28.277129 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:28.275781Z","caller":"traceutil/trace.go:171","msg":"trace[566970445] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"3.334180407s","start":"2025-07-15T11:02:24.941551Z","end":"2025-07-15T11:02:28.275732Z","steps":["trace[566970445] 'agreement among raft nodes before linearized reading' (duration: 3.333990393s)"],"step_count":1} Jul 15 11:02:28.277129 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.275968Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:24.941540Z","time spent":"3.334404162s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:02:28.672360 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.670167Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"3.731709338s","expected-duration":"1s"} Jul 15 11:02:28.673963 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:28.673455Z","caller":"traceutil/trace.go:171","msg":"trace[792193635] transaction","detail":"{read_only:false; response_revision:1746; number_of_response:1; }","duration":"3.735186953s","start":"2025-07-15T11:02:24.938162Z","end":"2025-07-15T11:02:28.673349Z","steps":["trace[792193635] 'process raft request' (duration: 3.733508555s)"],"step_count":1} Jul 15 11:02:28.674818 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.673799Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:24.938111Z","time spent":"3.735427488s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":490,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:28.753334 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:28.750838Z","caller":"traceutil/trace.go:171","msg":"trace[339955188] transaction","detail":"{read_only:false; response_revision:1747; number_of_response:1; }","duration":"3.812088922s","start":"2025-07-15T11:02:24.938700Z","end":"2025-07-15T11:02:28.750789Z","steps":["trace[339955188] 'process raft request' (duration: 3.756155916s)","trace[339955188] 'compare' (duration: 55.452063ms)"],"step_count":2} Jul 15 11:02:28.753334 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.751062Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:24.938686Z","time spent":"3.812257768s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":515,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:28.753334 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:28.751351Z","caller":"traceutil/trace.go:171","msg":"trace[1787417270] linearizableReadLoop","detail":"{readStateIndex:2133; appliedIndex:2130; }","duration":"3.809683845s","start":"2025-07-15T11:02:24.941581Z","end":"2025-07-15T11:02:28.751265Z","steps":["trace[1787417270] 'read index received' (duration: 3.730289s)","trace[1787417270] 'applied index is now lower than readState.Index' (duration: 79.393222ms)"],"step_count":2} Jul 15 11:02:28.753334 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:28.751573Z","caller":"traceutil/trace.go:171","msg":"trace[426486931] transaction","detail":"{read_only:false; response_revision:1748; number_of_response:1; }","duration":"3.810962613s","start":"2025-07-15T11:02:24.940592Z","end":"2025-07-15T11:02:28.751554Z","steps":["trace[426486931] 'process raft request' (duration: 3.810044876s)"],"step_count":1} Jul 15 11:02:28.755009 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.751702Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:24.940579Z","time spent":"3.811030457s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":625,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:28.755009 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.752006Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.800955717s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 11:02:28.755009 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:28.752062Z","caller":"traceutil/trace.go:171","msg":"trace[2128576192] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1748; }","duration":"3.801058204s","start":"2025-07-15T11:02:24.950988Z","end":"2025-07-15T11:02:28.752047Z","steps":["trace[2128576192] 'agreement among raft nodes before linearized reading' (duration: 3.800805875s)"],"step_count":1} Jul 15 11:02:28.755009 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.752112Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:24.950958Z","time spent":"3.801134111s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 11:02:28.756256 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.752376Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"477.160867ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f\" ","response":"range_response_count:1 size:996"} Jul 15 11:02:28.756256 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.753083Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"478.075156ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/tigerastatuses/\" range_end:\"/registry/operator.tigera.io/tigerastatuses0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:28.756256 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:28.753749Z","caller":"traceutil/trace.go:171","msg":"trace[544309126] range","detail":"{range_begin:/registry/operator.tigera.io/tigerastatuses/; range_end:/registry/operator.tigera.io/tigerastatuses0; response_count:0; response_revision:1748; }","duration":"478.999475ms","start":"2025-07-15T11:02:28.274722Z","end":"2025-07-15T11:02:28.753721Z","steps":["trace[544309126] 'agreement among raft nodes before linearized reading' (duration: 478.176551ms)"],"step_count":1} Jul 15 11:02:28.756256 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.754148Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:28.274666Z","time spent":"479.285415ms","remote":"172.24.4.149:54858","response type":"/etcdserverpb.KV/Range","request count":0,"request size":94,"response count":5,"response size":31,"request content":"key:\"/registry/operator.tigera.io/tigerastatuses/\" range_end:\"/registry/operator.tigera.io/tigerastatuses0\" count_only:true "} Jul 15 11:02:28.758019 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:28.753113Z","caller":"traceutil/trace.go:171","msg":"trace[575452812] range","detail":"{range_begin:/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f; range_end:; response_count:1; response_revision:1748; }","duration":"477.334031ms","start":"2025-07-15T11:02:28.275155Z","end":"2025-07-15T11:02:28.752489Z","steps":["trace[575452812] 'agreement among raft nodes before linearized reading' (duration: 477.086632ms)"],"step_count":1} Jul 15 11:02:28.758019 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.755141Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:28.275145Z","time spent":"479.927089ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":97,"response count":1,"response size":1020,"request content":"key:\"/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f\" "} Jul 15 11:02:28.758019 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.753372Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"478.350527ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/\" range_end:\"/registry/secrets0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:28.758019 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:28.756543Z","caller":"traceutil/trace.go:171","msg":"trace[301256110] range","detail":"{range_begin:/registry/secrets/; range_end:/registry/secrets0; response_count:0; response_revision:1748; }","duration":"481.54136ms","start":"2025-07-15T11:02:28.274973Z","end":"2025-07-15T11:02:28.756515Z","steps":["trace[301256110] 'agreement among raft nodes before linearized reading' (duration: 478.344255ms)"],"step_count":1} Jul 15 11:02:28.758603 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.756779Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:28.274961Z","time spent":"481.781556ms","remote":"172.24.4.149:33122","response type":"/etcdserverpb.KV/Range","request count":0,"request size":42,"response count":15,"response size":31,"request content":"key:\"/registry/secrets/\" range_end:\"/registry/secrets0\" count_only:true "} Jul 15 11:02:28.759690 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.758125Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.782069568s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/felixconfigurations.crd.projectcalico.org\" ","response":"range_response_count:1 size:55540"} Jul 15 11:02:28.759690 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:28.758236Z","caller":"traceutil/trace.go:171","msg":"trace[477738092] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/felixconfigurations.crd.projectcalico.org; range_end:; response_count:1; response_revision:1748; }","duration":"3.782223849s","start":"2025-07-15T11:02:24.975995Z","end":"2025-07-15T11:02:28.758219Z","steps":["trace[477738092] 'agreement among raft nodes before linearized reading' (duration: 3.777202812s)"],"step_count":1} Jul 15 11:02:28.759690 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:28.758325Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:24.975970Z","time spent":"3.782337503s","remote":"172.24.4.149:33078","response type":"/etcdserverpb.KV/Range","request count":0,"request size":100,"response count":1,"response size":55564,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/felixconfigurations.crd.projectcalico.org\" "} Jul 15 11:02:29.667612 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:29.667081Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885077798,"retry-timeout":"500ms"} Jul 15 11:02:30.033192 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:30.032309Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"1.253545441s","expected-duration":"1s"} Jul 15 11:02:32.141493 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.140014Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.118140535s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:02:32.141493 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:32.140171Z","caller":"traceutil/trace.go:171","msg":"trace[1733796780] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.11836882s","start":"2025-07-15T11:02:30.021773Z","end":"2025-07-15T11:02:32.140142Z","steps":["trace[1733796780] 'agreement among raft nodes before linearized reading' (duration: 2.118180358s)"],"step_count":1} Jul 15 11:02:32.141493 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.140255Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:30.021765Z","time spent":"2.118468131s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:02:32.628642 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.625516Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.592798277s","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure: >>","response":"size:16"} Jul 15 11:02:32.628642 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:32.625867Z","caller":"traceutil/trace.go:171","msg":"trace[1392238625] transaction","detail":"{read_only:false; response_revision:1750; number_of_response:1; }","duration":"3.84483979s","start":"2025-07-15T11:02:28.780979Z","end":"2025-07-15T11:02:32.625819Z","steps":["trace[1392238625] 'process raft request' (duration: 3.844655905s)"],"step_count":1} Jul 15 11:02:32.628642 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:32.625933Z","caller":"traceutil/trace.go:171","msg":"trace[24794593] transaction","detail":"{read_only:false; response_revision:1754; number_of_response:1; }","duration":"293.190871ms","start":"2025-07-15T11:02:32.332723Z","end":"2025-07-15T11:02:32.625914Z","steps":["trace[24794593] 'process raft request' (duration: 293.144917ms)"],"step_count":1} Jul 15 11:02:32.629574 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.626176Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:28.780953Z","time spent":"3.845018334s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":116,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:32.629574 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:32.626463Z","caller":"traceutil/trace.go:171","msg":"trace[706857364] transaction","detail":"{read_only:false; response_revision:1749; number_of_response:1; }","duration":"3.84779674s","start":"2025-07-15T11:02:28.778567Z","end":"2025-07-15T11:02:32.626364Z","steps":["trace[706857364] 'process raft request' (duration: 1.254009343s)","trace[706857364] 'compare' (duration: 2.592562399s)"],"step_count":2} Jul 15 11:02:32.629574 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.626579Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:28.778538Z","time spent":"3.847977368s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":981,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:32.629903 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:32.626656Z","caller":"traceutil/trace.go:171","msg":"trace[2062303058] transaction","detail":"{read_only:false; response_revision:1751; number_of_response:1; }","duration":"2.605866898s","start":"2025-07-15T11:02:30.020764Z","end":"2025-07-15T11:02:32.626631Z","steps":["trace[2062303058] 'process raft request' (duration: 2.604953427s)"],"step_count":1} Jul 15 11:02:32.629903 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.626750Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:30.020684Z","time spent":"2.606006813s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":699,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:32.629903 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:32.626801Z","caller":"traceutil/trace.go:171","msg":"trace[766389234] linearizableReadLoop","detail":"{readStateIndex:2136; appliedIndex:2134; }","duration":"3.740442982s","start":"2025-07-15T11:02:28.886318Z","end":"2025-07-15T11:02:32.626761Z","steps":["trace[766389234] 'read index received' (duration: 1.14644351s)","trace[766389234] 'applied index is now lower than readState.Index' (duration: 2.593997468s)"],"step_count":2} Jul 15 11:02:32.629903 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:32.626850Z","caller":"traceutil/trace.go:171","msg":"trace[1810784345] transaction","detail":"{read_only:false; response_revision:1753; number_of_response:1; }","duration":"294.32254ms","start":"2025-07-15T11:02:32.332512Z","end":"2025-07-15T11:02:32.626835Z","steps":["trace[1810784345] 'process raft request' (duration: 293.319495ms)"],"step_count":1} Jul 15 11:02:32.630238 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:32.626934Z","caller":"traceutil/trace.go:171","msg":"trace[758975645] transaction","detail":"{read_only:false; response_revision:1752; number_of_response:1; }","duration":"2.595517111s","start":"2025-07-15T11:02:30.031402Z","end":"2025-07-15T11:02:32.626919Z","steps":["trace[758975645] 'process raft request' (duration: 2.594379242s)"],"step_count":1} Jul 15 11:02:32.630238 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.627014Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.606805534s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/csistoragecapacities/\" range_end:\"/registry/csistoragecapacities0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:32.630238 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.627021Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:30.031359Z","time spent":"2.595609869s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":532,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:32.630238 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:32.627054Z","caller":"traceutil/trace.go:171","msg":"trace[1522188855] range","detail":"{range_begin:/registry/csistoragecapacities/; range_end:/registry/csistoragecapacities0; response_count:0; response_revision:1754; }","duration":"2.606895538s","start":"2025-07-15T11:02:30.020146Z","end":"2025-07-15T11:02:32.627042Z","steps":["trace[1522188855] 'agreement among raft nodes before linearized reading' (duration: 2.606808661s)"],"step_count":1} Jul 15 11:02:32.630567 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.627090Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:30.020121Z","time spent":"2.606956289s","remote":"172.24.4.149:33422","response type":"/etcdserverpb.KV/Range","request count":0,"request size":68,"response count":0,"response size":29,"request content":"key:\"/registry/csistoragecapacities/\" range_end:\"/registry/csistoragecapacities0\" count_only:true "} Jul 15 11:02:32.630567 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.627236Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"297.017881ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:02:32.630567 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:32.627271Z","caller":"traceutil/trace.go:171","msg":"trace[784795154] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1754; }","duration":"297.1104ms","start":"2025-07-15T11:02:32.330148Z","end":"2025-07-15T11:02:32.627259Z","steps":["trace[784795154] 'agreement among raft nodes before linearized reading' (duration: 297.048898ms)"],"step_count":1} Jul 15 11:02:32.630567 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.627350Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.606353955s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/stagedkubernetesnetworkpolicies/\" range_end:\"/registry/crd.projectcalico.org/stagedkubernetesnetworkpolicies0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:32.630822 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:32.627417Z","caller":"traceutil/trace.go:171","msg":"trace[1908253141] range","detail":"{range_begin:/registry/crd.projectcalico.org/stagedkubernetesnetworkpolicies/; range_end:/registry/crd.projectcalico.org/stagedkubernetesnetworkpolicies0; response_count:0; response_revision:1754; }","duration":"2.606414655s","start":"2025-07-15T11:02:30.020961Z","end":"2025-07-15T11:02:32.627376Z","steps":["trace[1908253141] 'agreement among raft nodes before linearized reading' (duration: 2.606362901s)"],"step_count":1} Jul 15 11:02:32.630822 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.627455Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:30.020951Z","time spent":"2.606489961s","remote":"172.24.4.149:54568","response type":"/etcdserverpb.KV/Range","request count":0,"request size":134,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/stagedkubernetesnetworkpolicies/\" range_end:\"/registry/crd.projectcalico.org/stagedkubernetesnetworkpolicies0\" count_only:true "} Jul 15 11:02:32.630822 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.628256Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"293.945213ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"range_response_count:1 size:886"} Jul 15 11:02:32.631286 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.629042Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.742717469s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/globalnetworkpolicies.crd.projectcalico.org\" ","response":"range_response_count:1 size:35079"} Jul 15 11:02:32.631286 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:32.630467Z","caller":"traceutil/trace.go:171","msg":"trace[173413758] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/globalnetworkpolicies.crd.projectcalico.org; range_end:; response_count:1; response_revision:1754; }","duration":"3.744180779s","start":"2025-07-15T11:02:28.886233Z","end":"2025-07-15T11:02:32.630414Z","steps":["trace[173413758] 'agreement among raft nodes before linearized reading' (duration: 3.742222929s)"],"step_count":1} Jul 15 11:02:32.631286 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:32.630865Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:28.886203Z","time spent":"3.74463234s","remote":"172.24.4.149:33078","response type":"/etcdserverpb.KV/Range","request count":0,"request size":102,"response count":1,"response size":35103,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/globalnetworkpolicies.crd.projectcalico.org\" "} Jul 15 11:02:32.632250 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:32.631672Z","caller":"traceutil/trace.go:171","msg":"trace[854257301] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; response_count:1; response_revision:1754; }","duration":"295.663468ms","start":"2025-07-15T11:02:32.334288Z","end":"2025-07-15T11:02:32.629952Z","steps":["trace[854257301] 'agreement among raft nodes before linearized reading' (duration: 292.901445ms)"],"step_count":1} Jul 15 11:02:33.206520 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:33.173162Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885077812,"retry-timeout":"500ms"} Jul 15 11:02:33.488165 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:33.487097Z","caller":"traceutil/trace.go:171","msg":"trace[1268076269] linearizableReadLoop","detail":"{readStateIndex:2141; appliedIndex:2140; }","duration":"821.814617ms","start":"2025-07-15T11:02:32.665235Z","end":"2025-07-15T11:02:33.487049Z","steps":["trace[1268076269] 'read index received' (duration: 729.977221ms)","trace[1268076269] 'applied index is now lower than readState.Index' (duration: 91.832438ms)"],"step_count":2} Jul 15 11:02:33.488165 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:33.487214Z","caller":"traceutil/trace.go:171","msg":"trace[897143834] transaction","detail":"{read_only:false; number_of_response:0; response_revision:1754; }","duration":"827.828095ms","start":"2025-07-15T11:02:32.659330Z","end":"2025-07-15T11:02:33.487158Z","steps":["trace[897143834] 'process raft request' (duration: 736.112822ms)","trace[897143834] 'compare' (duration: 91.523454ms)"],"step_count":2} Jul 15 11:02:33.488165 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:33.487362Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"822.132395ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/ipamblocks.crd.projectcalico.org\" ","response":"range_response_count:1 size:5469"} Jul 15 11:02:33.488165 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:33.487417Z","caller":"traceutil/trace.go:171","msg":"trace[1426449985] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/ipamblocks.crd.projectcalico.org; range_end:; response_count:1; response_revision:1754; }","duration":"822.327169ms","start":"2025-07-15T11:02:32.665080Z","end":"2025-07-15T11:02:33.487407Z","steps":["trace[1426449985] 'agreement among raft nodes before linearized reading' (duration: 822.114562ms)"],"step_count":1} Jul 15 11:02:33.488827 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:33.487450Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:32.665052Z","time spent":"822.382932ms","remote":"172.24.4.149:33078","response type":"/etcdserverpb.KV/Range","request count":0,"request size":91,"response count":1,"response size":5493,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/ipamblocks.crd.projectcalico.org\" "} Jul 15 11:02:33.488827 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:33.487470Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:32.659304Z","time spent":"827.991542ms","remote":"172.24.4.149:54498","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":29,"request content":"compare: success:> failure:<>"} Jul 15 11:02:37.966583 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:37.300412Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885077837,"retry-timeout":"500ms"} Jul 15 11:02:37.966583 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:37.801198Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885077837,"retry-timeout":"500ms"} Jul 15 11:02:38.217648 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.212589Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.249312871s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:02:38.217648 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:38.212832Z","caller":"traceutil/trace.go:171","msg":"trace[1042009648] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1756; }","duration":"2.249687675s","start":"2025-07-15T11:02:35.963069Z","end":"2025-07-15T11:02:38.212757Z","steps":["trace[1042009648] 'range keys from in-memory index tree' (duration: 2.249149895s)"],"step_count":1} Jul 15 11:02:38.217648 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.212920Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:35.963048Z","time spent":"2.24984415s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":29,"request content":"key:\"/registry/health\" "} Jul 15 11:02:38.217648 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.213182Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.919601605s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/globalnetworksets/\" range_end:\"/registry/crd.projectcalico.org/globalnetworksets0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:38.217648 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:38.213279Z","caller":"traceutil/trace.go:171","msg":"trace[1962953237] range","detail":"{range_begin:/registry/crd.projectcalico.org/globalnetworksets/; range_end:/registry/crd.projectcalico.org/globalnetworksets0; response_count:0; response_revision:1756; }","duration":"2.919728276s","start":"2025-07-15T11:02:35.293487Z","end":"2025-07-15T11:02:38.213215Z","steps":["trace[1962953237] 'count revisions from in-memory index tree' (duration: 2.919141549s)"],"step_count":1} Jul 15 11:02:38.220200 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.213421Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:35.293466Z","time spent":"2.919855459s","remote":"172.24.4.149:54536","response type":"/etcdserverpb.KV/Range","request count":0,"request size":106,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/globalnetworksets/\" range_end:\"/registry/crd.projectcalico.org/globalnetworksets0\" count_only:true "} Jul 15 11:02:38.220200 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.213630Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.292278499s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/storageclasses/\" range_end:\"/registry/storageclasses0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:38.220200 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:38.213693Z","caller":"traceutil/trace.go:171","msg":"trace[737124592] range","detail":"{range_begin:/registry/storageclasses/; range_end:/registry/storageclasses0; response_count:0; response_revision:1756; }","duration":"2.292389512s","start":"2025-07-15T11:02:35.921264Z","end":"2025-07-15T11:02:38.213654Z","steps":["trace[737124592] 'count revisions from in-memory index tree' (duration: 2.292130461s)"],"step_count":1} Jul 15 11:02:38.220200 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.213757Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:35.921187Z","time spent":"2.292538163s","remote":"172.24.4.149:33390","response type":"/etcdserverpb.KV/Range","request count":0,"request size":56,"response count":0,"response size":29,"request content":"key:\"/registry/storageclasses/\" range_end:\"/registry/storageclasses0\" count_only:true "} Jul 15 11:02:38.221095 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.213791Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.960901498s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/apiservers.operator.tigera.io\" ","response":"range_response_count:1 size:61144"} Jul 15 11:02:38.221095 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:38.214021Z","caller":"traceutil/trace.go:171","msg":"trace[34379852] range","detail":"{range_begin:/registry/apiextensions.k8s.io/customresourcedefinitions/apiservers.operator.tigera.io; range_end:; response_count:1; response_revision:1756; }","duration":"2.961137698s","start":"2025-07-15T11:02:35.252744Z","end":"2025-07-15T11:02:38.213881Z","steps":["trace[34379852] 'range keys from in-memory index tree' (duration: 2.959588625s)"],"step_count":1} Jul 15 11:02:38.221095 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.214262Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"3.076702768s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 11:02:38.221095 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.214257Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:35.252718Z","time spent":"2.96148342s","remote":"172.24.4.149:33078","response type":"/etcdserverpb.KV/Range","request count":0,"request size":88,"response count":1,"response size":61168,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/apiservers.operator.tigera.io\" "} Jul 15 11:02:38.221095 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:38.214314Z","caller":"traceutil/trace.go:171","msg":"trace[438237849] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1756; }","duration":"3.076792341s","start":"2025-07-15T11:02:35.137500Z","end":"2025-07-15T11:02:38.214292Z","steps":["trace[438237849] 'range keys from in-memory index tree' (duration: 3.076313038s)"],"step_count":1} Jul 15 11:02:38.221521 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.214358Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:35.137470Z","time spent":"3.076875833s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 11:02:38.221521 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.215560Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"1.473373968s","expected-duration":"1s"} Jul 15 11:02:38.221521 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:38.216672Z","caller":"traceutil/trace.go:171","msg":"trace[1886590989] transaction","detail":"{read_only:false; response_revision:1757; number_of_response:1; }","duration":"1.474604812s","start":"2025-07-15T11:02:36.742046Z","end":"2025-07-15T11:02:38.216651Z","steps":["trace[1886590989] 'process raft request' (duration: 1.474138011s)"],"step_count":1} Jul 15 11:02:38.221521 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.218421Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:36.742020Z","time spent":"1.474902444s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":516,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:38.243200 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:38.242506Z","caller":"traceutil/trace.go:171","msg":"trace[2006745841] linearizableReadLoop","detail":"{readStateIndex:2148; appliedIndex:2147; }","duration":"1.442797733s","start":"2025-07-15T11:02:36.799677Z","end":"2025-07-15T11:02:38.242475Z","steps":["trace[2006745841] 'read index received' (duration: 1.42069973s)","trace[2006745841] 'applied index is now lower than readState.Index' (duration: 22.09651ms)"],"step_count":2} Jul 15 11:02:38.244450 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:38.243704Z","caller":"traceutil/trace.go:171","msg":"trace[249157457] transaction","detail":"{read_only:false; response_revision:1758; number_of_response:1; }","duration":"1.488135375s","start":"2025-07-15T11:02:36.755514Z","end":"2025-07-15T11:02:38.243649Z","steps":["trace[249157457] 'process raft request' (duration: 1.486660916s)"],"step_count":1} Jul 15 11:02:38.244611 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.244193Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:36.755471Z","time spent":"1.488503966s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":490,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:38.245973 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.245251Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.445556102s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowschemas/\" range_end:\"/registry/flowschemas0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:38.245973 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:38.245413Z","caller":"traceutil/trace.go:171","msg":"trace[185230549] range","detail":"{range_begin:/registry/flowschemas/; range_end:/registry/flowschemas0; response_count:0; response_revision:1758; }","duration":"1.44573008s","start":"2025-07-15T11:02:36.799622Z","end":"2025-07-15T11:02:38.245352Z","steps":["trace[185230549] 'agreement among raft nodes before linearized reading' (duration: 1.445324701s)"],"step_count":1} Jul 15 11:02:38.246925 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.245736Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:36.799612Z","time spent":"1.446008948s","remote":"172.24.4.149:33438","response type":"/etcdserverpb.KV/Range","request count":0,"request size":50,"response count":13,"response size":31,"request content":"key:\"/registry/flowschemas/\" range_end:\"/registry/flowschemas0\" count_only:true "} Jul 15 11:02:38.285369 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.283020Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"1.444631293s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/validatingwebhookconfigurations/\" range_end:\"/registry/validatingwebhookconfigurations0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:38.285369 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:38.283129Z","caller":"traceutil/trace.go:171","msg":"trace[749202092] range","detail":"{range_begin:/registry/validatingwebhookconfigurations/; range_end:/registry/validatingwebhookconfigurations0; response_count:0; response_revision:1758; }","duration":"1.444781246s","start":"2025-07-15T11:02:36.838326Z","end":"2025-07-15T11:02:38.283107Z","steps":["trace[749202092] 'agreement among raft nodes before linearized reading' (duration: 1.444577615s)"],"step_count":1} Jul 15 11:02:38.285369 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.283173Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"131.504301ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/networkpolicies/\" range_end:\"/registry/networkpolicies0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:38.285369 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:38.283247Z","caller":"traceutil/trace.go:171","msg":"trace[1999777609] range","detail":"{range_begin:/registry/networkpolicies/; range_end:/registry/networkpolicies0; response_count:0; response_revision:1758; }","duration":"131.764015ms","start":"2025-07-15T11:02:38.151464Z","end":"2025-07-15T11:02:38.283228Z","steps":["trace[1999777609] 'agreement among raft nodes before linearized reading' (duration: 131.412284ms)"],"step_count":1} Jul 15 11:02:38.285369 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.283241Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"132.23808ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:02:38.286157 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:38.283422Z","caller":"traceutil/trace.go:171","msg":"trace[2020881572] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1758; }","duration":"132.374157ms","start":"2025-07-15T11:02:38.150995Z","end":"2025-07-15T11:02:38.283369Z","steps":["trace[2020881572] 'agreement among raft nodes before linearized reading' (duration: 132.224704ms)"],"step_count":1} Jul 15 11:02:38.286157 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.283789Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"127.241669ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/ippools/\" range_end:\"/registry/crd.projectcalico.org/ippools0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:38.286157 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:38.283855Z","caller":"traceutil/trace.go:171","msg":"trace[1693241125] range","detail":"{range_begin:/registry/crd.projectcalico.org/ippools/; range_end:/registry/crd.projectcalico.org/ippools0; response_count:0; response_revision:1758; }","duration":"127.336402ms","start":"2025-07-15T11:02:38.156490Z","end":"2025-07-15T11:02:38.283827Z","steps":["trace[1693241125] 'agreement among raft nodes before linearized reading' (duration: 127.160982ms)"],"step_count":1} Jul 15 11:02:38.286157 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.283948Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"826.297981ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f\" ","response":"range_response_count:1 size:996"} Jul 15 11:02:38.288739 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.283191Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:36.838315Z","time spent":"1.444855583s","remote":"172.24.4.149:33504","response type":"/etcdserverpb.KV/Range","request count":0,"request size":90,"response count":0,"response size":29,"request content":"key:\"/registry/validatingwebhookconfigurations/\" range_end:\"/registry/validatingwebhookconfigurations0\" count_only:true "} Jul 15 11:02:38.288739 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:38.284159Z","caller":"traceutil/trace.go:171","msg":"trace[1360918759] range","detail":"{range_begin:/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f; range_end:; response_count:1; response_revision:1758; }","duration":"826.45092ms","start":"2025-07-15T11:02:37.457535Z","end":"2025-07-15T11:02:38.283985Z","steps":["trace[1360918759] 'agreement among raft nodes before linearized reading' (duration: 826.152707ms)"],"step_count":1} Jul 15 11:02:38.288739 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:38.284315Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:37.457504Z","time spent":"826.690136ms","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":97,"response count":1,"response size":1020,"request content":"key:\"/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f\" "} Jul 15 11:02:39.734678 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:39.733923Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885077860,"retry-timeout":"500ms"} Jul 15 11:02:43.648083 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:41.106575Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885077860,"retry-timeout":"500ms"} Jul 15 11:02:43.655030 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.653629Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.54802617s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:02:43.655030 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.653776Z","caller":"traceutil/trace.go:171","msg":"trace[1324567583] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.548241074s","start":"2025-07-15T11:02:41.105506Z","end":"2025-07-15T11:02:43.653747Z","steps":["trace[1324567583] 'agreement among raft nodes before linearized reading' (duration: 2.548038363s)"],"step_count":1} Jul 15 11:02:43.655030 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.653886Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:41.105499Z","time spent":"2.548363127s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:02:43.655030 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.653993Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885077860,"retry-timeout":"500ms"} Jul 15 11:02:43.782690 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.781789Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"4.974663319s","expected-duration":"1s"} Jul 15 11:02:43.784409 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.783191Z","caller":"traceutil/trace.go:171","msg":"trace[36036645] transaction","detail":"{read_only:false; response_revision:1761; number_of_response:1; }","duration":"4.976403497s","start":"2025-07-15T11:02:38.806757Z","end":"2025-07-15T11:02:43.783160Z","steps":["trace[36036645] 'process raft request' (duration: 4.976127333s)"],"step_count":1} Jul 15 11:02:43.784409 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.783874Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:38.806687Z","time spent":"4.976639097s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":625,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:43.931346 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.928917Z","caller":"traceutil/trace.go:171","msg":"trace[85413727] transaction","detail":"{read_only:false; response_revision:1762; number_of_response:1; }","duration":"2.823675762s","start":"2025-07-15T11:02:41.105143Z","end":"2025-07-15T11:02:43.928819Z","steps":["trace[85413727] 'process raft request' (duration: 2.804942625s)","trace[85413727] 'compare' (duration: 18.189323ms)"],"step_count":2} Jul 15 11:02:43.931346 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.929159Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:41.105114Z","time spent":"2.823930196s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":489,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:43.931346 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.929655Z","caller":"traceutil/trace.go:171","msg":"trace[1112638775] transaction","detail":"{read_only:false; response_revision:1763; number_of_response:1; }","duration":"2.822868309s","start":"2025-07-15T11:02:41.106697Z","end":"2025-07-15T11:02:43.929565Z","steps":["trace[1112638775] 'process raft request' (duration: 2.822010752s)"],"step_count":1} Jul 15 11:02:43.934905 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.929920Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:41.106688Z","time spent":"2.823142099s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":515,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:43.934905 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.930148Z","caller":"traceutil/trace.go:171","msg":"trace[78303141] transaction","detail":"{read_only:false; response_revision:1764; number_of_response:1; }","duration":"260.125377ms","start":"2025-07-15T11:02:43.670003Z","end":"2025-07-15T11:02:43.930129Z","steps":["trace[78303141] 'process raft request' (duration: 259.4206ms)"],"step_count":1} Jul 15 11:02:43.934905 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.931529Z","caller":"traceutil/trace.go:171","msg":"trace[1880148898] linearizableReadLoop","detail":"{readStateIndex:2155; appliedIndex:2152; }","duration":"5.101114191s","start":"2025-07-15T11:02:38.830349Z","end":"2025-07-15T11:02:43.931463Z","steps":["trace[1880148898] 'read index received' (duration: 4.952164088s)","trace[1880148898] 'applied index is now lower than readState.Index' (duration: 148.94844ms)"],"step_count":2} Jul 15 11:02:43.934905 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.931698Z","caller":"traceutil/trace.go:171","msg":"trace[152041099] transaction","detail":"{read_only:false; response_revision:1765; number_of_response:1; }","duration":"260.135977ms","start":"2025-07-15T11:02:43.671542Z","end":"2025-07-15T11:02:43.931678Z","steps":["trace[152041099] 'process raft request' (duration: 258.398283ms)"],"step_count":1} Jul 15 11:02:43.935296 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.932351Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.101991462s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/tigera-operator/typha-certs\" ","response":"range_response_count:1 size:4586"} Jul 15 11:02:43.935296 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.932480Z","caller":"traceutil/trace.go:171","msg":"trace[1654868790] range","detail":"{range_begin:/registry/secrets/tigera-operator/typha-certs; range_end:; response_count:1; response_revision:1765; }","duration":"5.102121221s","start":"2025-07-15T11:02:38.830306Z","end":"2025-07-15T11:02:43.932427Z","steps":["trace[1654868790] 'agreement among raft nodes before linearized reading' (duration: 5.10149716s)"],"step_count":1} Jul 15 11:02:43.935296 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.932535Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:38.830285Z","time spent":"5.10223078s","remote":"172.24.4.149:33122","response type":"/etcdserverpb.KV/Range","request count":0,"request size":47,"response count":1,"response size":4610,"request content":"key:\"/registry/secrets/tigera-operator/typha-certs\" "} Jul 15 11:02:43.935296 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.932899Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.827567463s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/endpoints/calico-system/calico-typha\" ","response":"range_response_count:1 size:688"} Jul 15 11:02:43.935296 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.932954Z","caller":"traceutil/trace.go:171","msg":"trace[1882331142] range","detail":"{range_begin:/registry/services/endpoints/calico-system/calico-typha; range_end:; response_count:1; response_revision:1765; }","duration":"2.827622283s","start":"2025-07-15T11:02:41.105318Z","end":"2025-07-15T11:02:43.932940Z","steps":["trace[1882331142] 'agreement among raft nodes before linearized reading' (duration: 2.827438307s)"],"step_count":1} Jul 15 11:02:43.935885 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.933033Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:41.105257Z","time spent":"2.827747641s","remote":"172.24.4.149:33194","response type":"/etcdserverpb.KV/Range","request count":0,"request size":57,"response count":1,"response size":712,"request content":"key:\"/registry/services/endpoints/calico-system/calico-typha\" "} Jul 15 11:02:43.935885 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.933208Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"232.42669ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/clusterinformations/default\" ","response":"range_response_count:1 size:886"} Jul 15 11:02:43.935885 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.933289Z","caller":"traceutil/trace.go:171","msg":"trace[1168584180] range","detail":"{range_begin:/registry/crd.projectcalico.org/clusterinformations/default; range_end:; response_count:1; response_revision:1765; }","duration":"232.498371ms","start":"2025-07-15T11:02:43.700738Z","end":"2025-07-15T11:02:43.933237Z","steps":["trace[1168584180] 'agreement among raft nodes before linearized reading' (duration: 232.369595ms)"],"step_count":1} Jul 15 11:02:43.935885 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.934589Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"285.018689ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/operator.tigera.io/imagesets/\" range_end:\"/registry/operator.tigera.io/imagesets0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:43.936937 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.935065Z","caller":"traceutil/trace.go:171","msg":"trace[1091199508] range","detail":"{range_begin:/registry/operator.tigera.io/imagesets/; range_end:/registry/operator.tigera.io/imagesets0; response_count:0; response_revision:1765; }","duration":"285.465024ms","start":"2025-07-15T11:02:43.649517Z","end":"2025-07-15T11:02:43.934982Z","steps":["trace[1091199508] 'agreement among raft nodes before linearized reading' (duration: 285.011065ms)"],"step_count":1} Jul 15 11:02:43.936937 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.935968Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"241.931705ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/172.24.4.29\" ","response":"range_response_count:1 size:4112"} Jul 15 11:02:43.937927 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.936347Z","caller":"traceutil/trace.go:171","msg":"trace[340895736] range","detail":"{range_begin:/registry/minions/172.24.4.29; range_end:; response_count:1; response_revision:1765; }","duration":"242.373803ms","start":"2025-07-15T11:02:43.693952Z","end":"2025-07-15T11:02:43.936325Z","steps":["trace[340895736] 'agreement among raft nodes before linearized reading' (duration: 241.70942ms)"],"step_count":1} Jul 15 11:02:43.937927 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.936654Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"287.301139ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/horizontalpodautoscalers/\" range_end:\"/registry/horizontalpodautoscalers0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:43.937927 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.936795Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"275.718908ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/jobs/\" range_end:\"/registry/jobs0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:43.937927 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.936846Z","caller":"traceutil/trace.go:171","msg":"trace[205299649] range","detail":"{range_begin:/registry/jobs/; range_end:/registry/jobs0; response_count:0; response_revision:1765; }","duration":"275.85706ms","start":"2025-07-15T11:02:43.660975Z","end":"2025-07-15T11:02:43.936832Z","steps":["trace[205299649] 'agreement among raft nodes before linearized reading' (duration: 275.776332ms)"],"step_count":1} Jul 15 11:02:43.937927 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.937006Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"276.117946ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:02:43.940568 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.937043Z","caller":"traceutil/trace.go:171","msg":"trace[1386412757] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1765; }","duration":"276.192003ms","start":"2025-07-15T11:02:43.660839Z","end":"2025-07-15T11:02:43.937031Z","steps":["trace[1386412757] 'agreement among raft nodes before linearized reading' (duration: 276.141912ms)"],"step_count":1} Jul 15 11:02:43.940568 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.937171Z","caller":"traceutil/trace.go:171","msg":"trace[1293372296] range","detail":"{range_begin:/registry/horizontalpodautoscalers/; range_end:/registry/horizontalpodautoscalers0; response_count:0; response_revision:1765; }","duration":"287.456292ms","start":"2025-07-15T11:02:43.649243Z","end":"2025-07-15T11:02:43.936699Z","steps":["trace[1293372296] 'agreement among raft nodes before linearized reading' (duration: 287.310546ms)"],"step_count":1} Jul 15 11:02:43.940568 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.938061Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.831465926s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/mutatingwebhookconfigurations/\" range_end:\"/registry/mutatingwebhookconfigurations0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:43.940568 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.938222Z","caller":"traceutil/trace.go:171","msg":"trace[1988371309] range","detail":"{range_begin:/registry/mutatingwebhookconfigurations/; range_end:/registry/mutatingwebhookconfigurations0; response_count:0; response_revision:1765; }","duration":"2.83164926s","start":"2025-07-15T11:02:41.106553Z","end":"2025-07-15T11:02:43.938202Z","steps":["trace[1988371309] 'agreement among raft nodes before linearized reading' (duration: 2.830791053s)"],"step_count":1} Jul 15 11:02:43.941720 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.937756Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.832282607s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/globalnetworkpolicies/\" range_end:\"/registry/crd.projectcalico.org/globalnetworkpolicies0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:43.941720 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.939099Z","caller":"traceutil/trace.go:171","msg":"trace[1120733851] range","detail":"{range_begin:/registry/crd.projectcalico.org/globalnetworkpolicies/; range_end:/registry/crd.projectcalico.org/globalnetworkpolicies0; response_count:0; response_revision:1765; }","duration":"2.833403424s","start":"2025-07-15T11:02:41.105432Z","end":"2025-07-15T11:02:43.938835Z","steps":["trace[1120733851] 'agreement among raft nodes before linearized reading' (duration: 2.832106395s)"],"step_count":1} Jul 15 11:02:43.941720 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.939570Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:41.106544Z","time spent":"2.832147932s","remote":"172.24.4.149:33520","response type":"/etcdserverpb.KV/Range","request count":0,"request size":86,"response count":0,"response size":29,"request content":"key:\"/registry/mutatingwebhookconfigurations/\" range_end:\"/registry/mutatingwebhookconfigurations0\" count_only:true "} Jul 15 11:02:43.941720 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.940016Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:41.105376Z","time spent":"2.834154878s","remote":"172.24.4.149:54512","response type":"/etcdserverpb.KV/Range","request count":0,"request size":114,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/globalnetworkpolicies/\" range_end:\"/registry/crd.projectcalico.org/globalnetworkpolicies0\" count_only:true "} Jul 15 11:02:43.942107 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.940610Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.833674661s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/certificatesigningrequests/\" range_end:\"/registry/certificatesigningrequests0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:43.942107 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:43.940681Z","caller":"traceutil/trace.go:171","msg":"trace[1096692274] range","detail":"{range_begin:/registry/certificatesigningrequests/; range_end:/registry/certificatesigningrequests0; response_count:0; response_revision:1765; }","duration":"2.833775938s","start":"2025-07-15T11:02:41.106888Z","end":"2025-07-15T11:02:43.940664Z","steps":["trace[1096692274] 'agreement among raft nodes before linearized reading' (duration: 2.833570362s)"],"step_count":1} Jul 15 11:02:43.942107 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:43.940745Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:41.106877Z","time spent":"2.833835466s","remote":"172.24.4.149:33282","response type":"/etcdserverpb.KV/Range","request count":0,"request size":80,"response count":2,"response size":31,"request content":"key:\"/registry/certificatesigningrequests/\" range_end:\"/registry/certificatesigningrequests0\" count_only:true "} Jul 15 11:02:44.578246 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:44.574248Z","caller":"etcdserver/v3_server.go:920","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":7587888131885077870,"retry-timeout":"500ms"} Jul 15 11:02:49.308443 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:49.307898Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:43.975038Z","time spent":"5.332831873s","remote":"172.24.4.149:54498","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:02:50.408667 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:50.406697Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"5.832072795s","expected-duration":"1s"} Jul 15 11:02:54.975994 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:51.339871Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.000460513s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:02:54.975994 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:51.340029Z","caller":"traceutil/trace.go:171","msg":"trace[1846258787] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.000724717s","start":"2025-07-15T11:02:49.339253Z","end":"2025-07-15T11:02:51.339977Z","steps":["trace[1846258787] 'agreement among raft nodes before linearized reading' (duration: 2.000482223s)"],"step_count":1} Jul 15 11:02:54.975994 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:51.340181Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:49.339234Z","time spent":"2.000869142s","remote":"172.24.4.149:33030","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:02:54.975994 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:51.357100Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.00086278s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"} Jul 15 11:02:54.975994 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:51.357443Z","caller":"traceutil/trace.go:171","msg":"trace[2015671757] range","detail":"{range_begin:/registry/health; range_end:; }","duration":"2.001281557s","start":"2025-07-15T11:02:49.356143Z","end":"2025-07-15T11:02:51.357424Z","steps":["trace[2015671757] 'agreement among raft nodes before linearized reading' (duration: 2.000931236s)"],"step_count":1} Jul 15 11:02:54.980871 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:51.357619Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:49.356116Z","time spent":"2.001465426s","remote":"172.24.4.149:33034","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "} Jul 15 11:02:54.980871 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:51.574942Z","caller":"etcdserver/server.go:1198","msg":"failed to revoke lease","lease-id":"694d980db8699527","error":"etcdserver: request timed out"} Jul 15 11:02:54.980871 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:54.292512Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:49.300039Z","time spent":"4.992459629s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":0,"request size":0,"response count":0,"response size":0,"request content":""} Jul 15 11:02:55.168471 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.167634Z","caller":"wal/wal.go:805","msg":"slow fdatasync","took":"4.760436533s","expected-duration":"1s"} Jul 15 11:02:55.185855 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.184859Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"10.610094997s","expected-duration":"100ms","prefix":"","request":"header: txn: success:> failure:<>>","response":"size:5"} Jul 15 11:02:55.220940 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.216789Z","caller":"traceutil/trace.go:171","msg":"trace[961589753] linearizableReadLoop","detail":"{readStateIndex:2159; appliedIndex:2157; }","duration":"11.237997093s","start":"2025-07-15T11:02:43.978639Z","end":"2025-07-15T11:02:55.216636Z","steps":["trace[961589753] 'read index received' (duration: 596.460824ms)","trace[961589753] 'applied index is now lower than readState.Index' (duration: 10.641524869s)"],"step_count":2} Jul 15 11:02:55.220940 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.217485Z","caller":"traceutil/trace.go:171","msg":"trace[1238559980] transaction","detail":"{read_only:false; response_revision:1769; number_of_response:1; }","duration":"237.654257ms","start":"2025-07-15T11:02:54.979797Z","end":"2025-07-15T11:02:55.217451Z","steps":["trace[1238559980] 'process raft request' (duration: 237.498942ms)"],"step_count":1} Jul 15 11:02:55.220940 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.217813Z","caller":"traceutil/trace.go:171","msg":"trace[1725837731] transaction","detail":"{read_only:false; response_revision:1767; number_of_response:1; }","duration":"4.851290907s","start":"2025-07-15T11:02:50.366510Z","end":"2025-07-15T11:02:55.217801Z","steps":["trace[1725837731] 'process raft request' (duration: 4.850504955s)"],"step_count":1} Jul 15 11:02:55.220940 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.217980Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:50.366313Z","time spent":"4.851536037s","remote":"172.24.4.149:33296","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":516,"response count":0,"response size":40,"request content":"compare: success:> failure: >"} Jul 15 11:02:55.221752 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.218078Z","caller":"traceutil/trace.go:171","msg":"trace[1947701890] transaction","detail":"{read_only:false; response_revision:1768; number_of_response:1; }","duration":"240.213252ms","start":"2025-07-15T11:02:54.977854Z","end":"2025-07-15T11:02:55.218067Z","steps":["trace[1947701890] 'process raft request' (duration: 239.385924ms)"],"step_count":1} Jul 15 11:02:55.221752 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.219230Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"10.641841467s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/stagedglobalnetworkpolicies/\" range_end:\"/registry/crd.projectcalico.org/stagedglobalnetworkpolicies0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:55.221752 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.219319Z","caller":"traceutil/trace.go:171","msg":"trace[66517988] range","detail":"{range_begin:/registry/crd.projectcalico.org/stagedglobalnetworkpolicies/; range_end:/registry/crd.projectcalico.org/stagedglobalnetworkpolicies0; response_count:0; response_revision:1770; }","duration":"10.642010697s","start":"2025-07-15T11:02:44.577283Z","end":"2025-07-15T11:02:55.219293Z","steps":["trace[66517988] 'agreement among raft nodes before linearized reading' (duration: 10.641583102s)"],"step_count":1} Jul 15 11:02:55.221752 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.219307Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"11.240605611s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/secrets/tigera-operator/typha-certs-noncluster-host\" ","response":"range_response_count:1 size:4662"} Jul 15 11:02:55.222105 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.219440Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:44.577259Z","time spent":"10.642092587s","remote":"172.24.4.149:54524","response type":"/etcdserverpb.KV/Range","request count":0,"request size":126,"response count":0,"response size":29,"request content":"key:\"/registry/crd.projectcalico.org/stagedglobalnetworkpolicies/\" range_end:\"/registry/crd.projectcalico.org/stagedglobalnetworkpolicies0\" count_only:true "} Jul 15 11:02:55.222105 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.219693Z","caller":"traceutil/trace.go:171","msg":"trace[775425993] range","detail":"{range_begin:/registry/secrets/tigera-operator/typha-certs-noncluster-host; range_end:; response_count:1; response_revision:1769; }","duration":"11.24076358s","start":"2025-07-15T11:02:43.978601Z","end":"2025-07-15T11:02:55.219364Z","steps":["trace[775425993] 'agreement among raft nodes before linearized reading' (duration: 11.238743953s)"],"step_count":1} Jul 15 11:02:55.222105 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.219989Z","caller":"traceutil/trace.go:171","msg":"trace[479975476] transaction","detail":"{read_only:false; response_revision:1770; number_of_response:1; }","duration":"240.079118ms","start":"2025-07-15T11:02:54.979894Z","end":"2025-07-15T11:02:55.219973Z","steps":["trace[479975476] 'process raft request' (duration: 237.453559ms)"],"step_count":1} Jul 15 11:02:55.222105 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.219968Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:43.978582Z","time spent":"11.241301618s","remote":"172.24.4.149:33122","response type":"/etcdserverpb.KV/Range","request count":0,"request size":63,"response count":1,"response size":4686,"request content":"key:\"/registry/secrets/tigera-operator/typha-certs-noncluster-host\" "} Jul 15 11:02:55.222709 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.220158Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"240.007235ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/blockaffinities/\" range_end:\"/registry/crd.projectcalico.org/blockaffinities0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:55.222709 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.220204Z","caller":"traceutil/trace.go:171","msg":"trace[761127785] range","detail":"{range_begin:/registry/crd.projectcalico.org/blockaffinities/; range_end:/registry/crd.projectcalico.org/blockaffinities0; response_count:0; response_revision:1770; }","duration":"240.116827ms","start":"2025-07-15T11:02:54.980076Z","end":"2025-07-15T11:02:55.220193Z","steps":["trace[761127785] 'agreement among raft nodes before linearized reading' (duration: 240.011023ms)"],"step_count":1} Jul 15 11:02:55.222709 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.220375Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"240.632453ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/leases/tigera-operator/operator-lock\" ","response":"range_response_count:1 size:505"} Jul 15 11:02:55.224761 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.223007Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"244.706719ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/runtimeclasses/\" range_end:\"/registry/runtimeclasses0\" count_only:true ","response":"range_response_count:0 size:5"} Jul 15 11:02:55.224761 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.223095Z","caller":"traceutil/trace.go:171","msg":"trace[276857449] range","detail":"{range_begin:/registry/leases/tigera-operator/operator-lock; range_end:; response_count:1; response_revision:1770; }","duration":"243.34978ms","start":"2025-07-15T11:02:54.979724Z","end":"2025-07-15T11:02:55.223073Z","steps":["trace[276857449] 'agreement among raft nodes before linearized reading' (duration: 240.613909ms)"],"step_count":1} Jul 15 11:02:55.224761 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.223156Z","caller":"traceutil/trace.go:171","msg":"trace[1501144274] range","detail":"{range_begin:/registry/runtimeclasses/; range_end:/registry/runtimeclasses0; response_count:0; response_revision:1770; }","duration":"244.886618ms","start":"2025-07-15T11:02:54.978242Z","end":"2025-07-15T11:02:55.223129Z","steps":["trace[1501144274] 'agreement among raft nodes before linearized reading' (duration: 244.700808ms)"],"step_count":1} Jul 15 11:02:55.224761 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.223556Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"4.784043591s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/clusterrolebindings/\" range_end:\"/registry/clusterrolebindings0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:55.224761 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.223588Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"246.248579ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:5"} Jul 15 11:02:55.225256 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.223628Z","caller":"traceutil/trace.go:171","msg":"trace[1392500004] range","detail":"{range_begin:/registry/clusterrolebindings/; range_end:/registry/clusterrolebindings0; response_count:0; response_revision:1770; }","duration":"4.784123318s","start":"2025-07-15T11:02:50.439490Z","end":"2025-07-15T11:02:55.223614Z","steps":["trace[1392500004] 'agreement among raft nodes before linearized reading' (duration: 4.783768256s)"],"step_count":1} Jul 15 11:02:55.225256 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.223684Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:50.439339Z","time spent":"4.784314156s","remote":"172.24.4.149:33376","response type":"/etcdserverpb.KV/Range","request count":0,"request size":66,"response count":63,"response size":31,"request content":"key:\"/registry/clusterrolebindings/\" range_end:\"/registry/clusterrolebindings0\" count_only:true "} Jul 15 11:02:55.225256 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.223713Z","caller":"traceutil/trace.go:171","msg":"trace[1627058365] range","detail":"{range_begin:/registry/health; range_end:; response_count:0; response_revision:1770; }","duration":"246.519445ms","start":"2025-07-15T11:02:54.977122Z","end":"2025-07-15T11:02:55.223641Z","steps":["trace[1627058365] 'agreement among raft nodes before linearized reading' (duration: 246.41327ms)"],"step_count":1} Jul 15 11:02:55.225256 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.223940Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.85958518s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f\" ","response":"range_response_count:1 size:996"} Jul 15 11:02:55.226360 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.223981Z","caller":"traceutil/trace.go:171","msg":"trace[1618701248] range","detail":"{range_begin:/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f; range_end:; response_count:1; response_revision:1770; }","duration":"5.859662s","start":"2025-07-15T11:02:49.364306Z","end":"2025-07-15T11:02:55.223968Z","steps":["trace[1618701248] 'agreement among raft nodes before linearized reading' (duration: 5.859511004s)"],"step_count":1} Jul 15 11:02:55.226360 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.223981Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"2.95448842s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/kubecontrollersconfigurations/\" range_end:\"/registry/crd.projectcalico.org/kubecontrollersconfigurations0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:55.226360 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.224042Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:49.364284Z","time spent":"5.859728233s","remote":"172.24.4.149:33104","response type":"/etcdserverpb.KV/Range","request count":0,"request size":97,"response count":1,"response size":1020,"request content":"key:\"/registry/events/kube-system/kube-scheduler-ci-4230-2-100-6ef5831daf.novalocal.1852679c6626b25f\" "} Jul 15 11:02:55.226360 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.224058Z","caller":"traceutil/trace.go:171","msg":"trace[2084047389] range","detail":"{range_begin:/registry/crd.projectcalico.org/kubecontrollersconfigurations/; range_end:/registry/crd.projectcalico.org/kubecontrollersconfigurations0; response_count:0; response_revision:1770; }","duration":"2.954813786s","start":"2025-07-15T11:02:52.269223Z","end":"2025-07-15T11:02:55.224037Z","steps":["trace[2084047389] 'agreement among raft nodes before linearized reading' (duration: 2.954704806s)"],"step_count":1} Jul 15 11:02:55.228256 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.224139Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:52.269188Z","time spent":"2.95491902s","remote":"172.24.4.149:54920","response type":"/etcdserverpb.KV/Range","request count":0,"request size":130,"response count":1,"response size":31,"request content":"key:\"/registry/crd.projectcalico.org/kubecontrollersconfigurations/\" range_end:\"/registry/crd.projectcalico.org/kubecontrollersconfigurations0\" count_only:true "} Jul 15 11:02:55.229576 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.224596Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.958022737s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/masterleases/172.24.4.149\" ","response":"range_response_count:1 size:131"} Jul 15 11:02:55.229576 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.226539Z","caller":"traceutil/trace.go:171","msg":"trace[2095335179] range","detail":"{range_begin:/registry/masterleases/172.24.4.149; range_end:; response_count:1; response_revision:1770; }","duration":"5.959991389s","start":"2025-07-15T11:02:49.266529Z","end":"2025-07-15T11:02:55.226520Z","steps":["trace[2095335179] 'agreement among raft nodes before linearized reading' (duration: 5.957963508s)"],"step_count":1} Jul 15 11:02:55.229576 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.226591Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:49.266514Z","time spent":"5.960056739s","remote":"172.24.4.149:33052","response type":"/etcdserverpb.KV/Range","request count":0,"request size":37,"response count":1,"response size":155,"request content":"key:\"/registry/masterleases/172.24.4.149\" "} Jul 15 11:02:55.229576 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.226751Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.96043353s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/services/endpoints/\" range_end:\"/registry/services/endpoints0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:55.229576 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.226781Z","caller":"traceutil/trace.go:171","msg":"trace[38613135] range","detail":"{range_begin:/registry/services/endpoints/; range_end:/registry/services/endpoints0; response_count:0; response_revision:1770; }","duration":"5.960513928s","start":"2025-07-15T11:02:49.266259Z","end":"2025-07-15T11:02:55.226772Z","steps":["trace[38613135] 'agreement among raft nodes before linearized reading' (duration: 5.960393066s)"],"step_count":1} Jul 15 11:02:55.230279 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.226812Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:49.266238Z","time spent":"5.960562506s","remote":"172.24.4.149:33194","response type":"/etcdserverpb.KV/Range","request count":0,"request size":64,"response count":7,"response size":31,"request content":"key:\"/registry/services/endpoints/\" range_end:\"/registry/services/endpoints0\" count_only:true "} Jul 15 11:02:55.230279 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.222801Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"243.593647ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/csidrivers/\" range_end:\"/registry/csidrivers0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:55.230279 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.226890Z","caller":"traceutil/trace.go:171","msg":"trace[2098983028] range","detail":"{range_begin:/registry/csidrivers/; range_end:/registry/csidrivers0; response_count:0; response_revision:1770; }","duration":"247.716092ms","start":"2025-07-15T11:02:54.979163Z","end":"2025-07-15T11:02:55.226879Z","steps":["trace[2098983028] 'agreement among raft nodes before linearized reading' (duration: 243.529178ms)"],"step_count":1} Jul 15 11:02:55.230279 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.222909Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"243.878309ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/crd.projectcalico.org/tiers/\" range_end:\"/registry/crd.projectcalico.org/tiers0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:55.230779 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.226983Z","caller":"traceutil/trace.go:171","msg":"trace[36100930] range","detail":"{range_begin:/registry/crd.projectcalico.org/tiers/; range_end:/registry/crd.projectcalico.org/tiers0; response_count:0; response_revision:1770; }","duration":"247.97604ms","start":"2025-07-15T11:02:54.978998Z","end":"2025-07-15T11:02:55.226974Z","steps":["trace[36100930] 'agreement among raft nodes before linearized reading' (duration: 243.868342ms)"],"step_count":1} Jul 15 11:02:55.230779 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.227346Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.960788783s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/endpointslices/\" range_end:\"/registry/endpointslices0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:55.230779 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.227434Z","caller":"traceutil/trace.go:171","msg":"trace[1987794233] range","detail":"{range_begin:/registry/endpointslices/; range_end:/registry/endpointslices0; response_count:0; response_revision:1770; }","duration":"5.960844496s","start":"2025-07-15T11:02:49.266526Z","end":"2025-07-15T11:02:55.227370Z","steps":["trace[1987794233] 'agreement among raft nodes before linearized reading' (duration: 5.960719226s)"],"step_count":1} Jul 15 11:02:55.230779 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.227610Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:49.266515Z","time spent":"5.960946482s","remote":"172.24.4.149:33310","response type":"/etcdserverpb.KV/Range","request count":0,"request size":56,"response count":7,"response size":31,"request content":"key:\"/registry/endpointslices/\" range_end:\"/registry/endpointslices0\" count_only:true "} Jul 15 11:02:55.231188 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.226323Z","caller":"etcdserver/util.go:170","msg":"apply request took too long","took":"5.959965872s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/minions/\" range_end:\"/registry/minions0\" count_only:true ","response":"range_response_count:0 size:7"} Jul 15 11:02:55.231188 etcd-wrapper[1706]: {"level":"info","ts":"2025-07-15T11:02:55.228558Z","caller":"traceutil/trace.go:171","msg":"trace[2029011027] range","detail":"{range_begin:/registry/minions/; range_end:/registry/minions0; response_count:0; response_revision:1770; }","duration":"5.962208197s","start":"2025-07-15T11:02:49.266330Z","end":"2025-07-15T11:02:55.228538Z","steps":["trace[2029011027] 'agreement among raft nodes before linearized reading' (duration: 5.959856411s)"],"step_count":1} Jul 15 11:02:55.231188 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.228612Z","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2025-07-15T11:02:49.266241Z","time spent":"5.962346831s","remote":"172.24.4.149:33196","response type":"/etcdserverpb.KV/Range","request count":0,"request size":42,"response count":2,"response size":31,"request content":"key:\"/registry/minions/\" range_end:\"/registry/minions0\" count_only:true "} Jul 15 11:02:55.234750 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.233979Z","caller":"etcdserver/util.go:123","msg":"failed to apply request","took":"183.236µs","request":"header: lease_revoke:","response":"size:29","error":"lease not found"} Jul 15 11:02:55.234750 etcd-wrapper[1706]: {"level":"warn","ts":"2025-07-15T11:02:55.234180Z","caller":"etcdserver/server.go:1198","msg":"failed to revoke lease","lease-id":"694d980db8699527","error":"lease not found"}