kernel

ARP proxy going rogue, part 2: tracing the kernel

Introduction This is a story of ARP Proxy going rogue. Writing down that story took more than I expected so it’s split in two different posts. In the first part I explained what proxy ARP is and how it’s used in GRNET Ganeti clusters to provide public IPv4 to guest vms. I referred to the incident of a certain host hijacking all IPv4 addresses within a VLAN. In this second part I track down this particular behavior by reading the linux source code, setting up a Debian Buster testbed environment with network namespaces, and playing around with python scapy, eBPF Compiler Collection toolkit and linux kernel static tracepoints.
Read more

ARP proxy going rogue, part 1: the incident

Intro This is a story of “Proxy ARP” going rogue. Writing down that story took more than I expected so it’s split in two different posts. In this first part we explain what proxy ARP is and how it’s used in GRNET Ganeti clusters to provide public IPv4 to guest vms. I’m going to investigate a particular incident where certain hosts caused DOS by hijacking all IPv4 addresses within a VLAN.
Read more

Why kernel is dropping frames?

Kernel is dropping packets At work, when we did setup prometheus alerting we started getting nofitications like this: description = ok10-01.okeanos.grnet.gr:9100 is dropping frames at a rate of 0.10340541666666667 frames per sec To output this alert, prometheus-node-exporter reads /proc/net/softnet_stat filepath, which contains a line for every CPU. The 2nd column of each line counts dropped frames in hexadecimal. A nicer output can be provided if we query prometheus directly from within the host:
Read more