Published signals

Your K8s Pod Memory Is Being Eaten by Log File Cache — Here's How to Fix It

Score: 7/10 Topic: Kubernetes container memory leak caused by log file caching

A developer found that a Spring Boot pod's memory usage (4G+) far exceeded its heap+off-heap (1G). The culprit was the Linux kernel's page cache (active_file) caching log files from a mounted host directory. This is a common but often overlooked issue in K8s environments where log volumes are mounted without memory limits or cache control.

A recent debugging case highlights a subtle but impactful memory issue in Kubernetes: the Linux kernel's page cache can inflate container memory usage far beyond the application's actual heap and off-heap memory. In this instance, a Spring Boot pod was consuming over 4GB of memory, while the JVM heap plus off-heap totaled only about 1GB. Investigation revealed that the active_file (page cache) was caching the 3GB of log files stored on a mounted host directory (/logs). This is a known behavior: when a container mounts a host directory, the kernel caches file data in the page cache, which counts toward the container's memory cgroup limit. The fix involves either limiting the page cache via memory cgroup settings, using a tmpfs mount for logs, or configuring log rotation and compression to reduce the cached data size. This case is a valuable reminder for DevOps and SRE teams to monitor not just application memory but also kernel-level caching when using volume mounts in production.