Coarse_stat_fs: Zero-syscall access to /proc/self/{stat, schedstat}

coarse_stat_fs is a Linux kernel module that exposes coarse versions of /proc/self/schedstat and /proc/stat as read-only shared memory pages. After a one-time open and mmap, a snapshot costs about as much as rdtscp and makes no syscall.

One use is profiling a hot loop and reporting only the operations that are especially slow. Take a snapshot before each operation and time the operation with rdtscp. When an operation is slow, take a second snapshot and print where the wall time went:

#include "lib/coarse_stat_fs.h"

struct csf_snap before, after; char buf[CSF_DIFF_BUFSZ]; /* fits any csf_format_diff() output */

csf_thread_snap(&before); /* ~16 ns, no syscall */ uint64_t t0 = csf_rdtsc(); op(); if (csf_rdtsc() - t0 > slow) { csf_thread_snap(&after); csf_format_diff(buf, sizeof(buf), &before, &after); puts(buf); }

Output for an operation that slept for 25 ms:

wall 25.119ms = cpu 0.017 + runq 0.000 + sleep 25.102 + dsleep 0.000 + iowait 0.000 | preempt 0 block 1 migrate 0 | flt 4/0 | sysc r0 w0 | io r0B w0B

examples/eventloop.c is a complete program, and it also reads the fields one at a time; see Reading a snapshot.

"Coarse" is about freshness. The time buckets are exact to the cycle, but the other fields are only as fresh as the thread's last context switch or the cpu page's last refresh:

For comparison, /proc/stat and the utime and stime fields of /proc/<tid>/stat count in USER_HZ ticks of 10 ms.

Getting started

Repository layout

Developing

Testing

Continuous integration

Interface

Cost

Installing

Caveats

License

Getting started

Prerequisites

An x86-64 kernel with CONFIG_PREEMPT_NOTIFIERS and CONFIG_SCHED_INFO, version 6.1 or later. CI tests 6.1, 6.12 and 7.2. Most distribution kernels qualify: KVM selects preempt notifiers, and schedstats select SCHED_INFO.

The headers for that kernel, for example linux-headers on Arch or linux-headers-amd64 on Debian.

gcc and make. Testing also needs QEMU and virtme-ng. Linting needs clang-format, clang-tidy, bear and, optionally, shellcheck.

Building and trying it

Build the module, the tools, the example and the tests:

make

Run the tests in a throwaway VM before you load the module on your machine. A bug in a scheduler hook can crash the kernel that runs it, so the VM boots your kernel and runs the module there instead:

make check

The run ends with RESULT 0 when every test passes.

Load the module and mount its filesystem. Both steps need root:

sudo insmod kmod/coarse_stat_fs.ko sudo mount -t coarse_stat_fs none /sys/kernel/coarse_stat_fs

Read the pages:

tools/csfcat cpu # the cpu page, in /proc/stat format tools/csfcat self 100 # sleep 50 ms, spin 50 ms, print the diff build/eventloop # the example: reports only the slow operations

csfcat self 100 prints a line similar to this one:

sleep 50 ms + spin 50 ms: wall 100.141ms = cpu 50.014 + runq 0.000 + sleep 50.127 + ...

Unmount and unload when you're done:

sudo umount /sys/kernel/coarse_stat_fs sudo rmmod coarse_stat_fs

Repository layout

Path Contents

include/uapi/coarse_stat_fs.h The ABI: page layouts, constants, flags. Shared by the module and user space.

kmod/ The module. main.c (init, filesystem, clock), thread.c (per-thread pages, preempt notifiers), cpu.c (cpu page), compat.h (fallbacks for older kernels).

lib/coarse_stat_fs.h Header-only user-space library (MIT): mapping, snapshots, diff formatting.

tools/ csfcat, which prints the pages as text.

examples/ eventloop.c: profiles a loop and reports only the slow operations.

tests/ Test programs (t_*.c), the benchmark, and the VM harnesses.

tests/debian/ Tests against Debian kernels, as CI runs them.

scripts/ kmod-compdb.py (compile database for the module), pre-commit (git hook).

packaging/ systemd mount unit and modules-load configuration.

dkms.conf DKMS build configuration.

LICENSE, LICENSES/ GPL-2.0, and the texts of the MIT license and the syscall note. See License.

.github/workflows/ci.yml CI: lint, then tests on Debian kernels.

.clang-format, .clang-tidy, kmod/.clang-tidy, .editorconfig Code style and static-check configuration.

Developing

Running common tasks

Every task has a make target. Run make help to list them.

To reload the module on the host after a rebuild, optionally with a different refresh period, and then check the kernel log:

sudo umount /sys/kernel/coarse_stat_fs; sudo rmmod coarse_stat_fs sudo insmod kmod/coarse_stat_fs.ko period_us=2000 sudo mount -t coarse_stat_fs none /sys/kernel/coarse_stat_fs sudo dmesg | tail -n 50

rmmod fails while any registered thread is still alive; see Caveats.

make builds the module against the running kernel's headers, /lib/modules/$(uname -r)/build, which the headers package installs. If they are missing, the build stops and tells you which package to install. To build for a different kernel, set KDIR, for example make KDIR=/usr/src/linux-headers-6.12.0-amd64.

All C code, kernel and user space, uses Linux kernel style with a 100-column limit. .clang-format encodes it; run make format before committing.

The static checks are split by where the code runs:

make lint runs all of them. To run the fast checks on every commit, install the hook:

ln -s ../../scripts/pre-commit .git/hooks/pre-commit

Setting up your editor

Run make compdb once, and again after you add files. It writes compile_commands.json for user-space code and kmod/compile_commands.json for the module. kbuild generates the module's database with the host kernel's gcc flags, so scripts/kmod-compdb.py removes the flags that clang rejects. clangd then handles both kernel and user-space files. Without a database, clangd falls back to compile_flags.txt, which covers user space only.

The module targets current kernels. kmod/compat.h provides fallbacks back to 6.1, and each fallback names the release that introduced the real interface. When a new kernel breaks the build, add the fallback there rather than at the call site, and run make debian SUITE=<suite> for the oldest and newest suites.

Running the tests on your kernel

make check (tests/vm.sh) boots the host kernel in QEMU with virtme-ng, loads the module, and runs the tests inside the guest. The guest boots with panic_on_warn, slub_debug, init_on_free and KFENCE, so any kernel warning fails the run.

tests/vm.sh # cpu thread life stress tools tests/vm.sh thread life # only these VM_DEBUG=0 tests/vm.sh bench # benchmarks without memory debugging

The log is in build/vm/tests.log. If the guest dies, tests/vm.sh prints the tail of build/vm/console.log, which contains the kernel's oops or panic. VM_TIMEOUT (seconds, default 600) limits the run.

Every run also checks that rmmod fails while an object is alive and succeeds afterwards, and that the kernel log has no warnings.

tests/debian/run.sh SUITE reproduces a CI job locally:

tests/debian/run.sh bookworm # kernel 6.1 tests/debian/run.sh trixie-backports # newest stable kernel tests/debian/run.sh sid thread life # a subset of the tests

The script needs sudo, mmdebstrap, qemu-system-x86, e2fsprogs and curl. The console log, including all test output, is in build/debian/<suite>/console.log.

On hosts without mmdebstrap, such as Arch, use tests/debian/docker.sh with the same arguments. It runs run.sh in an Ubuntu container, as on the CI runner.

Tests are plain C programs that use tests/t.h:

#include "t.h"

int main(void) { struct csf_snap a, b;

TEST("my scenario") { t_snap(&a); /* dies if the snapshot fails */ t_sleep_ns(10000000); t_snap(&b); CHECK(t_diff(&a, &b).sleep > 9, "slept too little"); } return t_done("t_mine"); }

Add the program to TESTS in tests/Makefile, then add a line for it in tests/vm-inner.sh. Keep timing assertions loose: CI runners are shared VMs, and time stolen by the host counts as the thread's current state.

.github/workflows/ci.yml runs on every push and pull request, and weekly to pick up new Debian kernels:

The jobs don't use containers. GitHub's Ubuntu runners have KVM, so each guest runs at close to native speed.

The module registers a filesystem, coarse_stat_fs, and creates its mount point /sys/kernel/coarse_stat_fs:

Both files have mode 0444. Any thread can map them without privileges, but only read-only, with MAP_SHARED, at offset 0. To find the length to map, fstat() the open file: the kernel reports it as st_size. That is one page for thread-self, and the header plus one row per possible CPU for cpu. lib/coarse_stat_fs.h does this for you, so nothing hard-codes a page size.

include/uapi/coarse_stat_fs.h defines the ABI. Every page starts with a 64-byte struct csf_hdr that holds a magic number, the ABI version, the page kind and size, flags, and the TSC frequency and cycles-to-ns conversion.

The thread page records the time since the thread first opened thread-self, split into five buckets of TSC cycles:

The buckets add up exactly to elapsed wall time. The page also records:

The kernel updates the page from preempt notifiers at every context switch of a registered thread, under a seqcount. The reader extrapolates the current bucket with rdtscp - last_switch_tsc, so a snapshot is exact to the cycle even though the kernel writes only at switches.

Each open() of thread-self returns its own file descriptor, but all of a thread's descriptors share one refcounted object: one page and one notifier. The object goes away when the thread dies or when the thread closes its last file. If another thread holds the last reference, for example after the owner handed off its mapping, closing that reference only parks the object. A parked object stops accounting, and the owner's next open resets the page and resumes. The page memory is charged to the opener's memory cgroup.

lib/coarse_stat_fs.h maps the page once per thread on first use, keeps it in thread-local storage, and unmaps it at thread exit. After fork(), the child maps its own page, because the mapping is VM_DONTCOPY.

csf_thread_snap() fills a struct csf_snap: a private copy of the thread page in s.p, the TSC value at the time of the copy in s.tsc, and s.extrapolated, which is set when the current bucket was brought up to s.tsc. Because the copy is private, you can read its fields at any later time without retrying. Subtract two snapshots field by field:

#include <stdio.h>

#include "lib/coarse_stat_fs.h"

struct csf_snap a, b;

if (csf_thread_snap(&a)) return -1; /* errno says why, for example ENOENT when not mounted */ op(); csf_thread_snap(&b);

/* Time: TSC cycles; the header holds the conversion to ns. */ const struct csf_hdr *h = &b.p.hdr; double wall_ms = csf_cyc_to_ns(h, b.tsc - a.tsc) / 1e6; double iowait_ms = csf_cyc_to_ns(h, b.p.iowait_cyc - a.p.iowait_cyc) / 1e6;

/* Counters: plain differences. */ unsigned long long blocked = b.p.nr_block - a.p.nr_block; unsigned long long major_faults = b.p.maj_flt - a.p.maj_flt;

printf("%.1f ms, %.1f ms in iowait, blocked %llu times, %llu major faults\n", wall_ms, iowait_ms, blocked, major_faults);

The table at the top lists each field's unit and freshness. The five buckets of b minus a add up to b.tsc - a.tsc when both snapshots are extrapolated, so the largest bucket tells you what an operation mostly did.

The fields copied at switch-out lag behind a thread that stays on the CPU. In one run of the example, a 30 ms spin showed 30.0 ms in oncpu_cyc but 18.7 ms of sum_exec_ns, because the spin's last stretch on the CPU hadn't been copied yet. Use the buckets to measure CPU time.

For the whole machine, csf_cpu_snap(&c, CSF_CPU_ALL) copies the sum row, and csf_cpu_snap(&c, cpu) copies one CPU's row. Rows are in nanoseconds and only as fresh as period_us; c.update_mono_ns says when they were taken.

examples/eventloop.c puts all of this together. For each slow operation it prints:

slow op sleep wall 25.1 ms, mostly sleeping (poll, futex, timer, ...) time cpu 0.0 + runq 0.0 + sleep 25.1 + dsleep 0.0 + iowait 0.0 ms switches 0 preempted, 1 blocked, 0 migrated; tid 26071 now on cpu 0 faults 3 minor, 0 major i