Sample lesson: tracing a system call from EL0 to EL1

Before you start

You have a QEMU AArch64 virtual machine running a mainline Linux kernel you built yourself, with ftrace enabled. You also have a terminal on the host. Nothing in this lesson requires physical hardware, and every command is reproducible from the notes you keep.

The question

When userspace calls write(2), what actually happens between the C library issuing the instruction and the kernel's syscall handler running — and how long does each stage cost?

Most engineers can recite "it traps into the kernel." In this lesson you will show the whole path, with timestamps, and be able to defend every claim with a trace line.

Step 1 — the userspace view

Run a trivial writer under strace -tt and note the wall-clock span of the write call. This is your outer envelope; everything you measure from here must fit inside it. If a later number does not fit, your method is wrong — that is the first lesson of measurement discipline.

Step 2 — find the trap

On AArch64 the svc #0 instruction raises a synchronous exception to EL1. You locate the kernel's exception vector table (vectors in entry.S), follow the el0_svc path through the syscall table lookup, and identify the exact instructions that save the user register frame. You write down the sequence; reciting is not allowed, only reading.

Step 3 — instrument it

Enable the function graph tracer on the syscall path:

cd /sys/kernel/tracing
echo function_graph > current_tracer
echo '__arm64_sys_write' > set_graph_function
echo 1 > tracing_on; ./writer; echo 0 > tracing_on
cat trace

You now hold a timestamped record of the kernel-side entry, the VFS dispatch, and the return — per call, in nanoseconds.

Step 4 — reconcile the layers

Compare three numbers: strace's wall-clock span, ftrace's in-kernel span, and a perf stat cycle count for the same workload. The gap between userspace and in-kernel time is the exception entry/exit cost plus scheduling noise. You explain the gap quantitatively — or you measure again until you can.

Step 5 — break it on purpose

Repeat under CPU contention and under taskset pinning. Watch the entry cost move. The hypothesis you wrote in step 4 now either survives or gets revised, and you record which.

What you hand in

The trace excerpts, the three reconciled measurements, your annotated map of the EL0→EL1 path, and a paragraph stating what you observed, what you inferred, and what you could not verify. Facts, interpretations, assumptions — kept separate, always.

Why we teach this way

The syscall path is a vehicle. What transfers to your job is the loop: hypothesis, instrument, measure, reconcile, report. That loop works on a scheduler regression, a DMA coherency bug, or a 400 GbE fabric — which is why every course in the catalog is built around it.