DBG-201 · Debugging & Tracing · Practitioner
ftrace & trace-cmd — full syllabus
The kernel's built-in tracer, used properly: function graphs, events and latency tracers.
Who this course is for
Kernel, driver and performance engineers who have touched ftrace through folklore and one-liners and want to use the kernel's built-in tracer deliberately — filters, events and latency tracers included.
Prerequisites
- Command-line Linux
- Basic kernel internals (what a syscall and an interrupt are)
- Ability to read C helpful for interpreting traces
Course outline
Day 1 — The tracer interface
- tracefs layout and the tracing files that actually matter
- Available tracers and what each one records
- function and function_graph tracing with set_ftrace_filter and set_graph_function
- Buffer sizing, trace clock and the cost of tracing itself
- Reading a function graph: nesting, duration and where the time went
Day 2 — Events, filters and triggers
- Tracepoints vs function tracing: stability and cost
- Enabling events individually, by subsystem and system-wide
- Per-event filtering on fields: capturing only the case you care about
- Triggers and histograms: building answers inside the kernel instead of post-processing
- Synthetic events for correlating two points in time
Day 3 — Latency tracers and capture tooling
- irqsoff, preemptoff and wakeup: what each measures and when to use it
- Reading a latency tracer report: the stack it hands you and the one it hides
- trace-cmd record and report: scripting captures instead of clicking tracefs
- KernelShark for timeline analysis of a recorded session
- Production capture practice: overhead, wrap-around and what to leave running
Hands-on labs
- Lab: trace a specific kernel path with function and function_graph tracing, narrowing filters until the output answers one precise question
- Lab: enable tracepoint events with per-event field filters and build an in-kernel histogram that isolates one slow case
- Lab: capture worst-case latency with the irqsoff/preemptoff/wakeup tracers and explain the stacks they produce
- Lab: record a workload with trace-cmd and analyse it in KernelShark, producing an annotated timeline of one incident
Capstone project
Investigate an injected latency regression on a loaded system. You build the trace-cmd capture, use latency tracers and event filters to isolate the bad window, and deliver a KernelShark-annotated timeline plus a written causal chain from the first anomalous event to the culprit code path — with buffer sizing and measured tracing overhead stated so the evidence is reproducible.
What you leave with
- tracefs fluency without folklore
- function_graph tracing with disciplined filters
- Event filters, triggers and in-kernel histograms
- Latency tracer interpretation (irqsoff, preemptoff, wakeup)
- A trace-cmd plus KernelShark workflow for captures you can share
Upcoming dates
| Dates | Where | Seats | Early bird | Regular | |
|---|---|---|---|---|---|
| 1 Nov – 3 Nov 20263 full days | RiyadhIn person · KAFD Conference Centre | 6 of 14 | — | SAR 7,880 | |
| 8 Nov – 10 Nov 20263 full days | Kuwait CityIn person · Al Hamra Tower | 11 of 14 | KWD 580until 9 Oct | ||
| 8 Nov – 10 Nov 20263 full days | MuscatIn person · Knowledge Oasis Muscat | 6 of 14 | OMR 730until 9 Oct | ||
| 15 Nov – 22 Nov 20266 half-days | Gulf bandLive online · 09:00–13:00 GMT+3 | 18 of 20 | US$1,350until 16 Oct | ||
| 16 Nov – 18 Nov 20263 full days | OttawaIn person · Kanata North Tech Park | 11 of 14 | CAD 2,570until 17 Oct | ||
| 16 Nov – 23 Nov 20266 half-days | Europe bandLive online · 09:00–13:00 CET | 7 of 20 | US$1,350until 17 Oct | ||
| 23 Nov – 25 Nov 20263 full days | TorontoIn person · MaRS Discovery District | 6 of 14 | CAD 2,570until 24 Oct | ||
| 23 Nov – 25 Nov 20263 full days | LondonIn person · Shoreditch Works | 11 of 14 | GBP 1,480until 24 Oct | ||
| 23 Nov – 30 Nov 20266 half-days | Americas bandLive online · 13:00–17:00 ET | 12 of 20 | US$1,350until 24 Oct | ||
| 30 Nov – 2 Dec 20263 full days | BerlinIn person · Factory Görlitzer Park | 6 of 14 | EUR 1,740until 31 Oct |
Book a seat, or bring this course to your team
Seats can be reserved online; private delivery runs on-site or live online, adapted to your stack.
Questions about fit or prerequisites? Email hello@kernelsystems.academy. To save this syllabus, print this page to PDF from your browser.