DBG-220 · Debugging & Tracing
eBPF & bpftrace
Programmable observability: one-liners for immediate answers, custom programs for the questions nothing else answers.
Who this course is for
Observability, platform and kernel engineers who need programmable answers from a running system — one-liners today, custom CO-RE tools in production tomorrow — without destabilising the machine they are watching.
Prerequisites
Course outline
Day 1 — The eBPF machine
- The eBPF execution model: programs, attachment points and events
- The verifier: what it guarantees and what it forbids
- Maps and their types: hash, array, ring buffer, perf buffer
- How BCC and bpftrace tools work under the hood
- Reading the existing tool set before writing anything: execsnoop, opensnoop, biolatency and friends
Day 2 — bpftrace
- The bpftrace language: probes, actions, built-in variables
- One-liners for immediate answers on a live system
- Maps, aggregations and histograms in bpftrace
- Probe types compared: kprobes, uprobes, tracepoints and USDT — stability vs visibility
- Script structure for investigations you will rerun
Day 3 — Custom tools with libbpf and CO-RE
- Why CO-RE exists: the kernel-version fragility problem
- BTF and how it makes programs portable
- A minimal libbpf skeleton: load, attach, read maps
- Writing a custom tool from question to working program
- Packaging a tool so it survives kernel upgrades
Day 4 — Production safety
- Measuring your own overhead on a loaded system
- Stability across kernel versions: what breaks and what does not
- Verifier limits in practice: instruction counts, loops, stack depth
- When not to use eBPF: the questions better answered by ftrace, perf or a dump
- Case-study workshop: from a production symptom to a deployed tool
Hands-on labs
Labs follow the academy model — 35% principles, 20% guided investigation, 45% engineering studio. Every claim you make in a lab is backed by a trace, a counter or a measurement you captured yourself. How we teach
- Lab: answer five pointed system questions with bpftrace one-liners, then explain what each probe actually attaches to
- Lab: attach kprobe, uprobe, tracepoint and USDT probes to one workload and compare what each can and cannot see
- Lab: write a bpftrace script with maps and aggregations that quantifies a per-process latency distribution
- Lab: build a minimal libbpf CO-RE tool and run it on two different kernels without recompiling
- Lab: measure your tool's overhead under load and state its safe production envelope in numbers
Capstone project
Build a small production-grade observability tool for a real question the course sets (for example, per-process block-I/O latency outliers). You prototype in bpftrace for iteration speed, reimplement in libbpf CO-RE for deployment, measure the overhead under load, and deliver the tool plus a runbook entry: safe-use envelope, known blind spots, and the kernel-version compatibility evidence.
What you leave with
- bpftrace fluency from one-liners to structured scripts
- Probe-type selection: kprobe vs uprobe vs tracepoint vs USDT
- A working libbpf + CO-RE tool-building workflow
- Verifier and overhead literacy grounded in your own measurements
- Judgment about when eBPF is the wrong tool
How it runs
Every course follows the same model: 35% principles, 20% guided investigation, 45% engineering studio. You leave with working code, raw measurements and an evidence-based report — not a certificate of attendance. Read the methodology or see a full sample lesson.
Material is adapted to your kernel version, hardware and workload before a private delivery. For public cohorts, the environment is provided and configured.
Questions
Who is this course for?
Observability, platform and kernel engineers who need programmable answers from a running system — one-liners today, custom CO-RE tools in production tomorrow — without destabilising the machine they are watching. It sits at advanced level within the Debugging & Tracing track.
What do I need to know already?
Specific prerequisites for this course: Solid Linux systems knowledge; Ability to read C; A recent kernel (BTF-enabled) on your test machines. We confirm levels before the cohort starts and adapt if a group is stronger or weaker than expected.
Can this run privately for my team?
Yes. Any course runs on-site at your offices anywhere, or live online for a distributed team, with labs adapted to your hardware and codebase.
What is the difference between in-person and online?
In person is 4 full days with hardware on your desk, capped at 14. Online is 8 half-day sessions across about two weeks so you can keep working, capped at 20, with remote lab access.
Do you invoice companies?
Yes. Purchase orders are accepted and invoicing is available in USD, EUR, GBP, SAR and CAD.
Upcoming dates
| Dates | Where | Seats | Early bird | Regular | |
|---|---|---|---|---|---|
| 15 Nov – 18 Nov 20264 full days | RiyadhIn person · KAFD Conference Centre | 7 of 14 | SAR 10,800until 16 Oct | ||
| 22 Nov – 25 Nov 20264 full days | Kuwait CityIn person · Al Hamra Tower | 12 of 14 | KWD 890until 23 Oct | ||
| 22 Nov – 25 Nov 20264 full days | MuscatIn person · Knowledge Oasis Muscat | 7 of 14 | OMR 1,110until 23 Oct | ||
| 29 Nov – 8 Dec 20268 half-days | Gulf bandLive online · 09:00–13:00 GMT+3 | 15 of 20 | US$2,070until 30 Oct | ||
| 30 Nov – 3 Dec 20264 full days | OttawaIn person · Kanata North Tech Park | 12 of 14 | CAD 3,920until 31 Oct | ||
| 30 Nov – 9 Dec 20268 half-days | Europe bandLive online · 09:00–13:00 CET | 4 of 20 | US$2,070until 31 Oct | ||
| 7 Dec – 10 Dec 20264 full days | TorontoIn person · MaRS Discovery District | 7 of 14 | CAD 3,920until 7 Nov | ||
| 7 Dec – 10 Dec 20264 full days | LondonIn person · Shoreditch Works | 12 of 14 | GBP 2,250until 7 Nov | ||
| 7 Dec – 16 Dec 20268 half-days | Americas bandLive online · 13:00–17:00 ET | 9 of 20 | US$2,070until 7 Nov | ||
| 14 Dec – 17 Dec 20264 full days | BerlinIn person · Factory Görlitzer Park | 7 of 14 | EUR 2,650until 14 Nov |
Dates shown for the next few months. If nothing fits, tell us where and when — cohorts are added on demand, and private delivery can be scheduled any week.
More in Debugging & Tracing
DBG-1012 days
Reading an Oops & Panic Analysis
Turning a kernel splat into a precise location in the source, and knowing what the register dump is telling you.
Practitioner-taught
SAR 5,250Next 11 Oct
DBG-1103 days
kdump & the crash Utility
Capturing a crash dump in production and doing a real post-mortem on it.
Practitioner-taught
SAR 9,000Next 8 Nov
DBG-1203 days
kgdb & Live Kernel Debugging
Interactive kernel debugging over serial and network, plus dynamic debug for cases where stopping is not an option.
Practitioner-taught
SAR 9,000Next 25 Oct
DBG-2013 days
ftrace & trace-cmd
The kernel's built-in tracer, used properly: function graphs, events and latency tracers.
Practitioner-taught
SAR 7,880Next 1 Nov
DBG-2103 days
perf: Sampling to Flame Graphs
CPU and off-CPU analysis with perf, from first sample to a flame graph that tells you something actionable.
Practitioner-taught
SAR 7,880Next 11 Oct
DBG-3013 days
Race Conditions & Lock Contention
The bugs that only appear under load on someone else's machine, and a method for actually finding them.
Practitioner-taught
SAR 10,120Next 22 Nov
DBG-3103 days
Memory Corruption: KASAN & KFENCE
Finding use-after-free, out-of-bounds and uninitialised memory before they become a security advisory.
Practitioner-taught
SAR 9,000Next 1 Nov