DBG-101 · Debugging & Tracing · Practitioner
Reading an Oops & Panic Analysis — full syllabus
Turning a kernel splat into a precise location in the source, and knowing what the register dump is telling you.
Who this course is for
Driver, embedded and platform engineers — and the support engineers who back them — who get handed kernel splats and need to turn registers and a call trace into a source line and a next step.
Prerequisites
- Ability to read C
- Command-line Linux
- Basic familiarity with kernel build artifacts (vmlinux, System.map) helpful
Course outline
Day 1 — Anatomy of an oops
- What the kernel prints and why: the oops message section by section
- Registers that matter: the faulting instruction pointer, stack pointer and the address in the fault
- Reading the call trace: what each frame is and when to distrust it
- Tainted flags: every flag, what set it, and what it does to your bug report
- Decoding addresses to source with addr2line, scripts/decode_stacktrace and scripts/faddr2line
- Matching the splat to the exact vmlinux it came from
Day 2 — Fault classes and evidence preservation
- Common fault classes and their signatures: NULL dereference, page fault in atomic context, bad RIP, general protection fault, stack corruption
- Panic vs oops vs warning, and the configuration that controls each: panic_on_oops, panic_on_warn, hung-task and softlockup/hardlockup detectors
- Preserving evidence when the machine dies: pstore, ramoops and serial console capture; netconsole for headless systems
- Building a triage habit: from splat to fault class to hypothesis to the right next tool
- Workshop: classifying a bundle of real splats under time pressure
Hands-on labs
- Lab: decode a captured oops with addr2line and scripts/decode_stacktrace down to the exact faulting source line
- Lab: classify a set of raw splats by fault class using register and call-trace signatures alone, then check your answers
- Lab: configure pstore/ramoops on a test kernel, trigger a panic, and recover the preserved log after reboot
- Lab: set up serial console capture to a second machine and prove no line of a panic is lost
- Lab: compare panic, oops and warning behaviour under different panic_on_* settings and recommend a production configuration
Capstone project
You receive a bundle of raw kernel logs from several failing machines, some incomplete. For each incident you produce a triage report: the decoded faulting line, the fault class with the register/trace evidence supporting it, what evidence is missing, and the capture configuration (pstore, ramoops, serial) that would have preserved it — ending with a written first-response checklist you take back to your own fleet.
What you leave with
- The ability to take any oops to a precise source line
- Working fluency with addr2line, decode_stacktrace and faddr2line
- A fault-class signature reference you built from real cases
- An evidence-preservation configuration (pstore, ramoops, serial) ready to deploy
Upcoming dates
| Dates | Where | Seats | Early bird | Regular | |
|---|---|---|---|---|---|
| 11 Oct – 12 Oct 20262 full days | RiyadhIn person · KAFD Conference Centre | 5 of 14 | — | SAR 5,250 | |
| 18 Oct – 19 Oct 20262 full days | Kuwait CityIn person · Al Hamra Tower | 10 of 14 | — | KWD 430 | |
| 18 Oct – 19 Oct 20262 full days | MuscatIn person · Knowledge Oasis Muscat | 5 of 14 | — | OMR 540 | |
| 25 Oct – 28 Oct 20264 half-days | Gulf bandLive online · 09:00–13:00 GMT+3 | 17 of 20 | — | US$1,000 | |
| 26 Oct – 27 Oct 20262 full days | OttawaIn person · Kanata North Tech Park | 10 of 14 | — | CAD 1,900 | |
| 26 Oct – 29 Oct 20264 half-days | Europe bandLive online · 09:00–13:00 CET | 6 of 20 | — | US$1,000 | |
| 2 Nov – 3 Nov 20262 full days | TorontoIn person · MaRS Discovery District | 5 of 14 | — | CAD 1,900 | |
| 2 Nov – 3 Nov 20262 full days | LondonIn person · Shoreditch Works | 10 of 14 | — | GBP 1,090 | |
| 2 Nov – 5 Nov 20264 half-days | Americas bandLive online · 13:00–17:00 ET | 11 of 20 | — | US$1,000 | |
| 9 Nov – 10 Nov 20262 full days | BerlinIn person · Factory Görlitzer Park | 5 of 14 | EUR 1,160until 10 Oct |
Book a seat, or bring this course to your team
Seats can be reserved online; private delivery runs on-site or live online, adapted to your stack.
Questions about fit or prerequisites? Email hello@kernelsystems.academy. To save this syllabus, print this page to PDF from your browser.