PRF-110 · Performance Engineering · Practitioner
Benchmarking Without Fooling Yourself — full syllabus
Producing performance numbers that survive scrutiny, including your own six months later.
Who this course is for
Engineers who run or consume benchmarks — and who need their numbers to survive code review, procurement decisions and their own re-run six months later.
Prerequisites
- Basic Linux command line and shell scripting
- Elementary statistics (mean, standard deviation) — we rebuild the rest
- A workload or tool you currently benchmark
Course outline
Day 1 — Designing the experiment
- Experiment design, controls and confounders: what else moved while you measured
- Warm-up, steady state and choosing the measurement window
- Environment manifest: frequency scaling, topology, cache state, background load
- Micro vs macro benchmarks and what each can actually prove
- The observer effect: what your instrumentation does to the result
Day 2 — Statistics and honest reporting
- Variance, outliers and distributions: why the mean lies
- Percentiles and tail latency; coordinated omission in latency measurement
- How many runs are enough: confidence intervals without a statistics degree
- Regression detection: comparing two builds without fooling yourself
- Documenting a result so it is reproducible: data, environment, method
Hands-on labs
- Lab: expose benchmark noise — rerun a naive benchmark and quantify how much of the 'result' was environment
- Lab: build a controlled harness with pinned frequency, placement and warm-up; show the variance collapse
- Lab: catch coordinated omission in a load generator's latency report and correct it with response-time-aware measurement
- Lab: measure the observer effect — run a workload with and without perf/eBPF instrumentation and quantify the overhead
- Lab: write a reproducible benchmark report (raw data, environment manifest, method) and have a peer reproduce your number
Capstone project
Produce a benchmark result that survives scrutiny: pick a real comparison (two kernels, two configurations, two libraries), preregister the hypothesis and method, run the experiment with controls and an environment manifest, and deliver a report with raw data, distributions not just means, and a stated confidence in the conclusion — including the effects you could not control.
What you leave with
- An experiment-design checklist covering controls, warm-up and confounders
- Working habits for percentiles, tail latency and coordinated omission
- A controlled-harness template for your own benchmarks
- A report format with raw data and environment manifest attached
- The instinct to ask what moved besides the thing you changed
Upcoming dates
| Dates | Where | Seats | Early bird | Regular | |
|---|---|---|---|---|---|
| 11 Oct – 12 Oct 20262 full days | RiyadhIn person · KAFD Conference Centre | 4 of 14 | — | SAR 5,250 | |
| 18 Oct – 19 Oct 20262 full days | Kuwait CityIn person · Al Hamra Tower | 9 of 14 | — | KWD 430 | |
| 25 Oct – 26 Oct 20262 full days | MuscatIn person · Knowledge Oasis Muscat | 4 of 14 | — | OMR 540 | |
| 25 Oct – 28 Oct 20264 half-days | Gulf bandLive online · 09:00–13:00 GMT+3 | 10 of 20 | — | US$1,000 | |
| 26 Oct – 27 Oct 20262 full days | OttawaIn person · Kanata North Tech Park | 9 of 14 | — | CAD 1,900 | |
| 2 Nov – 3 Nov 20262 full days | TorontoIn person · MaRS Discovery District | 4 of 14 | — | CAD 1,900 | |
| 2 Nov – 5 Nov 20264 half-days | Europe bandLive online · 09:00–13:00 CET | 15 of 20 | — | US$1,000 | |
| 2 Nov – 5 Nov 20264 half-days | Americas bandLive online · 13:00–17:00 ET | 4 of 20 | — | US$1,000 | |
| 9 Nov – 10 Nov 20262 full days | LondonIn person · Shoreditch Works | 9 of 14 | GBP 980until 10 Oct | ||
| 9 Nov – 10 Nov 20262 full days | BerlinIn person · Factory Görlitzer Park | 4 of 14 | EUR 1,160until 10 Oct |
Book a seat, or bring this course to your team
Seats can be reserved online; private delivery runs on-site or live online, adapted to your stack.
Questions about fit or prerequisites? Email hello@kernelsystems.academy. To save this syllabus, print this page to PDF from your browser.