PRF-110 · Performance Engineering
Benchmarking Without Fooling Yourself
Producing performance numbers that survive scrutiny, including your own six months later.
Who this course is for
Engineers who run or consume benchmarks — and who need their numbers to survive code review, procurement decisions and their own re-run six months later.
Prerequisites
Course outline
Day 1 — Designing the experiment
- Experiment design, controls and confounders: what else moved while you measured
- Warm-up, steady state and choosing the measurement window
- Environment manifest: frequency scaling, topology, cache state, background load
- Micro vs macro benchmarks and what each can actually prove
- The observer effect: what your instrumentation does to the result
Day 2 — Statistics and honest reporting
- Variance, outliers and distributions: why the mean lies
- Percentiles and tail latency; coordinated omission in latency measurement
- How many runs are enough: confidence intervals without a statistics degree
- Regression detection: comparing two builds without fooling yourself
- Documenting a result so it is reproducible: data, environment, method
Hands-on labs
Labs follow the academy model — 35% principles, 20% guided investigation, 45% engineering studio. Every claim you make in a lab is backed by a trace, a counter or a measurement you captured yourself. How we teach
- Lab: expose benchmark noise — rerun a naive benchmark and quantify how much of the 'result' was environment
- Lab: build a controlled harness with pinned frequency, placement and warm-up; show the variance collapse
- Lab: catch coordinated omission in a load generator's latency report and correct it with response-time-aware measurement
- Lab: measure the observer effect — run a workload with and without perf/eBPF instrumentation and quantify the overhead
- Lab: write a reproducible benchmark report (raw data, environment manifest, method) and have a peer reproduce your number
Capstone project
Produce a benchmark result that survives scrutiny: pick a real comparison (two kernels, two configurations, two libraries), preregister the hypothesis and method, run the experiment with controls and an environment manifest, and deliver a report with raw data, distributions not just means, and a stated confidence in the conclusion — including the effects you could not control.
What you leave with
- An experiment-design checklist covering controls, warm-up and confounders
- Working habits for percentiles, tail latency and coordinated omission
- A controlled-harness template for your own benchmarks
- A report format with raw data and environment manifest attached
- The instinct to ask what moved besides the thing you changed
How it runs
Every course follows the same model: 35% principles, 20% guided investigation, 45% engineering studio. You leave with working code, raw measurements and an evidence-based report — not a certificate of attendance. Read the methodology or see a full sample lesson.
Material is adapted to your kernel version, hardware and workload before a private delivery. For public cohorts, the environment is provided and configured.
Questions
Who is this course for?
Engineers who run or consume benchmarks — and who need their numbers to survive code review, procurement decisions and their own re-run six months later. It sits at practitioner level within the Performance Engineering track.
What do I need to know already?
Specific prerequisites for this course: Basic Linux command line and shell scripting; Elementary statistics (mean, standard deviation) — we rebuild the rest; A workload or tool you currently benchmark. We confirm levels before the cohort starts and adapt if a group is stronger or weaker than expected.
Can this run privately for my team?
Yes. Any course runs on-site at your offices anywhere, or live online for a distributed team, with labs adapted to your hardware and codebase.
What is the difference between in-person and online?
In person is 2 full days with hardware on your desk, capped at 14. Online is 4 half-day sessions across about two weeks so you can keep working, capped at 20, with remote lab access.
Do you invoice companies?
Yes. Purchase orders are accepted and invoicing is available in USD, EUR, GBP, SAR and CAD.
Upcoming dates
| Dates | Where | Seats | Early bird | Regular | |
|---|---|---|---|---|---|
| 11 Oct – 12 Oct 20262 full days | RiyadhIn person · KAFD Conference Centre | 4 of 14 | — | SAR 5,250 | |
| 18 Oct – 19 Oct 20262 full days | Kuwait CityIn person · Al Hamra Tower | 9 of 14 | — | KWD 430 | |
| 25 Oct – 26 Oct 20262 full days | MuscatIn person · Knowledge Oasis Muscat | 4 of 14 | — | OMR 540 | |
| 25 Oct – 28 Oct 20264 half-days | Gulf bandLive online · 09:00–13:00 GMT+3 | 10 of 20 | — | US$1,000 | |
| 26 Oct – 27 Oct 20262 full days | OttawaIn person · Kanata North Tech Park | 9 of 14 | — | CAD 1,900 | |
| 2 Nov – 3 Nov 20262 full days | TorontoIn person · MaRS Discovery District | 4 of 14 | — | CAD 1,900 | |
| 2 Nov – 5 Nov 20264 half-days | Europe bandLive online · 09:00–13:00 CET | 15 of 20 | — | US$1,000 | |
| 2 Nov – 5 Nov 20264 half-days | Americas bandLive online · 13:00–17:00 ET | 4 of 20 | — | US$1,000 | |
| 9 Nov – 10 Nov 20262 full days | LondonIn person · Shoreditch Works | 9 of 14 | GBP 980until 10 Oct | ||
| 9 Nov – 10 Nov 20262 full days | BerlinIn person · Factory Görlitzer Park | 4 of 14 | EUR 1,160until 10 Oct |
Dates shown for the next few months. If nothing fits, tell us where and when — cohorts are added on demand, and private delivery can be scheduled any week.
More in Performance Engineering
PRF-1012 days
Performance Methodology & the USE Method
A repeatable process for performance investigation, so you stop tuning things that were never the bottleneck.
Practitioner-taught
SAR 5,250Next 1 Nov
PRF-2013 days
CPU & Scheduler Tuning
Getting the scheduler out of your way: placement, priorities, frequency scaling and isolation.
Practitioner-taught
SAR 9,000Next 22 Nov
PRF-2103 days
Memory & TLB Optimisation
Memory-bound workloads: huge pages, NUMA placement, allocator behaviour and reclaim pressure.
Practitioner-taught
SAR 9,000Next 1 Nov
PRF-2203 days
I/O & Block Layer Tuning
Storage performance from the filesystem to the device queue, including NVMe specifics.
Practitioner-taught
SAR 9,000Next 18 Oct
PRF-2303 days
Network Stack Tuning
Getting throughput and latency out of the kernel network path, and knowing when to leave it.
Practitioner-taught
SAR 9,000Next 15 Nov