Train your team

AIC-330 · GPU & AI Compute

AI Infrastructure Security & Observability

Harden a multi-tenant GPU platform and see what it is doing before users report a problem.

Advanced 2 days in person4 half-days online Max 14 in person

Who this course is for

Security-conscious platform engineers and architects running shared AI infrastructure — who must harden multi-tenant GPU clusters and see what is happening inside them.

Prerequisites

Kubernetes administrationAIC-300 or equivalent GPU platform experienceSecurity fundamentals (RBAC, network policy)

Course outline

Day 1 — AI infrastructure security

  • Kubernetes security: Pod Security Standards, network policies, admission controllers
  • Container runtime security: Falco and syscall monitoring
  • RBAC and access control: OIDC, SAML, AD integration
  • Dataset access controls and lineage
  • GPU anomaly detection with DCGM: error states, ECC faults
  • Secure multi-tenancy; model security basics; supply-chain security

Day 2 — Observability and monitoring

  • GPU monitoring: DCGM, DCGM Exporter, Prometheus, Grafana
  • System telemetry: CPU, memory, network
  • Logging: syslog, journald, Fluentd, ELK
  • Distributed tracing: Jaeger, OpenTelemetry
  • Alerting: Alertmanager, PagerDuty
  • Drift detection (data and model); performance regression tracking
  • Capstone workshop

Hands-on labs

Labs follow the academy model — 35% principles, 20% guided investigation, 45% engineering studio. Every claim you make in a lab is backed by a trace, a counter or a measurement you captured yourself. How we teach

  1. Lab: harden a GPU namespace: Pod Security Standards, network policy and RBAC — then try to break it
  2. Lab: deploy Falco with custom rules and catch a simulated runtime attack in a GPU workload
  3. Lab: build a DCGM → Prometheus → Grafana pipeline with dashboards for utilisation, ECC errors and anomalies
  4. Lab: wire distributed tracing and alerting for an inference endpoint; detect an injected performance regression

Capstone project

Secure and instrument a shared GPU cluster: apply the hardening baseline, deploy the full observability stack, then survive a live exercise — an injected anomaly and an attempted policy violation — producing an incident timeline from your own telemetry.

What you leave with

  • A GPU-cluster hardening checklist applied hands-on
  • Falco rule-writing and runtime detection experience
  • A production DCGM/Prometheus/Grafana monitoring stack
  • Drift and regression detection patterns for ML systems

How it runs

Every course follows the same model: 35% principles, 20% guided investigation, 45% engineering studio. You leave with working code, raw measurements and an evidence-based report — not a certificate of attendance. Read the methodology or see a full sample lesson.

Material is adapted to your kernel version, hardware and workload before a private delivery. For public cohorts, the environment is provided and configured.

Questions

Who is this course for?

Security-conscious platform engineers and architects running shared AI infrastructure — who must harden multi-tenant GPU clusters and see what is happening inside them. It sits at advanced level within the GPU & AI Compute track.

What do I need to know already?

Specific prerequisites for this course: Kubernetes administration; AIC-300 or equivalent GPU platform experience; Security fundamentals (RBAC, network policy). We confirm levels before the cohort starts and adapt if a group is stronger or weaker than expected.

Can this run privately for my team?

Yes. Any course runs on-site at your offices anywhere, or live online for a distributed team, with labs adapted to your hardware and codebase.

What is the difference between in-person and online?

In person is 2 full days with hardware on your desk, capped at 14. Online is 4 half-day sessions across about two weeks so you can keep working, capped at 20, with remote lab access.

Do you invoice companies?

Yes. Purchase orders are accepted and invoicing is available in USD, EUR, GBP, SAR and CAD.

Upcoming dates

DatesWhereSeatsEarly birdRegular
18 Oct – 19 Oct 20262 full days RiyadhIn person · KAFD Conference Centre 5 of 14 —SAR 6,000
25 Oct – 26 Oct 20262 full days Kuwait CityIn person · Al Hamra Tower 10 of 14 —KWD 500
1 Nov – 2 Nov 20262 full days MuscatIn person · Knowledge Oasis Muscat 5 of 14 —OMR 620
1 Nov – 4 Nov 20264 half-days Gulf bandLive online · 09:00–13:00 GMT+3 17 of 20 —US$1,150
2 Nov – 3 Nov 20262 full days OttawaIn person · Kanata North Tech Park 10 of 14 —CAD 2,180
9 Nov – 10 Nov 20262 full days TorontoIn person · MaRS Discovery District 5 of 14 CAD 1,960until 10 OctCAD 2,180
9 Nov – 12 Nov 20264 half-days Europe bandLive online · 09:00–13:00 CET 6 of 20 US$1,040until 10 OctUS$1,150
9 Nov – 12 Nov 20264 half-days Americas bandLive online · 13:00–17:00 ET 11 of 20 US$1,040until 10 OctUS$1,150
16 Nov – 17 Nov 20262 full days LondonIn person · Shoreditch Works 10 of 14 GBP 1,120until 17 OctGBP 1,250
16 Nov – 17 Nov 20262 full days BerlinIn person · Factory Görlitzer Park 5 of 14 EUR 1,320until 17 OctEUR 1,470

Dates shown for the next few months. If nothing fits, tell us where and when — cohorts are added on demand, and private delivery can be scheduled any week.

More in GPU & AI Compute

AIC-1003 days Foundations for AI Compute The architecture, operating system and networking groundwork every GPU systems engineer is assumed to have and often does not. Foundation Practitioner-taught SAR 6,750Next 25 Oct AIC-1103 days GPU Architecture, Memory & Interconnects How the hardware constrains your workload: SIMT execution, the memory hierarchy and the fabric between GPUs. FoundationNCA-AIIO Practitioner-taught SAR 6,750Next 11 Oct AIC-2004 days Linux for GPU Systems What the kernel is doing underneath your training job, and how to tune it. The layer almost nobody teaches. Practitioner Practitioner-taught SAR 10,500Next 15 Nov AIC-2104 days RDMA & AI Cluster Networking Build and debug the fabric distributed training runs on, from queue pairs up to a tuned NCCL all-reduce. PractitionerNCP-AIN Practitioner-taught SAR 10,500Next 1 Nov AIC-2205 days CUDA & HIP Programming Write, profile and optimise GPU kernels on both vendors, including the CUDA-to-HIP porting path. Practitioner Practitioner-taught SAR 13,120Next 11 Oct AIC-2304 days Distributed Training with PyTorch From a single-GPU training loop to sharded multi-node training that survives a node failure. Practitioner Practitioner-taught SAR 10,500Next 15 Nov AIC-3004 days Containers, Kubernetes & GPU Schedulers Run a shared GPU cluster multiple teams can actually use: partitioning, scheduling, quotas and isolation. PractitionerNCP-AII Practitioner-taught SAR 10,500Next 18 Oct AIC-3103 days Storage & Data Pipelines for AI Stop starving your GPUs: parallel filesystems, GPUDirect Storage and pipelines built for sustained throughput. Practitioner Practitioner-taught SAR 7,880Next 22 Nov AIC-3203 days MLOps & Inference Serving Get models off a laptop and onto a GPU endpoint that scales, with the pipeline machinery around them. PractitionerNCP-AIO Practitioner-taught SAR 7,880Next 1 Nov AIC-4003 days Large-Scale Training & Datacenter Architecture The thousand-GPU conversation: 3D parallelism, reference architectures, TCO and where the hardware is heading. Advanced Practitioner-taught SAR 9,000Next 8 Nov