Own the physical reality of the platform in one of our seven regions. You bring up new GPU racks, validate InfiniBand fabric end-to-end, and keep the cluster running at the SLA. This is a hands-on role; you will see the hardware.

Type Full-time · IC4-IC6

Stack Linux InfiniBand (NDR / XDR) NCCL / RCCL Kubernetes (host-level) Terraform Prometheus / Grafana Go or Python

Hiring manager replies within 5 business days.

The team

About the team

Cluster SRE is six engineers across the seven regions. Each region has a primary and a secondary; you will be one of those for your region. The team coordinates daily, deploys weekly, and rotates a global pager.

Reports to the head of cluster engineering. Primary on a single region; rotates secondary for one neighboring region.

What you'll do

- 01 Bring up current B200 / B300 / MI300X racks: cabling, ToR config, NCCL/RCCL all-reduce validation, MFU baseline tests.
- 02 Drive InfiniBand fabric to spec - NDR / XDR depending on the rack - and chase residual bit-error budget down to zero.




- 03 Run capacity planning across seven regions: forecast demand, model power and thermal headroom, work with procurement on lead times.
- 04 Own the regional incident response. P1 incidents page within fifteen minutes; resolution target is four hours.
- 05 Build and maintain the bring-up runbook so the second hire after you can do their first rack solo.
- 06 Carry the global pager about one week per six, alongside runtime and customer engineering.

The bar

What we're looking for

Five-plus years operating large compute clusters — supercomputing centers, hyperscaler infra, or HPC at a national lab count.

Deep InfiniBand and Ethernet RoCE experience: subnet manager tuning, fabric debugging, lossless networking.

Comfort writing Go or Python for tooling. We are not strict about which.

Calm under load. You will be the named person on a $50M-ARR

📌 Cluster Site Reliability Engineer (Canada)
🏢 iFrame
📍 Canada