Who you are

- Hands-on experience or a strong interest in SRE, infrastructure, platform engineering, DevOps, or production operations

- Familiarity with LLM evaluations, benchmarks, agents, or applied AI research
- An experimental mindset, with an interest in designing reproducible tests, examining imperfect results, and learning quickly
- A self-driven approach, with the initiative to make sound decisions and move ambitious projects forward with minimal guidance
- The ability to explain technical work clearly and enthusiasm for sharing it through articles, social media, and talks at meetups or conferences

- Comfort working in a hybrid setting and collaborating asynchronously with a curious, distributed team

- You do not need to match every qualification. We value practical experience, sound judgment, curiosity, and the ability to turn ambitious ideas into useful work

What the job involves

- You will manage SRE Skills Bench, Rootly AI Labs' open-source benchmark for measuring how well AI models handle real-world reliability engineering tasks

- You will help move the project from research questions to public releases. This includes designing realistic SRE scenarios, expanding the evaluation dataset, running model evaluations, improving the benchmark infrastructure, and sharing the findings with the AI and reliability communities
- The role combines applied AI research, technical project management, community building, and public communication.



You will report directly to the Head of Rootly AI Labs and work closely with fellows, researchers, and engineers across the community

- Participate in shaping the roadmap and research questions for SRE Skills Bench

- Help build an active community around the benchmark and encourage outside contributions

- Create realistic evaluation tasks based on incident response, infrastructure, debugging, observability, and production operations

- Help develop reliable and reproducible methods for evaluating frontier and open-source models

- Analyze results and turn them into reports, technical articles, and public datasets

- Collaborate with SREs, researchers, model providers, and open-source contributors

- Evaluate how new models, agents, and coding environments perform on operational work

- Help establish SRE Skills Bench as the industry standard for evaluating AI on reliability engineering

- Duration: Ongoing Internship with the possibility to conversion to a full-time role based on fit and project needs

Perks

- Medical, dental, vision, and life coverage for you and your dependents

- Flexible PTO, plus company holidays, team off-sites, and a year-end shutdown to recharge

- Meaningful equity in a high-growth company, and 401(k) matching for US employees

- Hybrid work (we hire across the US, Canada, Europe, and Singapore)

- Free lunches at HQ
- Fast-moving, high-impact environment where you have autonomy to own big problems from day one

- Top-tier hardware and tools, plus an annual stipends for home offices

📌 Member of Technical Staff (Toronto)
🏢 Rootly
📍 Toronto