17 Nov
|
GLOBAL INNOVATIVE SOLUTIONS
|
Mississauga
17 Nov
GLOBAL INNOVATIVE SOLUTIONS
Mississauga
Job DescriptionJob Description
1. Job Overview
We are looking for an experienced GPU Data Center Operations Engineer to manage and maintain our AI computing infrastructure deployed in a third-party colocation data center in Canada.
The successful candidate will be responsible for the daily operation, monitoring, maintenance, and troubleshooting of approximately 72 GPU servers, ensuring high availability and stable performance of our GPU computing infrastructure.
This role focuses on server-level hardware and software operations, GPU cluster maintenance, and coordination with the third-party data center operator. The candidate will work closely with our overseas operations team and internal technical team.
2. Key Responsibilities
A. GPU Server Operations & Maintenance
l Perform daily inspections and routine maintenance of 72 GPU servers.
l Monitor GPU health, utilization, temperature, power consumption, memory, CPU, storage, and network status.
l Maintain server hardware, including GPUs, CPUs, memory, storage, power supplies, and network components.
l Coordinate server installation, rack deployment, cabling, and hardware replacement with the data center operator.
l Maintain accurate hardware inventory, serial numbers, and equipment records.
B. Hardware Troubleshooting & Incident Management
l Diagnose and troubleshoot GPU server hardware failures and system abnormalities.
l Identify and resolve common issues involving GPUs, memory, storage, power supplies, and network connectivity.
l Coordinate hardware repairs, RMA processes, and warranty claims with hardware vendors.
l Maintain incident logs, troubleshooting records, and maintenance reports.
l Respond to critical incidents and coordinate timely resolution with relevant technical teams.
C. Linux & GPU Infrastructure Management
l Perform Linux server administration, system configuration, and routine maintenance.
l Install, configure, and maintain NVIDIA GPU drivers, CUDA, and related software environments.
l Use NVIDIA-SMI and other monitoring tools to monitor GPU health and performance.
l Assist with GPU cluster deployment, validation, and performance troubleshooting.
l Support software updates, system upgrades, and infrastructure optimization.
D. Data Center Coordination
l Act as the technical point of contact with the third-party colocation data center.
l Coordinate remote hands services, in office technical assistance, server access, and maintenance activities.
l Monitor and report issues related to rack power, PDU, cooling, temperature, and network connectivity.
l Ensure all maintenance activities comply with data center security and operational procedures.
l Coordinate service requests and escalate infrastructure-related issues to the data center operator.
E. Reporting & Cross-functional Collaboration
l Prepare regular infrastructure health reports and GPU utilization reports.
l Maintain documentation of hardware status, incidents, repairs,
and maintenance activities.
l Work closely with the company's overseas operations and internal engineering teams.
l Support new hardware deployment, infrastructure expansion, and equipment acceptance.
3. Qualifications
l 3+ years of experience in server operations, data center operations, or IT infrastructure support.
l Strong knowledge of Linux system administration and troubleshooting.
l Hands-on experience with GPU servers and enterprise server hardware.
l Familiarity with NVIDIA GPU architecture, GPU drivers, CUDA, and NVIDIA-SMI.
l Experience diagnosing hardware issues involving GPUs, memory, storage, CPUs, and power supplies.
l Understanding of data center infrastructure, including rack systems, PDU, power distribution, cooling, and networking.
l Good English communication skills and ability to coordinate with local data center operators and hardware vendors.
l Strong troubleshooting, communication, and problem-solving skills.
l Ability to work independently and handle urgent operational incidents.
4. Preferred Qualifications
l Experience maintaining NVIDIA H100, H200, or other high-performance GPU servers.
l Experience with large-scale AI computing infrastructure or GPU clusters.
l Knowledge of InfiniBand, RoCE, Mellanox/NVIDIA ConnectX networking.
l Familiarity with Kubernetes, Docker, or Slurm.
l Experience with GIGABYTE, Supermicro, Dell, or HPE enterprise servers.
l Previous experience working in a colocation data center environment in Canada.
l Availability for occasional after-hours maintenance and emergency incident response.
📌 AIDC GPU Server Cluster Operations Engineer (Mississauga)
🏢 GLOBAL INNOVATIVE SOLUTIONS
📍 Mississauga