Software Engineer, Fleet Hardware Health

Details

The Fleet team supports the computing environment that powers OpenAI’s research and product development, including large-scale systems spanning data centers, GPUs, networking, and related infrastructure. The team focuses on availability, performance, efficiency, safety, reliability, and responsible AI deployment.

As a software engineer on the Fleet Hardware team, you will be responsible for the reliability and uptime of OpenAI’s compute fleet. You will investigate hardware and system failures at scale and develop automated solutions for detection, remediation, and improved infrastructure health.

Responsibilities

• Build and maintain automation systems for provisioning and managing server fleets.
• Develop tools to monitor server health, performance, and lifecycle events.
• Collaborate with clusters, networking, and infrastructure teams.
• Partner with external operators to ensure a high level of quality.
• Identify and fix performance bottlenecks and inefficiencies.
• Continuously improve automation to reduce manual work.
• Conduct comprehensive system-level investigations and develop automated solutions for detection and remediation at scale.

Requirements

• Experience managing large-scale server environments.
• A balance of strengths in building and operationalizing systems.
• Proficiency in Python, Go, or similar programming languages.
• Strong Linux, networking, and server hardware knowledge.
• Comfort working with noisy data using SQL, PromQL, Pandas, or other tools.
• Prior hardware expertise is not required.

Bonus Skills

• Experience with low-level details of hardware components, protocols, and associated Linux tooling, including PCIe, InfiniBand, networking, power management, and kernel performance tuning.
• Knowledge of hardware management protocols such as IPMI and Redfish.
• High-performance computing or distributed systems experience.
• Experience developing, managing, or designing hardware.
• Familiarity with monitoring tools such as Prometheus and Grafana.

Benefits

• Base salary of $230,000–$490,000 per year.
• Equity and performance-related bonuses for eligible employees.
• Medical, dental, and vision insurance, with employer contributions to Health Savings Accounts.
• Pre-tax accounts for health and dependent care expenses, plus commuter benefits.
• 401(k) retirement plan with employer match.
• Paid parental, medical, and caregiver leave.
• Paid time off, company holidays, office closures, and paid sick or safe time as required by law.
• Mental health and wellness support.
• Employer-paid basic life and disability coverage.
• Annual learning and development stipend.
• Daily meals in offices and eligible meal delivery credits.
• Relocation support for eligible employees.
• Additional benefits may include charitable donation matching and wellness stipends.

OpenAI is an equal opportunity employer and provides reasonable accommodations to applicants with disabilities. Background checks are administered in accordance with applicable law.