Host Systems Software Engineer

Details

About the Team

OpenAI’s Hardware organization develops AI-native silicon and system-level solutions for the unique demands of advanced AI workloads. Building on efforts like Jalapeño, the team is developing future generations of AI-native silicon and tightly integrated systems to power the next generation of frontier models. By co-designing chips, systems, tools, and methodologies, the team helps deliver faster, more efficient, and production-ready hardware for OpenAI’s supercomputing platform.

About the Role

We’re looking for an experienced systems software engineer to help define and build the host software stack for our custom next-generation AI systems. You will work close to the hardware on performance-critical software, including Linux kernel drivers, high-throughput I/O paths, and system-scale networking and RDMA.

This role spans architecture, implementation, platform bring-up, debugging, and performance optimization. You will work across hardware and software boundaries to make new systems usable end to end, from low-level device interfaces through userspace tooling and production validation.

Responsibilities

• Design, implement, and debug host-side systems software for AI infrastructure, including Linux kernel drivers and supporting userspace components.
• Build and optimize software paths for high-throughput, low-latency communication, including RDMA and related networking functionality.
• Develop software around PCIe, DMA, NICs, accelerators, memory movement, and device interaction.
• Bring up new hardware platforms and diagnose complex issues across kernel, firmware, networking, and hardware boundaries.
• Build tooling for integration, testing, diagnostics, observability, qualification, and performance characterization.
• Collaborate with hardware, networking, and platform teams to define interfaces and integrate new capabilities.
• Work with external vendors where needed to integrate technologies and drive issues to resolution.
• Contribute across the systems software stack as the platform and team evolve.
• Help shape the technical direction and engineering practices for a growing systems software stack.

Requirements

• Experience building low-level or performance-critical systems software.
• Strong programming skills in C or C++, with proficiency using Python and Linux tooling for automation and debugging.
• Strong Linux systems fundamentals and the ability to debug across hardware and software boundaries.
• Hands-on experience in at least one relevant area, such as Linux kernel drivers, kernel networking, RDMA, PCIe, DMA, NIC software, accelerator software, or high-performance I/O.
• Experience owning complex software projects from design through implementation, bring-up, and validation.
• Ability to thrive in ambiguity, work across subsystem boundaries, and build systems from scratch.
• Strong cross-functional communication skills.

Preferred Qualifications

• Experience developing Linux kernel drivers or other OS-level performance-critical components.
• Familiarity with RDMA or RoCE, ibverbs, kernel networking, or congestion-control concepts such as ECN and DCQCN.
• Experience with PCIe, DMA, peer-to-peer communication, SR-IOV, IOMMU, dma-buf, or related accelerator and I/O subsystems.
• Experience bringing up accelerators, NICs, SoCs, or custom hardware platforms.
• Experience profiling and optimizing high-throughput, low-latency systems.
• Familiarity with Rust or experience using Rust for systems programming.

Additional Information

To comply with U.S. export control laws and regulations, candidates for this role may need to meet certain legal status requirements as provided in those laws and regulations.