Senior Software Engineer, AI Inference Systems

Details

We are seeking highly skilled and motivated software engineers to build AI inference systems that serve large-scale models with extreme efficiency. You will architect and implement high-performance inference stacks, optimize GPU kernels and compilers, drive industry benchmarks, and scale workloads across multi-GPU, multi-node, and multi-cloud environments. You will collaborate across inference, compiler, scheduling, and performance teams to advance accelerated computing for AI.

Responsibilities

• Contribute features to vLLM to support the newest models and NVIDIA GPU hardware features.
• Profile and optimize vLLM using speculative decoding, data, tensor, expert, and pipeline parallelism, and prefill-decode disaggregation.
• Develop, optimize, and benchmark GPU kernels using fusion, autotuning, and memory/layout optimization.
• Build and extend high-level DSLs and compiler infrastructure to improve kernel developer productivity and hardware utilization.
• Define and build inference benchmarking methodologies and tools.
• Contribute new benchmarks and NVIDIA submissions to the MLPerf Inference benchmarking suite.
• Architect scheduling and orchestration for containerized, large-scale inference deployments on GPU clusters across cloud platforms.
• Conduct and publish original research in ML systems, and integrate research ideas and prototypes into NVIDIA software products.

Requirements

• Bachelor's degree or equivalent experience in Computer Science, Computer Engineering, or Software Engineering with 7+ years of experience; alternatively, a master's degree with 5+ years of experience; or a PhD with a thesis and top-tier publications in ML systems, GPU architecture, or high-performance computing.
• Strong programming skills in Python and C/C++. Experience with Go or Rust is a plus.
• Strong knowledge of algorithms and data structures, operating systems, computer architecture, parallel programming, distributed systems, and deep learning theory.
• Knowledge of performance engineering in ML frameworks such as PyTorch and inference engines such as vLLM and SGLang.
• Familiarity with GPU programming and performance, including CUDA, memory hierarchy, streams, and NCCL.
• Proficiency with profiling and debugging tools such as Nsight Systems and Nsight Compute.
• Experience with Docker, Kubernetes, and Slurm, plus familiarity with Linux namespaces and cgroups.
• Excellent debugging, problem-solving, and communication skills, with the ability to work effectively in a fast-paced, cross-functional environment.

Preferred Qualifications

• Experience building and optimizing LLM inference engines such as vLLM and SGLang.
• Hands-on experience with ML compilers and DSLs, including Triton, TorchDynamo/Inductor, MLIR/LLVM, and XLA.
• Experience with GPU libraries and features such as CUTLASS, CUDA Graph, and Tensor Cores.
• Experience with containerization and virtualization technologies such as containerd, CRI-O, and CRIU.
• Experience with AWS, GCP, or Azure; infrastructure as code; CI/CD; and production observability.
• Contributions to open-source projects or publications, including GitHub pull requests, published papers, and related artifacts.

NVIDIA's mission is to advance AI research and development by creating technologies that enable people to benefit from AI. The team consists of experts in AI systems and performance optimization.

Compensation

For Poland, the base salary range is 292,500 PLN–507,000 PLN for Level 4 and 375,000 PLN–650,000 PLN for Level 5. Base salary is determined by location, experience, and the pay of employees in similar positions.