Software Engineer - Training/Inference (C++)

Details

SpaceXAI's mission is to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. The team is small, highly motivated, and focused on engineering excellence. It operates with a flat organizational structure, and employees are expected to be hands-on, contribute directly to the company's mission, communicate effectively, and demonstrate strong work ethic and prioritization skills.

Responsibilities

• Architect and implement scalable distributed infrastructure for model serving, including load balancing, auto-scaling, batch scheduling, and global KV cache.
• Optimize latency and throughput of model inference under real production workloads.
• Build reliable, high-concurrency serving systems with excellent tail latency and high availability.
• Benchmark, fine-tune, and accelerate inference engines, including low-level GPU kernel work and code generation.
• Develop tools to trace, replay, and resolve issues across the full stack, from orchestration to GPU kernels.
• Create robust CI/CD infrastructure for endpoint deployment, image publishing, and inference engine updates.
• Accelerate research on scaling test-time compute, reinforcement learning rollouts, and model-hardware co-design for next-generation systems.

Requirements

• Deep low-level systems programming experience in C, C++, or Rust.
• Experience with large-scale, high-concurrency production serving.
• Experience with GPU inference engines such as vLLM, SGLang, Triton, or TensorRT-LLM.
• Strong background in system optimizations, including batching, caching, load balancing, and parallelism.
• Experience with low-level inference optimizations, including GPU kernels and code generation.
• Experience with algorithmic inference optimizations, including quantization, speculative decoding, distillation, and low-precision numerics.
• Experience with testing, benchmarking, and reliability of inference services.
• Experience designing and implementing CI/CD infrastructure for inference.

Compensation And Benefits

• Base salary: $180,000-$440,000 USD per year.
• Equity.
• Comprehensive medical, vision, and dental coverage.
• 401(k) retirement plan.
• Short- and long-term disability insurance.
• Life insurance.
• Various other discounts and perks.

SpaceXAI is an equal opportunity employer.