AI Inference Engineer

Details

Perplexity is an AI-powered answer engine founded in December 2022 and growing rapidly as one of the world’s leading AI platforms. Perplexity has raised over $1B in venture investment and aims to build accurate, trustworthy AI that powers decision-making and assistive AI. The team handles large-scale deployment of machine learning models for real-time inference across multiple locations (New York City; Palo Alto; San Francisco).

Our current stack includes: Python, Rust, C++, PyTorch, Triton, CUDA, Kubernetes. You will have the opportunity to work on large-scale deployment of machine learning models for real-time inference and implement LLM inference optimizations.

Responsibilities

• Develop APIs for AI inference that will be used by both internal and external customers
• Benchmark and address bottlenecks throughout the inference stack
• Improve the reliability and observability of systems and respond to system outages
• Explore novel research and implement LLM inference optimizations

Requirements

• Experience with ML systems and deep learning frameworks (e.g. PyTorch, TensorFlow, ONNX)
• Familiarity with common LLM architectures and inference optimization techniques (e.g. continuous batching, quantization)
• Understanding of GPU architectures or experience with GPU kernel programming using CUDA
• Practical experience with components of the stack is expected (Python, Rust, C++, PyTorch, Triton, CUDA, Kubernetes)

Compensation & Benefits

• Cash compensation range: $190,000 - $250,000 per year. Final offer amounts are determined by multiple factors, including experience and expertise, and may vary from the amounts listed above.
• Equity: In addition to the base salary, equity may be part of the total compensation package.
• Benefits: Comprehensive health, dental, and vision insurance for you and your dependents. Includes a 401(k) plan.