Principal ML Solutions Architect - Token Factory

Details

Nebius is building a full-stack AI cloud platform for data and model training through production deployment. This position sits within Nebius Token Factory, a serverless platform for running and customizing open-source LLMs in production. Token Factory supports serverless inference and fine-tuning, including LoRA, full fine-tuning, and reinforcement fine-tuning, with optimizations such as speculative decoding, quantization, cache-aware routing, and dedicated endpoints.

The Principal ML Solutions Architect will serve as the senior technical authority for customers using Token Factory's serverless inference and fine-tuning platforms. The role includes designing and implementing optimized workflows, leading complex customer engagements, solving performance and quality challenges, mentoring Solutions Architects, and influencing the platform roadmap in collaboration with backend, product, and research teams. The position may be performed remotely from the United States.

Responsibilities

• Own complex, high-stakes customer engagements from architecture through production across multiple modalities, driving measurable business value.
• Optimize LLM inference at the framework and hardware level and codify best practices into reusable team playbooks.
• Lead supervised and reinforcement fine-tuning efforts to maximize model quality.
• Design and implement production-ready LLM solutions using Token Factory inference services.
• Provide expertise in prompt engineering, RAG architectures, model selection, and cost/performance trade-offs at scale.
• Partner with product, engineering, and research teams to identify customer needs, prototype platform features, and influence the roadmap.
• Guide customers from proof of concept to production with a focus on performance, reliability, and cost efficiency.
• Mentor Senior and mid-level Solutions Architects through reviews, enablement, and knowledge sharing.
• Represent Token Factory through talks, blog posts, and conferences.

Requirements

• 8+ years of experience in ML/AI systems, including at least 4 years focused on LLMs and generative AI.
• Demonstrated technical leadership, including ownership of ambiguous, high-impact problems and influence across teams and customers.
• Expert knowledge of the LLM ecosystem, including model architectures, fine-tuning approaches, and inference internals.
• Deep hands-on experience with inference optimization, including quantization, KV-cache management, batching, and routing.
• Experience running LLMs in production at scale, including deploying, operating, and debugging inference workloads at the framework level.
• Experience with LLM fine-tuning, including SFT, LoRA, data preparation and curation, and RL-based fine-tuning.
• Experience building task-specific benchmarks and offline/online evaluation pipelines, including LLM-as-a-judge setups.
• Experience with inference frameworks and libraries such as vLLM, SGLang, and TensorRT-LLM, including the ability to read, modify, and contribute to their internals.
• Experience deploying LLM-powered applications using OpenAI, Anthropic, or open-source model APIs.
• Strong Python programming skills.
• Excellent communication skills, with the ability to explain technical concepts to engineers and executives.

Preferred Qualifications

• Contributions to or maintainership of major open-source inference or ML projects, including vLLM, SGLang, or TensorRT-LLM.
• Published research, conference talks, or widely read technical writing in the LLM or serving space.
• Experience with multimodal AI models, including vision-language and speech models.
• Proficiency with Docker, Kubernetes, and infrastructure-as-code.
• Experience building or owning internal tooling and automation for ML workflows at scale.

Preferred Technical Stack

• Programming: Python
• ML frameworks and libraries: vLLM, TensorRT-LLM, SGLang, Transformers, OpenAI SDKs, Anthropic SDKs
• MLOps and DevOps: Kubernetes, Docker, Git
• Cloud platforms: AWS, SageMaker, Bedrock, GCP, Vertex AI, Azure, Azure ML

Benefits

• Base compensation of $208,000–$261,000 USD.
• 100% company-paid medical, dental, and vision coverage for employees and families.
• 401(k) plan with up to a 4% company match and immediate vesting.
• 20 weeks of paid parental leave for primary caregivers and 12 weeks for secondary caregivers.
• Remote work reimbursement of up to $85 per month for mobile and internet.
• Company-paid short-term disability, long-term disability, and life insurance.
• Career growth and learning opportunities.
• Flexibility and ownership.
• Collaborative and innovative culture.
• Opportunity to work on impactful AI projects.

Applicants must be authorized to work in the country in which they apply and must provide proof of employment eligibility as a condition of hire. Nebius is an equal opportunity employer.