Senior Site Reliability Engineer - Observability And Telemetry Platform

Details

Site Reliability Engineering (SRE) at NVIDIA is an engineering discipline to design, build and maintain large scale production systems with high efficiency and availability using the combination of software and systems engineering practices. This is a highly specialized discipline which demands knowledge across different systems, networking, coding, database, capacity management, continuous delivery and deployment and open source cloud enabling technologies like Kubernetes and OpenStack. SRE at NVIDIA ensures that our internal and external facing GPU cloud services run maximum reliability and uptime as promised to the users and at the same time enabling developers to make changes to the existing system through careful preparation and planning while keeping an eye on capacity, latency and performance. SRE is also a mindset and a set of engineering approaches to running better production systems and optimizations. Much of our software development focuses on eliminating manual work through automation, performance tuning and growing efficiency of production systems.

As SREs are responsible for the big picture of how our systems relate to each other, we use a breadth of tools and approaches to tackle a broad spectrum of problems. Practices such as limiting time spent on reactive operational work, blameless postmortems and proactive identification of potential outages factor into iterative improvement that is key to both product quality and interesting dynamic day-to-day work. SRE's culture of diversity, intellectual curiosity, problem solving and openness is important to our success. Our organization brings together people with a wide variety of backgrounds, experiences and perspectives. We encourage them to collaborate, think big and take risks in a blame-free environment. We promote self-direction to work on meaningful projects, while we also strive to build an environment that provides the support and mentorship needed to learn and grow.

Responsibilities

• Design, implement and support operational and reliability aspects of a large-scale Observability & Telemetry collection platform with a focus on performance at scale, real-time monitoring, logging and alerting.
• Engage in and improve the whole lifecycle of services — from inception and design through deployment, operation and refinement.
• Support services before they go live through system design consulting, developing software tools, platforms and frameworks, capacity management and launch reviews.
• Maintain services once live by measuring and monitoring availability, latency and overall system health.
• Scale systems sustainably through automation and evolve systems by pushing for changes that improve reliability and velocity.
• Practice sustainable incident response and blameless postmortems.
• Participate in an on-call rotation to support production systems.

Requirements

• BS degree in Computer Science or a related technical field involving coding (e.g., physics or mathematics), or equivalent experience.
• 5+ years of experience with infrastructure automation, distributed systems design, and developing tools for running large-scale private or public cloud systems in production.
• 8+ years of experience delivering foundational infrastructure and observability platforms.
• Experience in one or more programming languages: Python, Go, Perl or Ruby.
• In-depth knowledge of Linux, networking and containers.
• Familiarity with cloud enabling technologies such as Kubernetes and OpenStack.
• Experience and understanding of monitoring, logging, alerting, capacity management, continuous delivery and deployment.

Ways To Stand Out

• Interest in crafting, analyzing and fixing large-scale distributed systems.
• Systematic problem-solving approach, strong communication skills, ownership and drive, ability to debug and optimize code and automate routine tasks.
• Experience using or running large private and public cloud systems based on Kubernetes, OpenStack and Docker.
• Experience running Grafana, OpenTelemetry, Prometheus, and similar observability-focused tools.

Compensation & Other Details

• Base salary range (Level 4): 168,000 USD - 270,250 USD.
• Base salary range (Level 5): 208,000 USD - 333,500 USD.
• You will also be eligible for equity and benefits.
• Applications for this job will be accepted at least until October 17, 2025.
• NVIDIA is an equal opportunity employer and values diversity in its workforce.