Senior Storage Production Engineer - DGX Cloud

at Nvidia

USD 176,000-333,500 per year

SENIOR

✅ On-site

Tech Stack
Tag name is followed by "@" symbol and proficiency level value. About proficiency levels:

1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;

3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;

7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;

10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.

AI Algorithms @ 6 Ansible @ 4 Bash @ 4 CI/CD @ 4 Chef @ 4 Communication @ 6 Compliance Data Structures @ 6 Debugging @ 7 GPU Git @ 4 Go @ 4 Grafana @ 4 HPC IaC Java @ 4 Kubernetes @ 4 Linux @ 6 Machine Learning Networking @ 4 Observability @ 4 OpenStack @ 4 Performance Optimization Prometheus @ 4 Puppet @ 4 Python @ 4 Security Terraform @ 4

Details

Production engineering involves crafting, building, and maintaining large-scale production systems with high efficiency and availability. This role focuses on storage architecture, high-performance distributed storage, data management, systems, networking, coding, database management, continuous delivery and deployment, Kubernetes, containers, and virtualization.

Storage Production Engineers at NVIDIA ensure that internal and external-facing GPU cloud services meet reliability and uptime goals while enabling safe system changes. The role emphasizes storage operations automation, data access efficiency, storage performance optimization, proactive monitoring, automated fault detection and remediation, scalable data redundancy, and intelligent caching for HPC and AI/ML workloads.

Responsibilities

Design, implement, and support large-scale storage clusters for scalability, high availability, and data integrity.
Develop and maintain storage monitoring, logging, and alerting systems for proactive detection and resolution of performance issues.
Improve storage architectures for AI/ML workloads through low-latency access, efficient caching, and high-throughput performance.
Manage the lifecycle of storage services from inception and design through deployment, operation, and continuous optimization.
Support storage services through system build consulting, automation framework development, capacity management, and launch reviews.
Maintain production storage infrastructure by monitoring availability, latency, and system health, using predictive analytics and AI-driven automation.
Optimize storage efficiency through compression, deduplication, tiering strategies, and intelligent workload placement.
Scale storage systems using AI/ML-driven automation, policy-based tiering, and dynamic data migration techniques.
Implement encryption, access controls, and auditing mechanisms to ensure data security and compliance.
Participate in sustainable incident response and blameless root cause analysis.
Participate in an on-call rotation supporting storage and production systems.

Requirements

Bachelor's degree or equivalent experience in Computer Science, Storage Systems, or a related technical field, with 8+ years of practical experience.
Experience with distributed and high-performance storage solutions, including clustered and parallel file systems, distributed object storage, and enterprise-grade storage systems.
Solid understanding of block, file, and object storage technologies, including their scalability, reliability, performance characteristics, and standard processes.
Experience with storage networking protocols such as NFS, SMB, iSCSI, S3, Fibre Channel, RDMA, and NVMe over Fabrics.
Expertise in algorithms, data structures, complexity analysis, software design, and automating maintenance of large-scale Linux-based storage systems.
Experience with one or more of C/C++, Java, Python, Go, NodeJS, and Bash for storage automation, monitoring, and performance tuning.
Hands-on experience with infrastructure configuration management tools such as Ansible, Chef, Puppet, and Terraform.
Experience with observability and tracing tools such as InfluxDB, Prometheus, Grafana, and the Elastic stack.
Excellent written and oral communication skills, work ethic, teamwork, quality focus, and commitment to completing tasks.

Preferred Qualifications

Deep understanding of distributed storage systems, replication strategies, and erasure coding techniques.
Experience in capacity planning, performance tuning, and troubleshooting high-throughput storage systems.
Experience with Git, code review, pipelines, and CI/CD for infrastructure as code.
Experience analyzing and improving distributed storage system performance at scale.
Strong debugging and systematic problem-solving skills for identifying sophisticated storage issues.
Understanding of network protocols, architectures, and troubleshooting techniques related to storage performance, stability, and availability.
Experience operating private and public cloud storage solutions based on Kubernetes, OpenStack, or hybrid cloud architectures.
Ability to design and implement automated storage migration, backup, and disaster recovery strategies.
Ability to collaborate with various teams and adapt to emerging storage technologies.