Used Tools & Technologies
Not specified
Required Skills & Competences
Tag name is followed by "@" symbol and proficiency level value.
About proficiency levels:
- 1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;
- 3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;
- 7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;
- 10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.
Software Development @ 8
Scala @ 4
Spark @ 4
ETL @ 4
Java @ 4
Distributed Systems @ 4
AWS @ 4
AWS EMR @ 4
Parquet @ 4
Presto @ 3
JSON @ 4
Databricks @ 4
OSS @ 4
Oracle @ 4
CUDA @ 4
Cloud Computing @ 4
GPU @ 4
AI @ 4
- 1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;
- 3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;
- 7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;
- 10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.
Details
NVIDIA is seeking a Sr. Principal Systems Software Engineer for the Apache Spark Acceleration group. GPU-accelerated data processing has moved from proof of concept to production deployments. Enterprises now recognize the need for accelerated computing to handle large data processing needs. Multi-node GPU deployments will reduce cloud computing costs and lower latency batch ETL workloads.
At NVIDIA, we have been invested in accelerating Apache Spark, providing an open source plugin for Apache Spark. Apache Spark is the most popular data processing engine in data centers. We strive to accelerate Spark applications on GPUs without any code changes. Our OSS RAPIDS Spark library is integrated with on-premise and cloud services such as AWS EMR, Databricks, Google Dataproc, Oracle Cloud Data Flow, Bytedance Volcengine, Tencent Cloud and Cloudera. You will work with a team that uses open source libraries like cuDF to accelerate reading, writing and batch data operations in Spark.
Responsibilities
- Develop Java, Scala and CUDA/C++ libraries to accelerate DataFrames and I/O operations on common file formats such as Parquet, ORC and JSON.
- Enable interoperability with table formats such as Apache Iceberg and Delta Lake, and metastores such as Unity Catalog.
- Work with open source communities to enhance libraries like NVIDIA cuDF, CCCL and UCX through technical discussion and code contributions.
- Collaborate with distributed systems teams to craft solutions to distributed processing problems and challenges at large scale.
- Provide recommendations and feedback to teams regarding decisions surrounding infrastructure, continuous integration and testing strategy.
- Build, test and optimize CUDA/C++ libraries across different platforms.
Requirements
- BS, MS, or PhD in Computer Science, Computer Engineering, or a closely related field (or equivalent experience).
- 15+ years of work experience in software development.
- Outstanding technical skills in designing and implementing high-quality distributed systems.
- Excellent programming skills in C++, Java, and/or Scala.
- Ability to work with teams across organizational boundaries and geographies.
- Familiarity with the open source data platform ecosystem (Apache Spark, Velox, Presto, Apache Arrow, Apache DataFusion, etc.). Meaningful contributions to the OSS community are a plus.
- Highly motivated with strong interpersonal skills.
- Database query optimization is a strong plus.
Compensation and Benefits
- Base salary range: 272,000 USD - 431,250 USD (determined based on location, experience, and pay of employees in similar positions).
- Eligible for equity and company benefits.
Additional Information
- Applications for this job will be accepted at least until July 19, 2026.
- NVIDIA uses AI tools in its recruiting processes.
- NVIDIA is an equal opportunity employer committed to diversity and inclusion.