Used Tools & Technologies
Not specified
Required Skills & Competences
Tag name is followed by "@" symbol and proficiency level value.
About proficiency levels:
- 1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;
- 3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;
- 7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;
- 10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.
Software Development @ 7
Kafka @ 6
Kubernetes @ 8
Scala @ 4
Spark @ 4
Java @ 4
Distributed Systems @ 4
Flink @ 6
AWS @ 4
AWS EMR @ 4
Hadoop @ 4
Databricks @ 4
Hive @ 4
Oracle @ 4
CUDA @ 4
GPU @ 4
- 1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;
- 3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;
- 7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;
- 10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.
Details
NVIDIA is seeking a Distinguished Engineer for the Apache Spark Acceleration group. You will serve as a hands-on architect of the Nvidia Spark Acceleration Group, working with a team of distributed systems engineers (including PMC and committers of Apache Spark, Apache Hadoop, Apache Hive, and Apache Arrow) and engaging with open-source projects such as Apache Spark, RAPIDS, Apache Iceberg, Delta Lake, UCX and more. The goal is to accelerate Spark applications on GPUs without code changes and to enable GPU-accelerated data processing in on-premise and cloud deployments (AWS EMR, Databricks, Google Dataproc, Oracle Cloud Data Flow, Bytedance Volcengine, Tencent Cloud, Cloudera).
Responsibilities
- Lead the architecture, design and implementation of accelerated Apache Spark and related big-data frameworks.
- Engage with open source communities (Apache Spark, RAPIDS, Apache Iceberg, Delta Lake, UCX and others) for technical discussion and contribution; help expand NVIDIA's presence in new communities.
- Work with NVIDIA partners to deploy GPU-enabled data analytics solutions in public cloud and on-premises clusters.
- Present technical solutions at industry conferences and meetups.
- Collaborate with distributed systems teams to define large-scale distributed processing solutions.
- Provide recommendations and feedback on infrastructure, continuous integration and testing strategy.
- Build, test and optimize CUDA/C++ libraries across different platforms.
- Build automation and tools to increase productivity of teams developing distributed systems.
- Mentor members of the engineering team.
Requirements
- BS, MS, or PhD in Computer Science, Computer Engineering, or closely related field (or equivalent experience).
- 17+ years of work or research experience in software development.
- Prior experience delivering complex software projects as a lead architect.
- Outstanding technical skills in designing and implementing high-quality distributed systems.
- Excellent programming skills in C++, Java, and/or Scala.
- 5+ years working experience with key open-source big-data projects as a contributor or committer (examples given: Apache Spark, Apache Hadoop, Apache Flink, Apache Kafka, Apache Hive, Apache Arrow, Delta Lake).
- Excellent knowledge about distributed-system schedulers: Kubernetes, Hadoop YARN, Apache Spark.
- Able to delve into new areas quickly and work across teams and geographies.
Preferred / Ways to stand out
- Experience designing and developing columnar query engines.
- Committership at major open-source projects (Apache Spark, Apache Hadoop, Apache Flink).
- Working experience with acceleration libraries and ecosystems: CUDA, RAPIDS, UCX.
Compensation & Benefits
- Base salary range: 308,000 USD - 471,500 USD (determined based on location, experience, and internal pay equity).
- Eligibility for equity and company benefits (see NVIDIA benefits pages).
Additional information
- Location: Santa Clara, CA, United States.
- Employment type: Full time.
- Applications accepted at least until August 5, 2025.
- NVIDIA is an equal opportunity employer committed to fostering a diverse work environment.