Senior Software Engineer - Hadoop Infrastructure

USD 160,000-240,000 per year

SENIOR

✅ On-site

Tech Stack
Tag name is followed by "@" symbol and proficiency level value. About proficiency levels:

1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;

3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;

7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;

10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.

Airflow @ 4 Ansible @ 4 Communication @ 7 Distributed Systems @ 4 Hadoop @ 4 Hive @ 6 Java @ 6 Jenkins @ 4 Jira @ 4 Linux @ 4 Maven @ 4 Spark @ 6

Details

Bloomberg's Hadoop Infrastructure team provides multi-tenant, observable, and highly available infrastructure backed by the open-source Apache Hadoop platform. The team supports large-scale data applications across Bloomberg, managing clusters with tens of petabytes of storage across thousands of servers, handling hundreds of billions of requests per day, and running tens of thousands of jobs across hundreds of thousands of tables.

Responsibilities

Advance how tenants from multiple product domains leverage Hadoop Infrastructure services.
Provide and improve capabilities to migrate and support a massive data and compute footprint spanning tens of petabytes to newer versions.
Improve the user experience for tenants securely interacting with underlying infrastructure frameworks.
Understand and improve the usability, reliability, and scalability of open-source Apache Hadoop services.
Optimize Hadoop infrastructure for the needs of Bloomberg application teams.
Contribute to open-source software and collaborate within a tight-knit engineering team.

Requirements

4+ years of demonstrated experience working with an object-oriented programming language, including Java and associated technologies such as Spring, JMX, and JDBC.
A degree in Computer Science, Engineering, or a similar field, or equivalent work experience.
3+ years of experience in the Hadoop ecosystem and related technologies, including HBase, Hive 3, HDFS, Spark, and Oozie.
Knowledge of modern development methodologies and tools, including Jenkins, Maven, and Jira.
Solid understanding of the Linux operating system, shell scripting, and operating system troubleshooting.
Strong problem-solving and communication skills.

Preferred Qualifications

Experience working with open-source software and communities.
Experience with distributed systems architecture and system design.
Knowledge of Ansible.
Experience with workflow automation tools such as Airflow and Argo.

Benefits

Benefits may include merit increases, incentive compensation for exempt roles, paid holidays, paid time off, medical, dental, and vision coverage, short- and long-term disability benefits, a 401(k) match, life insurance, and wellness programs. Benefits are not provided directly to contingent workers, contractors, or interns.