Senior Cloud Data Infrastructure Engineer
SCRAPED
Used Tools & Technologies
Not specified
Required Skills & Competences ?
Software Development @ 6 Go @ 4 Kafka @ 4 Kubernetes @ 4 Python @ 4 SQL @ 4 Spark @ 4 Statistics @ 4 GCP @ 4 Algorithms @ 4 Distributed Systems @ 6 Data Science @ 4 AWS @ 4 Azure @ 4 Communication @ 4 FastAPI @ 4 Debugging @ 7 OLAP @ 4 Pandas @ 4Details
Established in 2009, ClickHouse leads the industry with its open-source column-oriented database system, driven by the vision of becoming the fastest OLAP database globally. The company empowers users to generate real-time analytical reports through SQL queries, emphasizing speed in managing escalating data volumes. Enterprises globally, including Lyft, Sony, IBM, GitLab, Twilio, HubSpot, and many more, rely on ClickHouse Cloud. It is available through open-source or on AWS, GCP, Azure, and Alibaba.
The Cloud AutoScaling team is dedicated to implementing robust vertical and horizontal auto-scaling capabilities within the ClickHouse cloud environment. We seek exceptional software engineers to develop and maintain the auto-scaling infrastructure to transform ClickHouse into a fully functional server-less database solution. Collaborating closely with the core database team, we are actively working on evolving ClickHouse into a cloud-native database system. Additionally, we engage with other cloud teams to drive continuous improvements in cloud infrastructure for enhanced performance and scalability.
Responsibilities
- Build a cutting-edge Cloud Native platform on top of the public cloud.
- Improve the metrics pipeline and build algorithms to generate better autoscaling statistics and recommendations.
- Work on the autoscale and Kubernetes operator to support seamless Vertical and Horizontal Auto-scaling.
- Work closely with our ClickHouse core development team and other data plane teams, partnering with them to support auto-scaling use cases as well as other internal infrastructure improvements.
- Architecting and building a robust, scalable, and highly available distributed infrastructure.
Requirements
- 5+ years of relevant software development industry experience building and operating scalable, fault-tolerant, distributed systems.
- Experience building operators with Kubernetes, controller runtime.
- Production experience with programming languages like Go, C++.
- You are not a stranger to PagerDuty On-call, debugging things in production and are a strong problem-solver.
- Expertise with a public cloud provider (AWS, GCP, Azure) and their infrastructure as a service offering (e.g., EC2).
- Experience with Data Storage, Ingestion, and Transformation (Spark, Kafka or similar tools).
- You are passionate about solving data problems at Scale.
- Experience with Python (uv, rye, fastAPI) Data Science (Pandas, NumPy etc) is good to have.
- You have excellent communication skills and the ability to work well within and across engineering teams.
Benefits
- Flexible work environment - ClickHouse is a globally distributed company and remote-friendly. We currently operate in 20 countries.
- Healthcare - Employer contributions towards your healthcare.
- Equity in the company - Every new team member who joins our company receives stock options.
- Time off - Flexible time off in the US, generous entitlement in other countries.
- A $500 Home office setup if youβre a remote employee.
- Global Gatherings β We believe in the power of in-person connection and offer opportunities to engage with colleagues at company-wide offsites.