Senior Software Engineer - ClickPipes (CDC/Streaming)

Details

We are hiring for this role remotely in any country where ClickHouse has a hiring presence.

About the Team

The ClickPipes - Database Integrations team builds the platform that enables real-time data replication from databases into ClickHouse at petabyte scale.

You will solve complex database-related and distributed-systems problems, including database internals and snapshotting strategies, schema evolution during live replication, data-type compatibility across systems, low end-to-end latency under unpredictable loads, and durable execution frameworks for data consistency over unreliable networks. The team works in the open using and contributing to PeerDB, an open-source CDC platform.

Responsibilities

Build Data-Intensive Systems

• Design and develop high-throughput integrations with PostgreSQL, MySQL, MongoDB, Iceberg, Delta Lake, BigQuery, and Snowflake.
• Handle real-world production edge cases, including unconventional database setups, data-type internals, database upgrades and failovers, and large transactions.
• Design integration solutions that enable users to fully harness ClickHouse's performance and throughput.

Own End-to-End Reliability

• Debug complex production issues using runtime diagnostics such as pprof and Parca, along with metrics, logging, and tracing.
• Build and improve infrastructure and tools to increase system reliability, reduce incident response time, and simplify or automate operations.
• Write clear public and internal documentation.
• Participate in an on-call rotation.

Drive Product Innovation

• Work directly with customers to understand integration requirements and identify gaps in the existing product.
• Collaborate cross-functionally with internal teams to ensure operational efficiency.
• Lead technical discussions and influence product roadmaps.

Requirements

• 5+ years of industry experience building data-intensive software solutions.
• Proficiency in Go, or systems-programming experience with willingness to ramp up quickly in Go.
• Cloud-native experience deploying and operating services on at least one major cloud platform: AWS, Google Cloud Platform, or Microsoft Azure.
• Practical experience with Kubernetes.
• Strong problem-solving and production-debugging skills.
• Clear written communication, including design documents and code reviews, and verbal communication during technical discussions, customer calls, and incident response.

Bonus Qualifications

• Experience with database replication technologies, including CDC and logical replication.
• Experience with durable execution frameworks such as Temporal.
• Experience with data formats and protocols such as Avro, Parquet, and Protobuf.
• Experience with modern data-processing frameworks such as Kafka, Spark, and Flink.
• Experience maintaining or contributing to open-source software.

Benefits

• Flexible work environment at a globally distributed, remote-friendly company operating in over 25 countries.
• Employer contributions toward healthcare.
• Stock options for every new team member.
• Flexible time off in the United States and generous entitlement in other countries.
• USD 500 home-office setup allowance for remote employees.
• Opportunities to engage with colleagues at company-wide offsites.

ClickHouse provides equal employment opportunities and prohibits discrimination and harassment based on protected characteristics under applicable law.