Staff Backend Engineer, Datalake Platform

Details

Stripe is a financial infrastructure platform used by millions of businesses worldwide. The Datalake team builds and maintains foundational data access and governance infrastructure for safe, fast, and compliant access to Stripe's critical big data assets. The team supports developers, data engineers, analysts, ML and AI teams, security teams, and business users across the company.

The role focuses on a major architectural transition toward modern, open-source data infrastructure. You will help lead the migration to solutions such as the Apache Iceberg REST Catalog, define storage abstractions and authorization models, and shape systems that support petabytes of data and hundreds of production pipelines.

Responsibilities

• Architect the unified Iceberg platform and lead the technical design of a metastore service serving as the single source of truth for Iceberg table management across Spark, Trino, Flink, and PyIceberg.
• Define API contracts, authorization models, per-table credential vending, and integration patterns for company-wide data pipelines.
• Own the metastore migration strategy, including sequencing, backward compatibility, rollback planning, and coordination with consuming teams while maintaining production operations.
• Shape the object storage abstraction layer, including bucket provisioning, access-control policy design, and developer-facing client libraries.
• Lead compliance architecture by translating regulatory requirements into preventative technical controls such as audit logging, access review infrastructure, data segregation, and lifecycle enforcement.
• Drive cost and efficiency improvements at petabyte scale by addressing storage layout, snapshot retention, and data lifecycle management.
• Design automated, self-service tooling that reduces the need for ongoing manual intervention.
• Lead critical design reviews, establish reliability, security, and developer-experience standards, and mentor senior engineers through complex architectural decisions.

Requirements

Minimum Requirements

• 10+ years of professional software engineering experience.
• Demonstrated experience designing, building, and operating large-scale distributed storage or data infrastructure systems.
• Deep experience with object storage such as S3, Azure Blob, or equivalent, including IAM, access-control policy design, lifecycle management, and petabyte-scale operations.
• Experience leading complex, multi-quarter infrastructure projects end to end, including cross-team dependency management and migrations across many consuming teams.
• Strong background in authorization and access-control design for distributed data systems.

Preferred Requirements

• Deep expertise in Apache Iceberg, including table format internals, the REST Catalog specification, snapshot lifecycle management, compaction, and integration with Spark, Trino, Flink, and PyIceberg.
• Experience with compliance-sensitive infrastructure and regulatory frameworks such as SOX or ICFR, including translating audit and access-review requirements into preventative technical controls.
• Experience executing large-scale data migrations with attention to sequencing, blast-radius reduction, rollback, and data-integrity validation.
• Strong developer-experience sensibility, including the ability to build ergonomic, well-documented abstractions that reduce toil for dependent engineering teams.