Software Engineer - Data Platform

USD 180,000-440,000 per year

MIDDLE

✅ On-site

Tech Stack
Tag name is followed by "@" symbol and proficiency level value. About proficiency levels:

1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;

3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;

7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;

10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.

API Communication @ 6 Debugging @ 6 Distributed Systems @ 3 Experimentation Flink @ 3 Go @ 5 Hadoop @ 3 Kafka @ 3 Machine Learning Observability Performance Optimization @ 6 Profiling @ 6 Rust @ 5 Scala @ 5 Spark @ 3 Trino @ 3

Details

SpaceXAI's Data Platform team builds and operates the infrastructure responsible for large-scale data transport and processing across the company. The team owns core systems including Apache Kafka, HDFS, Spark, Flink, and Trino, enabling real-time machine learning pipelines, feed ranking, experimentation, analytics, and observability at petabyte scale.

The role focuses on designing, building, and operating distributed systems that power data movement and compute. The systems process trillions of events daily and require scalability, performance, fault tolerance, and reliability.

Responsibilities

Design and implement high-throughput, low-latency data ingestion and transport systems.
Scale and optimize multi-tenant Kafka infrastructure supporting real-time workloads.
Extend and tune Spark, Flink, and Trino for demanding production pipelines.
Build interfaces, APIs, and pipelines enabling teams to query, process, and move data at petabyte scale.
Debug and optimize distributed systems, with a focus on reliability and performance under load.
Collaborate with machine learning, product, and infrastructure teams to unblock critical data workflows.

Requirements

Proven expertise in distributed systems, stream processing, or large-scale data platforms.
Proficiency in Rust, Go, Scala, or similar systems languages.
Hands-on production experience with Kafka, Flink, Spark, Trino, or Hadoop.
Strong debugging, profiling, and performance optimization skills.
Track record of shipping and maintaining critical infrastructure.
Comfortable working in fast-moving, high-stakes environments with minimal guardrails.
Strong communication skills and the ability to concisely and accurately share knowledge with teammates.