Staff Software Engineer - Databases, Tempo

📍 Germany
📍 Spain
📍 United Kingdom
📍 Ireland
📍 Sweden

EUR 94,000-112,800 per year

SENIOR

✅ Remote

Tech Stack
Tag name is followed by "@" symbol and proficiency level value. About proficiency levels:

1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;

3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;

7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;

10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.

AI @ 4 API @ 4 Communication @ 7 Grafana @ 4 Kubernetes @ 4 LLM @ 4 Observability @ 4 OpenTelemetry @ 4 Parquet @ 4 Prometheus @ 4 Rust @ 7 SQL @ 4 Security

Details

Grafana Labs is hiring a Staff Software Engineer to work on Tempo, the open-source distributed tracing backend behind Grafana Cloud Traces and Grafana Enterprise Traces. The role focuses on evolving Tempo into a scalable platform for observability products, AI-driven assistants, and external integrations.

Responsibilities

Lead multi-quarter technical initiatives from problem framing through rollout, including trace aggregation APIs, Limitless Tempo, autoscaling cells and customer limits, and query engine improvements.
Own the architecture of core Tempo components, including ingestion, storage, querying, and metrics generation.
Drive design reviews and make trade-offs involving performance, cost, and complexity.
Design structured, deterministic, discoverable APIs for Grafana products, LLM-driven assistants, and external integrators.
Drive operational excellence against SLOs such as P99 write latency, incident recurrence, and total cost of ownership per ingested GB.
Improve automation, parameterized rollouts, actionable alerts, and Zero Ops practices.
Partner with Product and sibling teams including App Observability, Asserts, Drilldown, and Grafana Assistant.
Mentor engineers through code reviews, design feedback, pairing, and technical writing.
Participate in on-call for services supported by the team and contribute to incident response and post-incident learning.
Contribute to the Tempo open-source project, review external contributions, and engage with the community.
Work on projects involving trace aggregation, TraceQL metrics, autoscaling, agent-scale ingestion and querying, query performance, multi-cell operations, and customer self-service configuration.

Grafana Labs supports the use of modern AI coding assistants for prototyping, test generation, refactoring, documentation, and incident follow-ups, subject to security guidelines and code review standards.

Requirements

Track record of leading complex, multi-quarter initiatives spanning design, delivery, and operations.
Substantial hands-on experience building and operating distributed data systems in production, such as ingestion pipelines, storage engines, or query execution systems.
Strong software craftsmanship, including the ability to write clean, robust, performant, and maintainable software.
Strong Go experience, or substantial experience with systems languages such as Rust, C, or C++ with a path to Go.
Experience owning production services, participating in on-call rotations, reducing operational toil, and managing services against SLOs.
Customer-focused and pragmatic approach with short feedback loops, MVP delivery, and iterative improvement.
Strong written and verbal communication skills, including experience leading through design documents, reviews, and shipped code in a remote, asynchronous environment.

Bonus Qualifications

Experience with tracing, OpenTelemetry, or large-scale observability systems.
Experience designing query languages, SQL- or TraceQL-like engines, or programmatically consumed APIs.
Experience with columnar storage formats such as Parquet or purpose-built on-disk analytical data formats.
Experience operating multi-tenant, multi-cell SaaS infrastructure on Kubernetes.
Experience building structured APIs, metadata or discovery endpoints, deterministic outputs, or evaluation harnesses for AI and LLM consumers.
Open-source contribution or maintainership experience.
Experience using Grafana, Prometheus, Loki, or Tempo on call or in a homelab.
Experience working in a fully remote, globally distributed team.

Compensation & Rewards

For Spain, the base compensation range is EUR 94,025–112,830 per year. Actual compensation may vary based on level, experience, and skillset. The role includes Restricted Stock Units (RSUs).

Benefits

100% remote, global culture.
In-person onboarding.
Global annual leave policy of 30 days per year, including three Grafana Shutdown Days where applicable.
Career growth pathways.
Transparent communication, empowered teams, and an open-source culture.