Staff Backend Engineer - Grafana Second Horizon

Details

Grafana Labs is hiring a Staff-level Backend Engineer to build the first production services for a context layer management system as part of a skunkworks initiative to bring observability to the rest of the business.

This is a remote opportunity, and applicants located in Spain, Sweden, UK, Ireland or Germany are of interest.

Responsibilities

• Build the core backend services: design, implement, test, and operate the first services for context ingestion, context indexing, retrieval orchestration, API access, source configuration, and system administration.
• Create a scalable SaaS foundation: define and build architecture for a multi-tenant service, including tenant isolation, usage tracking, quotas, audit logs, background jobs, and reliable service boundaries.
• Power agent-facing retrieval workflows: build APIs and service interfaces that allow AI agents, MCP tools, CLIs, and internal applications to retrieve relevant context, provenance, confidence signals, and warnings.
• Work across product and infrastructure: partner with the team to make practical tradeoffs between fast experimentation and long-term reliability as the project moves from prototype to production.
• Operate what you build: instrument services with metrics, logs, traces, alerts, and dashboards; use observability tools to understand system behavior and improve reliability.
• Contribute to technical direction: help shape architecture, service boundaries, storage choices, API contracts, deployment patterns, and engineering practices for a new product area.
• Effective communication: communicate effectively in a highly dynamic and collaborative environment and contribute across teams.
• Ownership and impact: take full ownership of the AI solutions developed, ensuring they are innovative, scalable, maintainable, and aligned with real user workflows.

Requirements

• Strong engineering skills with solid experience building production-grade, user-facing software systems.
• Proven initiative and the ability to tackle complex engineering problems, deal with ambiguity, and define scope where things are loosely defined.
• AI experience with a practical mindset, including familiarity with AI technologies and frameworks.
• Experience with LLMs, prompt engineering, and building applications powered by GenAI.
• Proven track record of delivering software that made it into production and is actively used by users.
• Exposure to working in cloud-native environments (AWS, GCP, Azure).
• Experience using observability tools to understand and troubleshoot system behavior.
• Quick iteration and experimentation: comfortable releasing prototypes, collecting feedback, and iterating pragmatically.
• Collaborative attitude: communicate effectively with peers and bring a solutions-oriented mindset.

Bonus Points for

• Experience building or working with agent frameworks or multi-agent workflows.
• Experience as a data analyst or working with data platforms (Looker, Tableau, PowerBI, Snowflake, DataBricks).
• Experience building tools for data engineering.

Compensation & Rewards

In the United Kingdom, the base compensation range for this role is GBP 103,958 - GBP 124,750. Actual compensation may vary based on level, experience, and skillset as assessed in the interview process. Benefits include equity, bonus (if applicable), and other benefits listed on Grafana’s careers page.

How You’ll Thrive

• Remote-first team with regular video meetings and most work asynchronous in writing.
• In-person onboarding.
• Global annual leave policy of 30 days per annum, with 3 days reserved for Grafana Shutdown Days.