Staff Backend Engineer - Second Horizon

Details

Grafana Labs is building an AI-native data intelligence system to provide agents reliable, governed access to enterprise context. The new team will design and ship production services for context ingestion, context storage, retrieval APIs, and agent-facing integrations. This is an early-stage, high-autonomy role focused on pragmatic architecture, rapid iteration, and operating production SaaS systems. The role is remote and open to candidates in the U.S. or Canada (residents of Quebec are not eligible).

Responsibilities

• Design, implement, test, and operate core backend services for context ingestion, indexing, retrieval orchestration, API access, source configuration, and system administration.
• Define and build a scalable, multi-tenant SaaS foundation (tenant isolation, usage tracking, quotas, audit logs, background jobs, reliable service boundaries).
• Build APIs and service interfaces for agent-facing retrieval workflows (AI agents, MCP tools, CLIs, internal applications) delivering context, provenance, confidence signals, and warnings.
• Partner across product and infrastructure to balance fast experimentation with long-term reliability as the project moves from prototype to production.
• Instrument and operate services with metrics, logs, traces, alerts, and dashboards; use observability tools to understand behavior and improve reliability.
• Contribute to technical direction: architecture, service boundaries, storage choices, API contracts, deployment patterns, and engineering practices.
• Communicate effectively across a dynamic, cross-functional environment and take ownership of delivered AI solutions.

Requirements

• Experience with large language models (LLMs), prompt engineering, and building applications powered by Generative AI.
• Proven track record of delivering software that reached production and is actively used by users.
• Exposure to cloud-native environments (e.g., AWS, GCP, Azure).
• Experience using observability tools to understand and troubleshoot system behavior.
• Strong engineering skills building production-grade, user-facing software systems; self-starter comfortable with ambiguity and pragmatic decision-making.
• Comfortable with rapid iteration, prototyping, and collecting feedback.

Nice to have

• Experience building or working with agent frameworks or multi-agent workflows.
• Experience as a data analyst or working with data platforms (Looker, Tableau, PowerBI, Snowflake, Databricks).
• Experience building tools for data engineering.

Compensation and Benefits

• Base compensation range in Canada: CAD 186,368 - CAD 223,642 (actual compensation may vary by level, experience, and skillset).
• Benefits include equity, potential bonus (if applicable), and other benefits (link provided in original posting).
• 30 days per annum global annual leave policy; 3 days reserved for Grafana Shutdown Days. In-person onboarding is provided.

Other details

• Remote role; candidates must be located in the U.S. or Canada (Quebec residents not eligible).
• Grafana Labs is a 100% remote company with a global culture and strong open-source roots.