Backend Engineer - Platform - Stacks

Details

Responsibilities

• Design, build, and operate reconciliation systems, including the SSS backend, to track desired stack state, detect and repair drift across stack templates, grafana.com state, Hosted Grafana, and actual customer stack configuration
• Collaborate across SSS, grafana.com, and deployment configurations to ensure stack lifecycle workflows remain reliable, observable, and resilient
• Improve operational efficiency by reducing deployment complexity (e.g., aiming for single PR regional SSS deployment) and contributing to the Stack Config Reconciliation project
• Manage rollout mechanisms for provisioned plugins, dashboards, data sources, Grafana versions, release channels, and stack-level configuration
• Support new region and cluster rollouts, including the operational paths required to bring stacks online safely in new Grafana Cloud regions
• Improve incident response and recovery paths for stack misalignment, reconciliation failures, plugin rollout issues, and Hosted Grafana integration failures
• Partner with Product, Hosted Grafana, Infrastructure, Support, and adjacent AppCore squads on customer-impacting stack lifecycle work
• Contribute to roadmap planning, technical design, OnCall improvements, and long-term simplification of stack operations
• Help own the production behavior of the systems you build, including improving runbooks, dashboards, alerts, reconciliation safety, rollout controls, and recovery procedures; be comfortable debugging across service boundaries and making careful changes in systems that affect customer stacks

Requirements

• At least 1 year of fully remote work experience
• Some experience working on a SaaS platform and familiar with common distributed systems concepts (e.g., scalability, multi-tenancy, HA)
• Professional experience with Golang and willingness to work across both backend service and application code
• Care deeply about developer and user experience and the quality of the products you work on
• Some experience contributing to delivery of projects from initial brainstorming to shipping a product to the customer
• Write clean, well-tested software that other engineers can understand, operate, and maintain
• Ability to take on well-defined tasks, break them down, execute iteratively, and gather feedback
• Willingness to collaborate across teams and ensure work aligns with the needs of other squads and external stakeholders
• Familiarity with Kubernetes in AWS, GCP, or Azure, and exposure to infrastructure-as-code tooling (Helm, Terraform, Jsonnet, etc.)
• Experience participating in blameless incident response and contributing to post-incident reviews

Bonus Points For

• Experience with TypeScript/Node.js
• Experience with Kubernetes control-plane patterns, operators, reconcilers, or desired-state systems
• Experience with Jsonnet/Tanka, Terraform, Flux, Argo, or similar deployment/configuration tooling
• Experience working on SaaS provisioning, tenancy, regional expansion, plugin rollout, or customer lifecycle systems
• Experience with incident response involving configuration drift, partial failure, or cross-service state mismatch

Benefits

• 100% Remote, Global Culture
• Scaling Organization
• Transparent Communication
• Innovation-Driven
• Open Source Roots
• Empowered Teams
• Career Growth Pathways
• Approachable Leadership
• Passionate People
• In-Person onboarding (to help thrive from day 1)
• Balance is Key: global annual leave policy of 30 days per annum, with 3 days reserved for Grafana Shutdown Days
• Restricted Stock Units (RSUs): included in all roles, providing ownership in Grafana Labs' success