Staff Backend Engineer - Second Horizon | US | Remote

📍 Canada
📍 United States
USD 175,000-210,000 per year
SENIOR
✅ Remote

Tech Stack

AI @ 4 API AWS @ 4 Audit Azure @ 4 BI @ 4 Communication @ 6 Data Engineering @ 4 Databricks @ 4 Experimentation GCP @ 4 GenAI Generative AI @ 4 Grafana @ 4 LLM Looker @ 4 Observability @ 4 Power BI @ 4 Prompt Engineering @ 4 Snowflake @ 4 System Administration Tableau @ 4

Details

Grafana Labs is building an AI-native data intelligence system that gives agents reliable, governed access to enterprise context, including data, metadata, definitions, lineage, quality signals, and institutional knowledge. The system is designed so that the context system retrieves information, agents decide, and data teams maintain the intelligence.

This remote role is open to candidates in the United States or Canada. Residents of Quebec are not eligible.

Responsibilities

  • Design, implement, test, and operate backend services for context ingestion, indexing, retrieval orchestration, API access, source configuration, and system administration.
  • Define and build a scalable, multi-tenant SaaS foundation, including tenant isolation, usage tracking, quotas, audit logs, background jobs, and reliable service boundaries.
  • Build APIs and service interfaces that enable AI agents, MCP tools, CLIs, and internal applications to retrieve relevant context, provenance, confidence signals, and warnings.
  • Make practical tradeoffs between rapid experimentation and long-term reliability as the project evolves from prototype to production.
  • Instrument services with metrics, logs, traces, alerts, and dashboards, and use observability tools to understand system behavior and improve reliability.
  • Help shape architecture, service boundaries, storage choices, API contracts, deployment patterns, and engineering practices for a new product area.
  • Communicate effectively and collaborate across teams in a dynamic environment.
  • Take ownership of AI solutions and ensure they are scalable, maintainable, and aligned with real user workflows.

Requirements

  • Strong experience building production-grade, user-facing software systems.
  • Familiarity with AI technologies and frameworks, with a practical focus on delivering high-quality solutions.
  • Experience with LLMs, prompt engineering, and building applications powered by generative AI.
  • A proven track record of delivering software that reached production and is actively used by customers or users.
  • Experience working in cloud-native environments such as AWS, GCP, or Azure.
  • Experience using observability tools to understand and troubleshoot system behavior.
  • Ability to iterate quickly, release prototypes, collect feedback, and work pragmatically through ambiguity.
  • Initiative, ownership, effective communication, openness to feedback, and a solutions-oriented mindset.

Nice to Have

  • Experience building or working with agent frameworks or multi-agent workflows.
  • Experience as a data analyst or working with data platforms such as Looker, Tableau, Power BI, Snowflake, or Databricks.
  • Experience building tools for data engineering.

Benefits

  • Equity and bonus, if applicable.
  • 100% remote work and a global culture.
  • Career growth pathways.
  • In-person onboarding.
  • A global annual leave policy of 30 days per annum, including 3 Grafana Shutdown Days, subject to local legislation.
  • Transparent communication, empowered teams, and an open-source-oriented culture.

Compensation

The US base compensation range is $174,986–$209,983. Actual compensation may vary based on level, experience, and skillset.

More jobs at Grafana Labs

Similar jobs