Staff AI Engineer

USD 175,000-220,000 per year
SENIOR
✅ Remote

Used Tools & Technologies

GenAI

Required Skills & Competences

Docker @ 4 Grafana @ 4 Kubernetes @ 4 DevOps @ 4 Terraform @ 4 GCP @ 4 AWS @ 4 Azure @ 4 Experimentation @ 4 LLM @ 4 Compliance @ 4 Codex @ 4 Observability @ 4 Generative AI @ 4 AI @ 4 Prompt Engineering @ 4

Details

Grafana Labs builds observability tools and a managed Grafana Cloud to help users understand, respond to, and improve their systems. The Grafana AI teams develop AI-driven features that help users make sense of complex observability data, reduce toil, and surface meaningful signals from noisy environments. This is a remote opportunity for applicants in USA time zones only.

Responsibilities

  • Build and deliver AI solutions: take ownership of developing high-performance AI features to help users detect, triage, and resolve incidents using observability data and tools.
  • Rapid experimentation and iteration: prototype, test, and validate with real users; ship and evolve LLM- or agent-powered workflows for incident lifecycle management and automated analysis tasks.
  • Collaborate cross-functionally with data analysts, product managers, and designers to shape AI-driven product features and integrate agentic components with internal tools, alerting systems, runbooks, and developer workflows.
  • Utilize AI and automation tools to enhance product functionality and development workflows.
  • Communicate effectively across teams and take ownership to ensure solutions are scalable, maintainable, and aligned with real user workflows.
  • Use modern AI coding assistants and company-funded usage budgets; access to frontier models is provided (examples given: GPT-Codex 5/3, Claude Opus 4.6, Gemini 3 Pro).

Requirements

  • Strong software engineering background with experience building production software systems (backend and/or full stack).
  • Experience with LLMs, prompt engineering, and building applications powered by Generative AI.
  • Proven track record of delivering software into production that is actively used by users.
  • Exposure to cloud-native environments (e.g., AWS, GCP, Azure).
  • Experience using observability tools to understand and troubleshoot system behavior.

Bonus Points

  • Experience building or working with agent frameworks or multi-agent workflows.
  • Experience with infrastructure / devops tooling such as Kubernetes, Docker, Terraform or similar for deployments.
  • Familiarity with model fine-tuning techniques.
  • Experience building observability tooling.

Compensation & Rewards

  • Base compensation range in the United States: USD 174,986 - USD 220,000 (actual compensation may vary based on level, experience, and skillset).
  • All roles include Restricted Stock Units (RSUs). Benefits include equity, bonus (if applicable), and other benefits referenced on the company careers page.

Company & Work Environment

  • Grafana Labs is a 100% remote company with team members across 40+ countries. For this role, applicants should be in USA time zones.
  • The company emphasizes autonomy, transparency, open-source roots, and a culture of collaboration and experimentation.
  • In-person onboarding is provided. Annual leave: 30 days per annum with 3 days reserved for company shutdown days (compliance with local legislation where applicable).

Equal Opportunity & Recruitment

  • Grafana Labs is an equal opportunity employer and may utilize AI tools in its recruitment process while continuing manual review of CVs.