Senior AI Engineer - Grafana Ops, AI/ML

Details

Grafana Labs is a remote-first, open-source company building observability tools used by millions globally. The Grafana AI teams focus on helping users make sense of complex observability data through AI-driven features that reduce toil, surface meaningful signals, and assist with incident response and automation.

This role is remote within the United States (applicants must be in USA time zones).

Responsibilities

• Build and deliver AI solutions that help users detect, triage, and resolve incidents using observability data and tools.
• Rapid experimentation and iteration: prototype, test, validate with real users, and evolve LLM- or agent-powered workflows for incident lifecycle management and automated analysis tasks.
• Collaborate cross-functionally with data analysts, product managers, and designers to shape AI-driven product features and integrate agentic components with internal tools, alerting systems, runbooks, and developer workflows.
• Utilize AI and automation tools to enhance product functionality and developer workflows.
• Communicate effectively in a dynamic, collaborative environment and take ownership of the AI solutions you develop, ensuring they are scalable, maintainable, and aligned with user workflows.

Requirements

• Experience with LLMs, prompt engineering, and building applications powered by Generative AI.
• Proven track record of delivering production software (backend and/or full stack) that is actively used by users.
• Exposure to cloud-native environments (e.g., AWS, GCP, Azure).
• Experience using observability tools to understand and troubleshoot system behavior.
• Strong software engineering skills, ability to iterate quickly, and a pragmatic focus on shipping and scaling features.
• Ability to work autonomously, drive projects, handle ambiguity, and collaborate effectively with cross-functional teams.

Bonus Points

• Experience building or working with agent frameworks or multi-agent workflows.
• Experience with infrastructure / DevOps tooling: Kubernetes, Docker, Terraform, or similar for deployments.
• Familiarity with model fine-tuning techniques.
• Experience building observability tooling.

Compensation & Rewards

• Base compensation range in the United States: USD 154,445 - USD 185,334 (actual compensation may vary by level, experience, and skillset).
• Roles include Restricted Stock Units (RSUs), potential bonus (if applicable), and other benefits.

Why You’ll Thrive at Grafana Labs

• Remote-first, global culture with in-person onboarding.
• High-autonomy, innovation-driven environment with transparent communication and clear career growth pathways.
• Open-source roots and collaborative teams.
• generous leave policy and other benefits as detailed by the company.

Additional Notes

• Grafana Labs may use AI tools in the recruitment process to assist in matching CVs to job postings; applications are also reviewed manually.
• Applicants should be located in USA time zones for this role.