Senior AI Engineer

Details

Grafana Labs is seeking a Senior Engineer specializing in AI and automation to own the AI agent infrastructure and automation platform supporting Marketing Operations. The role involves building multi-agent architectures, LLM integrations, and backend services connecting AI models to internal and third-party data platforms.

This is a high-autonomy position responsible for defining the technical direction of the automation platform, including data models, API contracts, shared libraries, and reference architectures. The engineer will partner with Data Engineering, GTM Systems, Field Operations, Marketing, RevOps, and SDR teams to build scalable, self-service automation that reduces manual work and improves operational efficiency.

Responsibilities

Agentic Systems and AI Infrastructure

• Own the end-to-end development of multi-agent AI systems, including architecture, implementation, testing, deployment, and ongoing operations.
• Build modular, composable agentic systems using orchestration frameworks such as LangChain, CrewAI, Anthropic MCP, or similar technologies.
• Develop reusable agentic skills that can be invoked through Slack, dashboards, internal applications, and CLIs.
• Implement observability and feedback loops, including logging, performance metrics, prompt iteration, model evaluation, and cost management.
• Establish governance and compliance standards for AI workflows, including access controls, audit trails, PII handling, and human-in-the-loop escalation paths.

Systems Integration and Backend Services

• Build MCP servers, APIs, CLIs, and microservices connecting AI models to BigQuery, Slack, CRMs, email, calendars, analytics tools, and other business systems.
• Architect retrieval-augmented generation (RAG) data flows connecting LLMs to internal knowledge bases, customer data, and real-time business context.
• Build serverless or containerized services using GCP Cloud Functions and Cloud Run that scale with usage and integrate with Grafana's cloud infrastructure.

Automation and Workflow Enablement

• Partner with RevOps, Demand Generation, Regional Marketing, and SDR teams to identify high-impact automation opportunities and deliver measurable business outcomes.
• Design and deploy workflows using n8n, Workato, or custom platforms with CI/CD, testing, and production reliability standards.
• Build self-service systems supported by documentation, playbooks, and enablement materials so partner teams can operate independently.
• Use AI-assisted development tools such as Claude Code, Gemini CLI, OpenAI Codex, GitHub Copilot, and Cursor within security guidelines, while maintaining strong code review and quality standards.

Requirements

• 8+ years of software engineering experience, with depth in backend development, systems integration, or data and analytics engineering.
• 2+ years of hands-on experience applying LLMs or AI to production workflows, beyond prototypes.
• Strong proficiency in Python and JavaScript/Node.js.
• Experience with Git-based workflows, code reviews, and disciplined testing practices.
• Hands-on experience with prompt engineering, RAG, function calling and tool use, structured output parsing, and model evaluation.
• Experience building and operating multi-agent systems at scale, including agent decomposition, sequential chains, router or dispatcher patterns, parallel fan-out, state management, and production monitoring.
• Familiarity with Google Cloud Platform, BigQuery, and serverless or containerized services such as Cloud Functions and Cloud Run.
• Understanding of LLM failure modes and production mitigations, including confidence thresholds, fallback logic, human escalation, and cost and latency management.
• Ability to diagnose business problems before writing code and focus on workflows and outcomes.
• Ability to identify high-leverage problems, challenge low-impact requests, and deliver end-to-end with minimal direction.
• Clear technical communication skills, including the ability to explain complex systems to engineers and business stakeholders.

Bonus Qualifications

• Experience with vector databases or retrieval pipelines, including Pinecone, Weaviate, ChromaDB, Qdrant, or pgvector.
• Familiarity with Salesforce, Customer.io, HubSpot, Marketo, or Outreach.
• Experience with React or Slack Block Kit for building user-facing AI tool interfaces.
• Experience with AI observability tooling such as LangSmith, Weights & Biases, or custom evaluation frameworks.
• Experience with workflow orchestration platforms such as n8n, Temporal, Prefect, or Airflow.
• Familiarity with Model Context Protocol (MCP) or similar standards for connecting AI systems to data sources.
• Experience automating marketing, sales, or customer success workflows in a B2B SaaS environment.
• Active participation in open-source communities.

Benefits

• Restricted Stock Units (RSUs) are included with all roles.
• 100% remote, global work culture.
• Career growth pathways.
• In-person onboarding.
• Global annual leave policy of 30 days per annum, including 3 Grafana Shutdown Days, subject to local legislation.
• Transparent communication, empowered teams, and an innovation-driven environment.

Compensation

The United States base compensation range is USD $154,445–$185,334. Actual compensation may vary based on level, experience, and skillset. Compensation ranges are country-specific, and the recruiter will discuss the applicable market pay range and benefits for candidates applying from another location.

Grafana Labs is an equal opportunity employer.