Senior AI Engineer

Details

Grafana Labs is seeking a Senior Engineer specializing in AI and automation to own the AI agent infrastructure and automation platform supporting Marketing Operations. The role involves building multi-agent architectures, LLM integrations, backend services, and production systems that connect AI models to internal and third-party data platforms.

This is a high-autonomy position responsible for identifying high-leverage problems across Marketing, RevOps, and SDR teams, defining the technical direction of the automation platform, and partnering with Data Engineering, GTM Systems, and Field Operations to build scalable, self-service automation.

The role is remote and open to candidates located in Canada. Residents of Quebec are not eligible.

Responsibilities

Agentic Systems and AI Infrastructure

• Own the end-to-end development of multi-agent AI systems, including architecture, implementation, testing, deployment, and ongoing operations.
• Build modular, composable agentic systems using orchestration frameworks such as LangChain, CrewAI, Anthropic MCP, or similar technologies.
• Develop reusable agentic skills for Slack, dashboards, internal applications, and command-line interfaces.
• Implement observability and feedback loops, including logging, performance metrics, prompt iteration, model evaluation, and cost management.
• Establish governance and compliance standards for AI workflows, including access controls, audit trails, PII handling, and human-in-the-loop escalation paths.

Systems Integration and Backend Services

• Build MCP servers, APIs, CLIs, and microservices connecting AI models to BigQuery, Slack, CRMs, email, calendars, and analytics tools.
• Architect retrieval-augmented generation (RAG) data flows connecting LLMs to internal knowledge bases, customer data, and real-time business context.
• Build serverless or containerized services using GCP Cloud Functions and Cloud Run that scale with usage and integrate with Grafana's cloud infrastructure.

Automation and Workflow Enablement

• Partner with RevOps, Demand Generation, Regional Marketing, and SDR teams to identify high-impact automation opportunities and deliver measurable business outcomes.
• Design and deploy workflows using n8n, Workato, or custom platforms with CI/CD, testing, and production reliability standards.
• Build self-service systems supported by documentation, playbooks, and enablement materials.

Developer Productivity

• Use AI coding assistants such as Claude Code, Gemini CLI, OpenAI Codex, GitHub Copilot, and Cursor within security guidelines.
• Apply strong code review and quality standards to AI-assisted development.

Requirements

• 8+ years of software engineering experience, with depth in backend development, systems integration, or data and analytics engineering.
• 2+ years of hands-on experience applying LLMs or AI to production workflows.
• Strong proficiency in Python and JavaScript/Node.js.
• Experience with Git-based workflows, code reviews, and software testing.
• Hands-on experience with prompt engineering, RAG, function calling and tool use, structured output parsing, and model evaluation.
• Experience building and operating multi-agent systems at scale, including agent decomposition, sequential chains, router or dispatcher patterns, parallel fan-out, state management, and production monitoring.
• Strong familiarity with Google Cloud Platform, BigQuery, Cloud Functions, and Cloud Run.
• Understanding of LLM failure modes and production mitigations, including confidence thresholds, fallback logic, human escalation, and cost and latency management.
• Ability to diagnose business problems, identify high-leverage opportunities, challenge low-impact requests, and deliver end-to-end with minimal direction.
• Clear technical communication skills, with the ability to explain complex systems to engineers and business stakeholders.

Bonus Qualifications

• Experience with vector databases or retrieval pipelines, including Pinecone, Weaviate, ChromaDB, Qdrant, or pgvector.
• Familiarity with Salesforce, Customer.io, HubSpot, Marketo, or Outreach.
• Experience with React or Slack Block Kit.
• Experience with AI observability tools such as LangSmith, Weights & Biases, or custom evaluation frameworks.
• Experience with n8n, Temporal, Prefect, or Airflow.
• Familiarity with the Model Context Protocol (MCP) or similar standards.
• Experience automating marketing, sales, or customer success workflows in a B2B SaaS environment.
• Active participation in open-source communities.

Compensation

The base compensation range in Canada is CAD 164,490–197,389. Actual compensation may vary based on level, experience, and skill set. The role also includes Restricted Stock Units (RSUs).

Benefits and Culture

• 100% remote, global work environment.
• In-person onboarding.
• 30 days of annual leave, including three Grafana Shutdown Days, subject to local legislation.
• Career growth pathways.
• Collaborative, transparent, and innovation-driven culture.
• Equal opportunity employment practices.