Software Engineer - Synthetic Monitoring

Details

Grafana Labs is seeking a Software Engineer to join the Synthetic Monitoring squad. The team builds and operates Grafana Cloud Synthetic Monitoring, a multi-tenant distributed system used to proactively verify critical user journeys, APIs, and services.

Synthetic Monitoring is implemented in Go and runs on Kubernetes across AWS, GCP, and Azure. It executes billions of synthetic tests each month from probe locations worldwide and customer-operated private probes. The role is full-stack, covering backend services, execution pipelines, telemetry, and user-facing workflows.

Responsibilities

• Own features end-to-end across the stack, from API and data model design in Go through execution pipelines and TypeScript/React user workflows.
• Design and evolve systems, contributing to architectural decisions covering scheduling, probe lifecycle, telemetry ingestion, and API design.
• Build cross-product and AI-assisted workflows connecting Synthetic Monitoring to the wider Grafana Cloud platform.
• Develop governance and organization primitives that enable enterprise teams to manage large numbers of checks as code.
• Break down complex and ambiguous problems into incremental deliverables and iterate based on feedback.
• Ensure quality through testing, system observability, and documentation.
• Participate in the team’s on-call rotation and take ownership of production services.
• Use AI coding assistants for prototyping, test generation, refactoring, documentation, code review, and incident follow-ups, while maintaining strong code review and quality standards.

Technologies

• Go for backend services, APIs, and the synthetic monitoring agent
• TypeScript and React for the Synthetic Monitoring application
• Kubernetes across AWS, GCP, and Azure
• k6 as the execution engine for scripted and browser checks
• Grafana ecosystem technologies including Mimir, Loki, and Prometheus

Requirements

• Comfortable working in a remote-first company with strong written and spoken English communication skills.
• Collaborative, friendly, kind, and respectful approach, with the ability to contribute to discussions, disagree constructively, and commit to team decisions.
• Strong Go experience or a path to becoming proficient in Go. Production backend experience in Rust, Java, or C++ may also translate well. Backend experience limited to JavaScript or Node.js runtimes is not sufficient.
• Experience owning production services and working with operational responsibilities such as on-call rotations, reliability, and toil reduction.
• Systems-thinking ability, including reasoning about component interactions, data flow, failure modes, cost, and operational behavior.
• Pragmatic approach to development, with the ability to analyze, design, deliver MVPs, gather feedback, and iterate.
• Ability to work independently, design solutions, obtain feedback, and write prototypes.
• Understanding of a user-centered software development process and the ability to understand users’ contexts and goals.
• Natural and thoughtful use of AI tools in engineering workflows.

Bonus Points

• Experience designing or operating multi-tenant SaaS systems at scale.
• Experience with observability tools such as Grafana, Datadog, New Relic, or Sentry.
• Production Kubernetes experience.
• Familiarity with k6, Playwright, or browser automation at scale.
• Experience contributing to or maintaining open-source projects.

Benefits

• Equity and bonus, if applicable.
• 100% remote global culture.
• In-person onboarding.
• 30 days of annual leave per year, including 3 Grafana Shutdown Days, subject to local legislation.
• Career growth pathways, transparent communication, and an innovation-driven environment.

Compensation

The base compensation range in Germany is EUR 76,000–EUR 95,000 per year. Actual compensation may vary based on level, experience, and skill set. Compensation and benefits may differ for applicants applying from other listed locations.