Software Engineer, Model Evaluation Systems

Details

Groq delivers fast, efficient AI inference. Our LPU-based system powers GroqCloud™, giving businesses and developers the speed and scale they need. Headquartered in Silicon Valley, we are on a mission to make high performance AI compute more accessible and affordable. When real-time AI is within reach, anything is possible. Build fast.

Mission

Build and optimize model evaluation systems to ensure that AI models deployed on Groq’s platform achieve best-in-class quality, throughput, and efficiency. Develop benchmarking frameworks, optimize model performance, and drive improvements that push the boundaries of evaluation systems.

Responsibilities & Opportunities in this Role

• Software Development: Write simple, concise, high-performance code in Python and Golang to support evaluation pipelines and tooling.
• Model Evaluation & Benchmarking: Develop and implement benchmarking tools to assess the performance of models hosted on Groq, ensuring Groq’s AI infrastructure remains at the cutting edge of efficiency and performance.
• Automation & Tooling: Build automated evaluation systems and testing infrastructure to streamline model deployment and validation processes.
• System Integration & Testing: Work closely with cross-functional teams to integrate models into Groq’s infrastructure, ensuring seamless operation and accurate evaluation across various system configurations.
• Data-Driven Insights: Collect and analyze performance data, providing actionable insights and recommendations for improving model performance, scalability, and reliability.

Requirements

• Proven experience building or working with evaluation systems or working on novel applied AI projects.
• Proficiency in Python and Golang, with strong software engineering fundamentals and best practices.
• Self-Driven: Ability to see projects through end-to-end independently, taking initiative to optimize systems proactively.
• Deep problem-solving skills — candidates should be able to tackle non-trivial challenges with minimal guidance.
• Ability to work across unconventional AI problems, iterate quickly, and ship impactful solutions.
• Contributions to open-source projects or prior startup experience is a big plus.

Attributes of a Groqster

• Humility: Egos are checked at the door.
• Collaborative & Team Savvy: We make up the smartest person in the room, together.
• Growth & Giver Mindset: Learn it all versus know it all, we share knowledge generously.
• Curious & Innovative: Take a creative approach to projects, problems, and design.
• Passion, Grit, & Boldness: No limit thinking, fueling informed risk taking.

If this sounds like you, we’d love to hear from you!

Compensation

At Groq, a competitive base salary is part of our comprehensive compensation package, which includes equity and benefits. For this role, the base salary range is $154,445 to $208,955, determined by your skills, qualifications, experience, and internal benchmarks.