Tech Stack
Tag name is followed by "@" symbol and proficiency level value.
About proficiency levels:
- 1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;
- 3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;
- 7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;
- 10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.
AI
Communication @ 6
Data Science @ 8
Experimentation @ 6
Leadership @ 6
Machine Learning @ 6
Mentoring @ 4
Python @ 6
R @ 6
SQL @ 6
Statistics
- 1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;
- 3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;
- 7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;
- 10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.
Details
Reddit is seeking a senior technical leader to improve relevance measurement and evaluation across its Consumer organization. The role focuses on personalized content ranking, community discovery, search, recommendations, and retrieval across Reddit's large-scale user-generated content platform.
Responsibilities
- Serve as the technical authority on relevance metrics and evaluation methodology across Consumer, setting standards for measuring the quality of feeds, search results, and recommendations.
- Develop metrics frameworks and offline evaluation approaches for ranking and recommendation systems, including proxy metrics that predict long-term outcomes such as retention, community health, and user satisfaction.
- Design and analyze experiments for relevance features, accounting for spillover effects between communities, interference between contributors and consumers, and long-term impacts of ranking changes on content supply.
- Identify opportunities where improved measurement and analysis can unlock product insights around content quality, search intent understanding, and personalization effectiveness.
- Partner with ML engineers and product teams to translate model performance metrics into user-facing impact.
- Influence long-term product strategy for Feeds and Search by synthesizing experimentation, observational analysis, and metric deep-dives into actionable recommendations for senior leadership.
- Mentor and elevate data scientists on relevance evaluation, experimentation best practices for ranking systems, causal reasoning, and statistical rigor.
- Publish and share methodological advances internally and, where appropriate, externally.
Requirements
- Ph.D. in Statistics, Computer Science, Information Retrieval, Economics, or a related quantitative field focused on recommendation systems, ranking, causal inference, or evaluation methodology; or an M.S. with equivalent depth of expertise.
- M.S. holders require 12+ years of industry experience in applied science, data science, or relevance- and ranking-focused roles.
- Ph.D. holders require 8+ years of industry experience in applied science, data science, or relevance- and ranking-focused roles.
- Deep expertise in metrics design and evaluation for ranking and recommendation systems, including offline metrics and counterfactual evaluation.
- Strong understanding of causal inference and experimentation methodology, including novelty effects, position bias, long-term effect estimation, and ecosystem-level impacts.
- Experience defining and validating quality metrics for content ranking, search, or recommendations at scale.
- Strong theoretical grounding in experimental design, including power analysis, variance reduction techniques, and sequential testing as applied to relevance experiments.
- Expert knowledge of SQL and proficiency in R and/or Python for statistical computing.
- Ability to influence product and organizational strategy through data-driven insights about content quality and user experience.
- Excellent communication skills, including the ability to explain nuanced statistical and machine learning concepts and tradeoffs to technical and non-technical senior stakeholders.
- Experience mentoring data scientists and building organizational capability in relevance evaluation and experimentation.
- Comfortable working in innovative, fast-paced environments with a bias toward action.
Preferred Qualifications
- Published research or industry contributions in recommendation systems or causal inference for ranking.
- Experience with social network or user-generated content platforms where community-level dynamics create complex relevance and experimentation challenges.
Benefits
- Comprehensive healthcare benefits and income replacement programs.
- 401(k) with employer match.
- Global benefit programs supporting workspace, professional development, caregiving, and other needs.
- Family planning support.
- Gender-affirming care.
- Mental health and coaching benefits.
- Flexible vacation and paid volunteer time off.
- Generous paid parental leave.
Compensation
The base salary range is $232,500–$325,500 USD. The position is also eligible for equity in the form of restricted stock units and may be eligible for a commission depending on the position offered. Reddit provides U.S.-based employees with medical, dental, and vision insurance, a 401(k) program with employer match, vacation time, and parental leave. In select roles and locations, interviews may be recorded, transcribed, and summarized by artificial intelligence, with the option to opt out before scheduled interviews.