People Research Data Scientist, AI Fairness & Bias

Details

OpenAI’s People Analytics team helps leaders make rigorous, evidence-based talent decisions and ensures that the systems supporting those decisions are valid, reliable, fair, and accountable.

As a People Data Scientist focused on AI fairness and bias testing, you will help establish how OpenAI evaluates AI-assisted People systems and high-impact talent processes. You will design and conduct rigorous assessments to identify, measure, and mitigate potential bias across the lifecycle of models, agents, decision-support tools, and automated workflows.

Your work will span the employee lifecycle, including hiring, performance, promotion, employee development, and workforce planning. You will evaluate technical systems and broader human-AI decision processes, examining model performance, data quality, measurement validity, differential outcomes, human oversight, and unintended consequences.

The role is preferred to be based in San Francisco, California.

Responsibilities

• Define and lead fairness and bias-testing strategies for AI-assisted People processes, models, agents, and decision-support systems from development through deployment and ongoing monitoring.
• Design algorithmic audits and validation studies, including adverse-impact analysis, subgroup and intersectional evaluation, error-rate analysis, calibration, measurement invariance, reliability, criterion-related validity, and sensitivity testing.
• Identify appropriate fairness criteria for each use case and evaluate tradeoffs among competing definitions of fairness.
• Document assumptions, limitations, residual risks, and the strength of evidence for each approach.
• Evaluate end-to-end human-AI decision systems, including model outputs, user behavior, human overrides, escalation pathways, and whether AI assistance changes decision quality, consistency, or equity.
• Develop evaluation approaches for generative and agentic AI, including test-set design, counterfactual testing, behavioral evaluation, human-rating studies, robustness testing, and analysis of disparate performance across populations and contexts.
• Investigate sources of observed disparities, including data representation, label and measurement bias, proxy variables, model design, decision thresholds, workflow design, and differential adoption or usage.
• Partner with engineering, People Operations, Legal, Privacy, Security, and People Systems teams to recommend and evaluate mitigations, including data improvements, model changes, threshold adjustments, workflow redesign, monitoring controls, and additional human oversight.
• Build scalable fairness-evaluation infrastructure, including reusable datasets, automated validation pipelines, regression tests, monitoring systems, self-service tools, and standardized reporting.
• Establish research and documentation standards for fairness test plans, dataset and model documentation, validation reports, limitations, monitoring plans, and decision records.
• Translate complex findings into concise, decision-ready narratives for technical teams, senior leaders, and other stakeholders.

Requirements

• Deep expertise in algorithmic fairness, bias measurement, responsible AI, psychometrics, applied statistics, or evaluation of high-impact decision systems.
• Strong research design, measurement, experimentation, causal inference, and statistical modeling skills.
• Hands-on experience with subgroup and intersectional analysis, adverse-impact testing, equalized-odds and equal-opportunity analysis, demographic-parity assessment, calibration analysis, counterfactual testing, measurement invariance, reliability analysis, and validation studies.
• Strong judgment regarding the limitations of fairness metrics and the ability to select measures appropriate to a particular decision context.
• Experience evaluating machine-learning models, generative AI systems, agents, or human-AI workflows using quantitative and qualitative evidence.
• High proficiency in Python or R and SQL, including experience working with complex, sensitive, and imperfect datasets.
• Experience building reproducible evaluation pipelines, automated testing frameworks, analytical tools, monitoring systems, or governed research workflows.
• Ability to distinguish statistical disparities from potential causes and communicate findings without overstating certainty or making unsupported causal or legal conclusions.
• Ability to work effectively with technical, operational, legal, privacy, and executive stakeholders.
• Deep curiosity, intellectual humility, attention to detail, and commitment to developing AI systems and organizational processes that work well for people across different backgrounds and circumstances.

Preferred Qualifications

• Experience conducting fairness assessments, algorithmic audits, model-risk reviews, adverse-impact analyses, or validation studies in employment or another high-impact domain.
• Familiarity with Fairlearn, AI Fairness 360, responsible-AI evaluation frameworks, explainability methods, or comparable internal tooling.
• Experience evaluating large language models, generative AI systems, safety classifiers, or agentic workflows, including behavioral testing and human evaluation.
• Experience with employment selection, talent assessment, psychometrics, organizational research, or validation of hiring, performance, promotion, or workforce decisions.
• Familiarity with responsible-AI frameworks and emerging requirements related to automated employment decision systems, algorithmic auditing, data privacy, and AI governance.
• Experience creating model cards, dataset documentation, fairness scorecards, audit reports, monitoring plans, or other review artifacts for high-impact systems.
• Advanced degree in Quantitative Psychology, Computer Science, Statistics, Economics, Data Science, Behavioral Science, or a related quantitative field. A PhD is preferred but not required.

Benefits

• Base salary of $198,000–$220,000 per year, plus equity.
• Performance-related bonuses for eligible employees.
• Medical, dental, and vision insurance, with employer contributions to Health Savings Accounts.
• Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses.
• 401(k) retirement plan with employer match.
• Paid parental, medical, and caregiver leave.
• Paid time off, paid company holidays, office closures, and paid sick or safe time as required by law.
• Mental health and wellness support.
• Employer-paid basic life and disability coverage.
• Annual learning and development stipend.
• Daily office meals and eligible meal delivery credits.
• Relocation support for eligible employees.
• Additional taxable fringe benefits may include charitable donation matching and wellness stipends.