Introduction

The Problem:

  • AI hiring tools amplify bias from training data
  • Users exhibit automation bias—over-trusting outputs
  • Existing solutions focus on technical fixes, not users

The Gap:

Post-training, user-facing interventions remain underdeveloped despite AI's opacity.

Key Definitions
🛡️ Psychological Inoculation
Exposing users to weakened forms of bias with explanations to build cognitive resistance
⚠️ Counterfactual Audit Flags
"What-if" explanations showing how small input changes alter AI rankings
🤖 Automation Bias
Tendency to over-trust AI outputs without critical scrutiny
⚖️ AI Bias (Hiring Context)
Systematic discrimination in AI recommendations based on protected attributes
Study Snapshot
226 Participants
8 Bias Scenarios
3 Conditions
84% Retention
Research Questions
RQ1: Do counterfactual explanations improve bias recognition?
RQ2: Do they improve trust calibration?
RQ3: Do they increase confidence and decision quality?
RQ4: How do user characteristics influence effectiveness?
Hypotheses
H1: Trained participants will show higher bias recognition accuracy
H2: Trained participants will demonstrate better trust calibration
H3: Effects stronger for users with lower AI literacy
H4: Demographic factors will moderate impact
Study Materials

Scan to Access

Inoculation Video
Inoculation
Video
Control Video
Control
Video
Full Dissertation
Full
Dissertation
Key Results
LARGE EFFECT
👤
36% Effect Size for
Gender Bias Recognition
LARGE EFFECT
🌍
31% Effect Size for
Ethnicity Bias Recognition
💡 Users spotted gender bias 36% better after brief training with counterfactual flags
Effect Sizes by Scenario
ANCOVA Effect Sizes
Partial η² effect sizes across 8 scenarios and 4 dependent variables
Method

Design: 3-arm randomized controlled trial (N=226)

Group A: Inoculation video + Counterfactual audit flags
Group B: Inoculation video + Neutral explanations
Group C: Control video + Neutral explanations

Stimuli: 8 hiring scenarios (4 biased, 4 neutral)

Gender/name, ethnicity, age, alma mater, employer prestige, parental leave, national origin, disability

Analysis: ANCOVA with planned contrasts; moderation by AI literacy, age, domain expertise

H1: Bias Recognition
✓ SUPPORTED
Name–Gender
η²=.36, p<.001
Name–Ethnicity
η²=.31, p<.001
Age
η²=.13, p<.001
Employer
η²=.06, p=.001
H2: Trust Calibration
✓ SUPPORTED
  • Trust became more selective, not collapsed
  • Lower trust in biased scenarios
  • Maintained trust in neutral scenarios
  • Name–Gender: η²=.24, p<.001
  • Age: η²=.17, p<.001

Users shifted from blind acceptance → calibrated judgment

What I Found

A brief inoculation video + counterfactual audit flags helped users:

  • Spot bias more accurately—especially gender and ethnicity
  • Calibrate trust—appropriately skeptical without blanket distrust
  • Generalize benefits across AI literacy levels
🎯 Training is brief (90 seconds), works across user types, and requires no model retraining
How It Works
👤
User
→
🎥
Inoculation
Video
→
⚠️
Audit
Flag
→
🤔
Critical
Evaluation
→
✅
Better
Decision

Mechanism:

  • Counterfactual flags interrupt reflexive acceptance
  • Inoculation primes users to look for bias signals
  • Together: shift from blind acceptance → calibrated judgment
Who Does It Work For?
H3: ✓ Partial
H4: ✗ Not Supported
Works across all AI literacy levels
Effective across demographics
Generalizable benefits for most users
Small AI-literacy effects in 3 scenarios only
Practical Implications
💡
Embed counterfactual audit flags in AI hiring tools
🎓
Provide brief inoculation training for hiring managers
⚖️
Add decision steps: Proceed | Escalate | Override
📊
Monitor calibration metrics (differential trust)
Get Involved
🤝
Implement in your organization
🔬
Replicate the study
💼
Consulting opportunities
📚
Request collaboration
Theoretical Foundation

This research builds on:

  • Psychological Inoculation Theory (McGuire, 1964)
  • Explainable AI (XAI) frameworks
  • Human-AI trust calibration literature
  • Bias mitigation and fairness research
  • Automation bias studies
Limitations & Future Research

Limitations:

  • Self-reported outcomes; simulated workflow
  • Effects varied by scenario
  • Short-term effects; durability unknown

Future Directions:

  • Longitudinal studies with boosters
  • Behavioral endpoints (overrides)
  • Cross-domain validation
  • Field experiments
📬 GET IN TOUCH
🌐 elizabethvolini.ai
✉️ contact@elizabethvolini.ai
🎯 THE BOTTOM LINE
A 90-second video + simple "what-if" audit flags = measurably better bias detection
WITHOUT RETRAINING ANY AI MODELS
Scalable
Model-agnostic
Low-cost
User-empowered