Variance Reduction System keeps Meta housing ads within agreed limits

Guidehouse verified that Meta met its housing-ad targets from January to April 2024. Of ads with at least 300 impressions, 94.2% had a sex variance of 10% or less, against a required 90.2%.

  • Not rated society

    The reviewer gives no figure for delivery without the system, so the size of the reduction in skew cannot be established.

Depth runs from 1, a convenience or time saved, to 5, a life or a livelihood preserved.

Status
Reported benefit.
Evidence

foundstudiedreportedclaimed

Only the evaluators support the benefit.

Compared with

The VRS Compliance Metrics that the Department of Justice and Meta agreed and the court adopted. Measured.

Without the system
Required coverage of 90.2% (sex, variance of 10% or less, ads with at least 300 impressions)
With it
94.2%
The system’s part
Not isolated. Guidehouse recomputed coverage from aggregated data that Meta supplied and compared it with the agreed metrics. It did not compare delivery with and without the system. Its report defines the VRS only as a Meta-developed system designed to reduce variance and does not say that it learns from data. In a white paper of January 2023, Meta describes the system as an offline reinforcement learning framework.
Later
Persisted, 30 October 2024. Guidehouse verified that Meta again met the agreed metrics for May to August 2024.
Not yet checked
LimitsCosts and harmsLearns from data

Download data Cite Suggest a correction

Benefit type
Increased Inclusion
System
Variance Reduction System (VRS)
Made by
Meta Platforms, Inc.
Used by
Meta Platforms, Inc. (housing ads on Facebook, Instagram and Messenger)
Where
United States
Stage
In routine use
Date
Kind
Aggregate result
Broad benefit
Monitoring Processes, Materials, or Surroundings, Making Decisions and Solving Problems
Collections
Scrutiny and findings
Last revised
31 October 2026

Sources

  1. Evaluation report, by an evaluator that the makers or users commissioned.

    Guidehouse Inc., independent third-party reviewer under the Settlement Agreement in United States v. Meta Platforms, Inc., No. 22-Civ-5187 (S.D.N.Y.). Meta VRS Compliance Metrics Verification, 28 June 2024. I. Executive Summary, p. 1, with Tables 1 and 2, p. 3. Meta proposed Guidehouse as reviewer with the consent of the Department of Justice. Published by the U.S. Department of Justice, Civil Rights Division.

    Guidehouse reviewed the Meta Compliance Report dated May 30, 2024 for the reporting period from January 1, 2024 to April 30, 2024 (Reporting Period) and verified that Meta complied with the relevant VRS Compliance Metrics for both sex and estimated race / ethnicity

  2. Evaluation report, by an evaluator that the makers or users commissioned.

    Guidehouse Inc. Meta VRS Compliance Metrics Verification, 30 October 2024, for the reporting period of 1 May to 31 August 2024. I. Executive Summary, p. 1. Published by the U.S. Department of Justice, Civil Rights Division.

    Guidehouse reviewed the Meta Compliance Report dated September 30, 2024 for the reporting period from May 1, 2024 to August 31, 2024 (Reporting Period) and verified that Meta complied with the relevant VRS Compliance Metrics for both sex and estimated race / ethnicity

  3. Report, by the makers. Context or contrary evidence.

    Meta. Toward fairness in personalized ads. White paper by Miranda Bogen and eight co-authors, January 2023. Section on ad delivery outcomes, How it works, p. 14.

    we formulate the Variance Reduction System as an offline reinforcement learning framework with the explicit goal of minimizing an ad's impression variance across the demographic subgroups

Details

AI system

Workflow

Meta ran housing ads through the VRS, which aims to reduce the variance between the eligible and actual audiences for sex and estimated race or ethnicity. Meta estimated race or ethnicity with privacy-enhanced Bayesian Improved Surname Geocoding and measured variance with Earth Mover's Distance. Meta measured each eligible audience from a sample of users, with a target of 6,000 and an expected minimum of 4,500.

Follow-up

Conditions for replication

The sampling process must return large enough samples. An API issue left some samples too small from 1 to 15 January 2024. Users' sex and ZIP codes must be known for their impressions to count in the calculation. Housing ads tagged with other special ad categories must be counted, since Meta's calculation left some out.

Entry details

Sector

Housing advertising.

Purpose

Meta built the Variance Reduction System to reduce the gap between the sex and estimated race or ethnicity mix of the people eligible to see a housing ad and of the people who see it.

31 more questions not yet checked

model, release, provider, role, configuration, inputs and outputs, autonomy, human oversight, AI contribution, beneficiaries, reach, distribution and access, duration, every measurement question, every money question, ownership and control, dependencies, cross-border connections, geopolitical implications, replication attempts, availability, institutional decision, grounds, action and result, success criterion, every evidence question, observed or intended, alternative assessments, related entries, other identifiers, category mappings.

Cite this record

Peter H. Vartanian. (2026). Variance Reduction System keeps Meta housing ads within agreed limits (AIBD-0925). Artificial Intelligence Benefits Database, edition of 1 November 2026. https://petervartanian.xyz/aibd/records/AIBD-0925.html (archived at https://doi.org/10.7910/DVN/AADSII, version 7.0)

BibTeX Data Field definitions