Benefits
- Benefit
-
The ChatGPT group showed a 17.00 percent learning gain from pre-test to post-test.
Awaiting review
- Outcome identifiers
-
Unresolved
Not reviewedUnresolved
- Category
-
Improved Learning
Awaiting review
- Status
-
Reported benefit
Awaiting review
- Beneficiary groups
-
People.
Awaiting review
- Beneficiaries
-
Unresolved
Not reviewedUnresolved
- Reach
-
Unresolved
Not reviewedUnresolved
- Distribution and access
-
Unresolved
Not reviewedUnresolved
- Duration
-
Unresolved
Not reviewedUnresolved
- Expected?
-
Yes
Awaiting review
- Costs and harms
-
Unresolved
Not reviewedUnresolved
AI system
- System
-
ChatGPT 3.5 ("Dec 15 Version"), prompted to write worked-solution hints shown in the OATutor tutoring system
Awaiting review
- Model
-
This is an explicit unresolved assignment: the sources on record do not settle this question.
Not reviewedUnresolved
- Reported configuration
-
ChatGPT 3.5, "Dec 15 Version"
Awaiting review
- Provider
-
Unresolved
Not reviewedUnresolved
- Role
-
Unresolved
Not reviewedUnresolved
- Configuration
-
Unresolved
Not reviewedUnresolved
- Inputs and outputs
-
Unresolved
Not reviewedUnresolved
- Autonomy
-
Unresolved
Not reviewedUnresolved
- Workflow
-
ChatGPT wrote worked solutions that served as hints on textbook problems. Learners saw the hints in a tutoring system during short practice sessions.
Awaiting review
- Human oversight
-
Unresolved
Not reviewedUnresolved
- Relation to AI
-
Participants were randomly assigned to ChatGPT hints, human-tutor hints or no hints on the same problems, so the comparison isolates the effect of the ChatGPT hints.
Awaiting review
- AI contribution
-
Participants were randomly assigned to ChatGPT hints, human-tutor hints or no hints on the same problems, so the comparison isolates the effect of the ChatGPT hints.
Awaiting review
Measurement
- Studies
-
Randomized. [1]
Fully reviewed: supporting details for Studies
Review date: 29 October 2026.
Sources: [1]
- Study design
-
Randomized. [1]
Fully reviewed: supporting details for Study design
Review date: 29 October 2026.
Sources: [1]
- Measurement
-
Three-item pre-test and post-test on the assigned skill. [1]
Fully reviewed: supporting details for Measurement
Review date: 29 October 2026.
Sources: [1]
- Setting and sample
-
274 Amazon Mechanical Turk workers with at least a high school degree, 28 February to 10 May 2023. [1]
Fully reviewed: supporting details for Setting and sample
Review date: 29 October 2026.
Sources: [1]
- Measures
-
Percentage-point learning gain from pre-test to post-test. [1]
Fully reviewed: supporting details for Measures
Review date: 29 October 2026.
Sources: [1]
- Comparison
-
No-hint control and human-tutor hints. [1]
Fully reviewed: supporting details for Comparison
Review date: 29 October 2026.
Sources: [1]
- Results
-
17.00 percent gain with ChatGPT hints against 1.85 percent with no hints and 11.62 percent with human-tutor hints. [1]
Fully reviewed: supporting details for Results
Review date: 29 October 2026.
Sources: [1]
- Additional value from AI
-
ChatGPT hints produced a gain significant against no hints and similar to human-tutor hints. [1]
Fully reviewed: supporting details for Additional value from AI
Review date: 29 October 2026.
Sources: [1]
- Limitations
-
Adult crowd workers; short sessions; three-item tests; 32 percent of ChatGPT hints failed quality checks before mitigation. [1]
Fully reviewed: supporting details for Limitations
Review date: 29 October 2026.
Sources: [1]
Money
- Kind of value
-
The kind of monetary value is unclear.
Not reviewedUnresolved
- Amount
-
The monetary value is not established.
Not reviewedUnresolved
- Currency and prices
-
The currency and price basis are unclear.
Not reviewedUnresolved
- Period and unit
-
The period and unit of the monetary estimate are unclear.
Not reviewedUnresolved
- Realization
-
Whether the monetary value was realized is unclear.
Not reviewedUnresolved
- Recipients and payers
-
The recipients and payers are unclear.
Not reviewedUnresolved
- Gross or net
-
Whether the monetary figure includes costs is unclear.
Not reviewedUnresolved
- Included costs
-
The costs included in the estimate are unclear.
Not reviewedUnresolved
- Comparison
-
The financial comparison is not established.
Not reviewedUnresolved
- Method
-
The method used to calculate monetary value is unclear.
Not reviewedUnresolved
- Return and payback
-
The return on investment or payback period is not established.
Not reviewedUnresolved
People and places
- Participants
-
Developer:
OpenAI. Operator: University of California, Berkeley (Pardos and Bhandari), in an online study on Amazon Mechanical Turk.Awaiting review
- Jurisdictions
-
Benefit: Online (Amazon Mechanical Turk); study run from Berkeley, California,
United States. Developer: United States. Operator: United States.Awaiting review
- Ownership and control
-
Unresolved
Not reviewedUnresolved
- Dependencies
-
Unresolved
Not reviewedUnresolved
- Cross-border connections
-
Unresolved
Not reviewedUnresolved
- Geopolitical implications
-
The geopolitical implications are unclear.
Not reviewedUnresolved
Follow-up
- Conditions for replication
-
Hints must be checked or generated several times to catch errors. Problems must be text only.
Awaiting review
- Replication attempts
-
1 follow-up entry. Benefit did not recur: 1.
Awaiting review
- Later findings
-
Unresolved
Not reviewedUnresolved
- Availability
-
Unresolved
Not reviewedUnresolved
- Institutional decision
-
A relevant institutional decision has not been established.
Not reviewedUnresolved
- Grounds
-
The grounds for an institutional decision are unclear.
Not reviewedUnresolved
- Action and result
-
The result of an institutional decision is not established.
Not reviewedUnresolved
- Success criterion
-
The criterion for a successful institutional decision is unclear.
Not reviewedUnresolved
Evidence
- Source numbers
-
AIBD-0618-source-0001
Awaiting review
- Citations
-
Numbered references below.
Awaiting review
- Language
-
Unresolved
Not reviewedUnresolved
- Source relationships
-
Independent
Awaiting review
- Original evidence
-
Unresolved
Not reviewedUnresolved
- Access and versions
-
Unresolved
Not reviewedUnresolved
- Supported findings
-
Unresolved
Not reviewedUnresolved
- Evidence links
-
Unresolved
Not reviewedUnresolved
- Evidence strength
-
Independently studied. Sources: one independent peer-reviewed study. The cited evidence includes no finding by a regulator or court. [1]
Fully reviewed: supporting details for Evidence strength
Review date: 29 October 2026.
Sources: [1]
- Missing evidence
-
Unresolved
Not reviewedUnresolved
Grades
- Assessed benefits
-
People.
Awaiting review
- Depth
-
People: 2.
Awaiting review
- Reasons
-
People: Participants given ChatGPT hints showed a 17.00 percent learning gain from pre-test to post-test, significant against a no-hint control.
Awaiting review
- Observed or intended
-
Unresolved
Not reviewedUnresolved
- Rubric
-
Unresolved
Not reviewedUnresolved
- Assessor and date
-
Unresolved
Not reviewedUnresolved
- Alternative assessments
-
Unresolved
Not reviewedUnresolved
- Fill
-
Independently studied
Awaiting review
Entry details
- Kind
-
The case describes an evaluation or a study. The case is a randomized online experiment. [1]
Fully reviewed: supporting details for Kind
Review date: 29 October 2026.
Sources: [1]
- Related entries
-
No incident in the AI Incident Database snapshot of 26 October 2026 concerns the same kind of system harming the same kind of stakeholder.
Partly reviewedUnresolved: supporting details for Related entries
Compared with the incidents in the AI Incident Database snapshot of 26 October 2026.
- Field
-
Education.
Awaiting review
- Purpose
-
ChatGPT is a general-purpose chatbot. The authors used it to write hints for tutoring software, and learning from those hints was the purpose of the study.
Awaiting review
- Dates
-
Observed: 10 May 2023. Updated: 29 October 2026.
Awaiting review
- Stage
-
The system was tested in a laboratory. The hints were tested in a short online experiment with crowd workers. [1]
Fully reviewed: supporting details for Stage
Review date: 29 October 2026.
Sources: [1]
- Version
-
SHA-256: 187ea83521fb…
Awaiting review: supporting details for Version
SHA-256 of the public case, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.
References
- Other identifiers
-
Unresolved
Not reviewedUnresolved
- Category mappings
-
Unresolved
Not reviewedUnresolved
- Citable summary
-
Unresolved
Not reviewedUnresolved
- Revisions
-
1 revision. Latest: 29 October 2026.
Awaiting review
Sources
-
Peer-reviewed study Independent author
Pardos ZA, Bhandari S. ChatGPT-generated help produces learning gains equivalent to human tutor-authored help on mathematics skills. PLoS One. 2024;19(5):e0304013. doi:10.1371/journal.pone.0304013
We find that only the ChatGPT condition produces statistically significant learning gains compared to a no-help control, with no statistically significant differences in gains or time-on-task observed between learners receiving ChatGPT vs human tutor help.