Benefits
- Benefit
-
Raised productivity on five of six tasks by 50% to 130%
Awaiting review
- Outcome identifiers
-
Unresolved
Not reviewedUnresolved
- Category
-
Improved Task Accuracy
Awaiting review
- Status
-
Reported benefit
Awaiting review
- Beneficiary groups
-
People.
Awaiting review
- Beneficiaries
-
Unresolved
Not reviewedUnresolved
- Reach
-
Unresolved
Not reviewedUnresolved
- Distribution and access
-
Unresolved
Not reviewedUnresolved
- Duration
-
Unresolved
Not reviewedUnresolved
- Expected?
-
Yes
Awaiting review
- Costs and harms
-
Unresolved
Not reviewedUnresolved
AI system
- System
-
Vincent AI (vLex, retrieval-augmented generation) and OpenAI o1-preview
Awaiting review
- Model
-
This is an explicit unresolved assignment: the sources on record do not settle this question.
Not reviewedUnresolved
- Reported configuration
-
O1-preview; Vincent AI as of 2024
Awaiting review
- Provider
-
Unresolved
Not reviewedUnresolved
- Role
-
Unresolved
Not reviewedUnresolved
- Configuration
-
Unresolved
Not reviewedUnresolved
- Inputs and outputs
-
Unresolved
Not reviewedUnresolved
- Autonomy
-
Unresolved
Not reviewedUnresolved
- Workflow
-
Students used either a retrieval-augmented legal tool or a reasoning model on realistic legal tasks. Students received training in using both tools before the tasks. Graders scored the work blind to the condition.
Awaiting review
- Human oversight
-
Unresolved
Not reviewedUnresolved
- Relation to AI
-
Students were randomly assigned to one of the two tools or to no AI, so the differences isolate the effect of each tool.
Awaiting review
- AI contribution
-
Students were randomly assigned to one of the two tools or to no AI, so the differences isolate the effect of each tool.
Awaiting review
Measurement
- Studies
-
Randomized. [1]
Fully reviewed: supporting details for Studies
Review date: 29 October 2026.
Sources: [1]
- Study design
-
Randomized. [1]
Fully reviewed: supporting details for Study design
Review date: 29 October 2026.
Sources: [1]
- Measurement
-
Graded quality and productivity on six legal tasks. [1]
Fully reviewed: supporting details for Measurement
Review date: 29 October 2026.
Sources: [1]
- Setting and sample
-
153 enrolled upper-level law students at Michigan and Minnesota; 137 completed at least one task, 125 all six. [1]
Fully reviewed: supporting details for Setting and sample
Review date: 29 October 2026.
Sources: [1]
- Measures
-
Quality grades and productivity. [1]
Fully reviewed: supporting details for Measures
Review date: 29 October 2026.
Sources: [1]
- Comparison
-
No AI. [1]
Fully reviewed: supporting details for Comparison
Review date: 29 October 2026.
Sources: [1]
- Results
-
Significant quality gains for both tools; productivity gains of 50% to 130% on five of six tasks. [1]
Fully reviewed: supporting details for Results
Review date: 29 October 2026.
Sources: [1]
- Additional value from AI
-
Productivity up 50% to 130% on five tasks. [1]
Fully reviewed: supporting details for Additional value from AI
Review date: 29 October 2026.
Sources: [1]
- Limitations
-
Students in place of practicing lawyers; designed tasks; graders were co-authors; o1-preview work had more hallucinated citations. [1]
Fully reviewed: supporting details for Limitations
Review date: 29 October 2026.
Sources: [1]
Money
- Kind of value
-
The kind of monetary value is unclear.
Not reviewedUnresolved
- Amount
-
The monetary value is not established.
Not reviewedUnresolved
- Currency and prices
-
The currency and price basis are unclear.
Not reviewedUnresolved
- Period and unit
-
The period and unit of the monetary estimate are unclear.
Not reviewedUnresolved
- Realization
-
Whether the monetary value was realized is unclear.
Not reviewedUnresolved
- Recipients and payers
-
The recipients and payers are unclear.
Not reviewedUnresolved
- Gross or net
-
Whether the monetary figure includes costs is unclear.
Not reviewedUnresolved
- Included costs
-
The costs included in the estimate are unclear.
Not reviewedUnresolved
- Comparison
-
The financial comparison is not established.
Not reviewedUnresolved
- Method
-
The method used to calculate monetary value is unclear.
Not reviewedUnresolved
- Return and payback
-
The return on investment or payback period is not established.
Not reviewedUnresolved
People and places
- Participants
-
Developer: vLex (Vincent AI);
OpenAI (o1-preview). Operator: Law school researchers at the University of Minnesota and the University of Michigan.Awaiting review
- Jurisdictions
-
Benefit: University of Michigan Law School and University of Minnesota Law School,
United States (students worked remotely). Developer: United States (OpenAI); not verified for vLex. Operator: United States.Awaiting review
- Ownership and control
-
Unresolved
Not reviewedUnresolved
- Dependencies
-
Unresolved
Not reviewedUnresolved
- Cross-border connections
-
Unresolved
Not reviewedUnresolved
- Geopolitical implications
-
The geopolitical implications are unclear.
Not reviewedUnresolved
Follow-up
- Conditions for replication
-
The tool must ground its analysis in legal sources or reason through the problem before writing. Users must check citations for hallucinations.
Awaiting review
- Replication attempts
-
2 follow-up entries. Benefit recurred: 2.
Awaiting review
- Later findings
-
Unresolved
Not reviewedUnresolved
- Availability
-
Unresolved
Not reviewedUnresolved
- Institutional decision
-
A relevant institutional decision has not been established.
Not reviewedUnresolved
- Grounds
-
The grounds for an institutional decision are unclear.
Not reviewedUnresolved
- Action and result
-
The result of an institutional decision is not established.
Not reviewedUnresolved
- Success criterion
-
The criterion for a successful institutional decision is unclear.
Not reviewedUnresolved
Evidence
- Source numbers
-
AIBD-0668-source-0001–0002
Awaiting review
- Citations
-
Numbered references below.
Awaiting review
- Language
-
Unresolved
Not reviewedUnresolved
- Source relationships
-
Independent
Awaiting review
- Original evidence
-
Unresolved
Not reviewedUnresolved
- Access and versions
-
Unresolved
Not reviewedUnresolved
- Supported findings
-
Unresolved
Not reviewedUnresolved
- Evidence links
-
Unresolved
Not reviewedUnresolved
- Evidence strength
-
Independently studied. Sources: one independent peer-reviewed study and one independent preprint. The cited evidence includes no finding by a regulator or court. [1] [2]
- Missing evidence
-
Unresolved
Not reviewedUnresolved
Grades
- Assessed benefits
-
People.
Awaiting review
- Depth
-
People: 2.
Awaiting review
- Reasons
-
People: Students using either tool produced legal work of higher graded quality and were 50% to 130% more productive on five of six tasks.
Awaiting review
- Observed or intended
-
Unresolved
Not reviewedUnresolved
- Rubric
-
Unresolved
Not reviewedUnresolved
- Assessor and date
-
Unresolved
Not reviewedUnresolved
- Alternative assessments
-
Unresolved
Not reviewedUnresolved
- Fill
-
Independently studied
Awaiting review
Entry details
- Kind
-
The case describes an evaluation or a study. A randomized trial. [1]
Fully reviewed: supporting details for Kind
Review date: 29 October 2026.
Sources: [1]
- Related entries
-
Both cases concern a legal research AI tool built on retrieval-augmented generation that a person uses to prepare legal work, where the person is held responsible for false quotations or citations in the result.
Fully reviewed: supporting details for Related entries
Compared with the incidents in the AI Incident Database snapshot of 26 October 2026; a second reading confirmed that the incident concerns the same kind of system and stakeholder.
Sources: incidentdatabase.ai
- Field
-
Legal services.
Awaiting review
- Purpose
-
Vincent AI was built for legal research and drafting, and o1-preview is a general reasoning model. Better and faster legal work was the purpose that the trial tested.
Awaiting review
- Dates
-
Source information as of: 31 October 2024. Updated: 29 October 2026.
Awaiting review
- Stage
-
The system was tested in a laboratory. Students completed designed tasks in a study. [1]
Fully reviewed: supporting details for Stage
Review date: 29 October 2026.
Sources: [1]
- Version
-
SHA-256: a54423ecfe89…
Awaiting review: supporting details for Version
SHA-256 of the public case, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.
References
- Other identifiers
-
AI Incident Database: 1572.
Awaiting review
- Category mappings
-
Unresolved
Not reviewedUnresolved
- Citable summary
-
Unresolved
Not reviewedUnresolved
- Revisions
-
2 revisions. Latest: 29 October 2026.
Awaiting review
Sources
-
Peer-reviewed study Independent author
Schwarcz D, Manning S, Prescott JJ, Barry P, Cleveland DR, Rich B. AI-Powered Lawyering: AI Reasoning Models, Retrieval Augmented Generation, and the Future of Legal Practice. Journal of Law and Empirical Analysis. 2026;3(1):220-250. doi:10.1177/2755323X261427048 (University of Minnesota repository copy, archived).
Our findings also show that these AI tools significantly boost productivity in five out of six tested legal tasks, with statistically significant gains of anywhere from 50% to 130%.
-
Preprint Independent author
We find that both AI tools significantly enhance legal work quality, a marked contrast with previous research examining older large language models like GPT-4.