{"id":"AIBD-0667","title":"GPT-4 speeds law students' drafting of complaints, contracts, handbooks and memos by 12% to 32% in a randomized trial at the University of Minnesota","summary":"In a randomized trial with 60 Minnesota law students, GPT-4 cut time on four legal tasks by 11.8% to 32.1%, with slight and uneven gains in quality.","date_observed":"2023-08-01","date_basis":"source information","date_precision":"month","benefit_status":"reported","source_collection":null,"source_entry":null,"account":"Jonathan Choi of the University of Southern California and Amy Monahan and Daniel Schwarcz of the University of Minnesota recruited students from the University of Minnesota Law School in April 2023. They split 60 students at random into two groups of thirty, and each group used GPT-4 on two of four tasks typical of junior lawyers and no AI on the other two: drafting a complaint, a contract, a section of an employee handbook and a client memo. Fifty-nine students finished. The students first took about two hours of online training in GPT-4 and then worked remotely from June to early August 2023, and graders scored the work blind. Access to GPT-4 cut time on every task: by 24.1% on the complaint, 32.1% on the contract, 21.1% on the handbook and 11.8% on the memo. Quality rose only slightly and inconsistently, with a significant gain on the contract task, and where the tool helped, the lowest-skilled participants gained most. On the tasks where GPT-4 helped most, participants reported more satisfaction from using it. The Minnesota Law Review published the study in November 2024. The participants were students, the sample was small, and the tasks were written for the study. A University of Minnesota Law School faculty fund paid for the work.","category":"general benefit","unexpected":false,"purpose":"GPT-4 is a general-purpose language model. The study tested its use for legal drafting and analysis.","system":"GPT-4","model_version":null,"developer":"OpenAI","deployer":"University of Minnesota Law School researchers (Choi, Monahan and Schwarcz)","location":"University of Minnesota Law School, Minneapolis, United States","domain":"Legal services","segment":null,"benefit_type":"Reduced Task Time","taxonomy_note":null,"registry_ids":{},"ratings":[{"group":"personal","depth":1,"reason":"Students with GPT-4 finished each legal task faster, saving 11.8% to 32.1% of their time."}],"ai_role":{"status":"assessed","statement":"Students were randomly assigned to complete each task with or without GPT-4, and graders were blind to the assignment, so the time and grade differences isolate the effect of the tool."},"evidence":[{"kind":"report","citation":"Choi JH, Monahan AB, Schwarcz D. Lawyering in the Age of Artificial Intelligence. Minnesota Law Review. 2024;109:147. doi:10.24926/265535.4225 (student-edited law review).","title":"Lawyering in the Age of Artificial Intelligence","url":"https://doi.org/10.24926/265535.4225","date":"2024-11-01","written_by":"independent","quote":"We found that access to GPT-4 only slightly and inconsistently improved the quality of participants' legal analysis but induced large and consistent increases in speed."},{"kind":"report","citation":"Choi JH, Monahan AB, Schwarcz D. Lawyering in the Age of Artificial Intelligence. 109 Minn. L. Rev. 147 (2024), full text (archived copy).","title":"Lawyering in the Age of Artificial Intelligence (full text)","url":"https://web.archive.org/web/2025id_/https://scholarship.law.umn.edu/cgi/viewcontent.cgi?article=1002&context=minnlrev","date":"2024-11-01","written_by":"independent","quote":"We recruited students from the University of Minnesota Law School in April 2023 to participate in our study over"}],"replication":{"produced_by":["Students took about two hours of online training in using GPT-4 for legal work.","Tasks mirrored the work of junior associates.","Graders scored the work blind to the assignment."],"must_hold":["The user must check the model's legal output for errors.","The task must be one where the model helps; it helped least on the client memo."],"echoes":[{"date":"2025-03-02","setting":"Upper-level law students using o1-preview or Vincent AI on six legal tasks","model_version":null,"outcome":"recurred","source":"https://web.archive.org/web/2025/https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5162111"}]},"paired_incident":{"database":"AI Incident Database","identifier":"541","title":"ChatGPT Reportedly Produced False Court Case Law Presented by Legal Counsel in Court","url":"https://incidentdatabase.ai/cite/541","severity":null,"group":"personal","basis":"Both cases concern a person who uses ChatGPT or GPT-4 to draft legal documents, where the drafter is held responsible for fabricated or wrong content in the result."},"headline":"*GPT-4* speeds up legal drafting for law students","metric":"cut time on four legal tasks by 11.8% to 32.1%","comparator":{"against":"Students completing the same task without GPT-4","before":"69.72 minutes on average for the contract drafting task","after":"47.59 minutes on average with GPT-4 (32.1% less)","basis":"measured","quote":"the largest gain in speed (in percentage terms) occurs in the task for which GPT-4 was the most useful in terms of grade improvement (contract drafting)","url":"https://web.archive.org/web/2025id_/https://scholarship.law.umn.edu/cgi/viewcontent.cgi?article=1002&context=minnlrev"},"later":null,"value":null,"developer_country":"United States","deployer_country":"United States","unrated_because":null,"brief_notes":null,"brief":null,"revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://doi.org/10.24926/265535.4225","editor":"AIBD"},{"date":"2026-10-29","change":"Linked a comparable incident in the AI Incident Database (incident 541).","source":"https://incidentdatabase.ai/cite/541","editor":"AIBD"}],"confidence":"studied","shade":"high","claim_status":"Independently studied","claim_basis":"Sources: two independent reports. The cited evidence includes no finding by a regulator or court.","case_jurisdictions":["United States"],"profile":{"case_id":"AIBD-0667","catalog_version":"1.0.0","source_fingerprint":"f95083d9d9ef8724f7ba988517cf7b93226425afc3b8ada3b71f67dc1e1aeda4","collection_refs":[{"collection_id":"aibd","version":"2026-10-29"}],"source_ids":["AIBD-0667-source-0001","AIBD-0667-source-0002"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0667-evaluation-1"],"role":"Evaluations of the benefit"},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":[],"source_refs":["AIBD-0667-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"partial","record_ids":["AIBD-0667-evaluation-1"]},"monetary observations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"OpenAI","deployer":"University of Minnesota Law School researchers (Choi, Monahan and Schwarcz)"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer","deployer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"value_present","value":{"kind":"record_fields","fields":{"ai_role":{"status":"assessed","statement":"Students were randomly assigned to complete each task with or without GPT-4, and graders were blind to the assignment, so the time and grade differences isolate the effect of the tool."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"general benefit","ai_role":{"status":"assessed","statement":"Students were randomly assigned to complete each task with or without GPT-4, and graders were blind to the assignment, so the time and grade differences isolate the effect of the tool."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":1,"reason":"Students with GPT-4 finished each legal task faster, saving 11.8% to 32.1% of their time."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"high","evidence_word":"studied","claim_status":"Independently studied","basis":"Sources: two independent reports. The cited evidence includes no finding by a regulator or court.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":1,"reason":"Students with GPT-4 finished each legal task faster, saving 11.8% to 32.1% of their time."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":1,"reason":"Students with GPT-4 finished each legal task faster, saving 11.8% to 32.1% of their time."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"value_present","value":{"kind":"structured","data":{"classification":"evaluation or study","answer":"The case describes an evaluation or a study. A randomized trial.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0667-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The case describes an evaluation or a study. A randomized trial.","inferred":false}},"case_links[]":{"state":"value_present","value":{"kind":"structured","data":{"answer":"Both cases concern a person who uses ChatGPT or GPT-4 to draft legal documents, where the drafter is held responsible for fabricated or wrong content in the result.","links":[{"database":"AI Incident Database","identifier":"541","title":"ChatGPT Reportedly Produced False Court Case Law Presented by Legal Counsel in Court","url":"https://incidentdatabase.ai/cite/541","group":"personal","basis":"Both cases concern a person who uses ChatGPT or GPT-4 to draft legal documents, where the drafter is held responsible for fabricated or wrong content in the result.","relation":"comparable_harm"}]}},"provenance":{"kind":"curatorial","basis":"Compared with the incidents in the AI Incident Database snapshot of 26 October 2026; a second reading confirmed that the incident concerns the same kind of system and stakeholder.","record_fields":["paired_incident"],"source_refs":["https://incidentdatabase.ai/cite/541"]},"review_state":"reviewed","coverage":"complete","display":{"answer":"Both cases concern a person who uses ChatGPT or GPT-4 to draft legal documents, where the drafter is held responsible for fabricated or wrong content in the result.","inferred":false}},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"Independently studied. Sources: two independent reports. The cited evidence includes no finding by a regulator or court.","evidence_word":"studied","claim_status":"Independently studied"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0667-source-0001","AIBD-0667-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Independently studied. Sources: two independent reports. The cited evidence includes no finding by a regulator or court.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"laboratory test","answer":"The system was tested in a laboratory. Students completed designed tasks in a study.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0667-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was tested in a laboratory. Students completed designed tasks in a study.","inferred":false}},"economics[].amount":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].costs[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].counterfactual":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].currency_price_basis":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].gross_net":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].measure_type":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].period_unit":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].realization":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].recipient_payer":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].return_payback":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].valuation_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"evaluations[].ai_uplift":{"state":"value_present","value":{"kind":"text","text":"Faster on all four tasks."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0667-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Faster on all four tasks.","inferred":false}},"evaluations[].comparator":{"state":"value_present","value":{"kind":"text","text":"Same tasks without GPT-4."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0667-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Same tasks without GPT-4.","inferred":false}},"evaluations[].design":{"state":"value_present","value":{"kind":"text","text":"Randomized."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0667-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Randomized.","inferred":false}},"evaluations[].effect":{"state":"value_present","value":{"kind":"text","text":"Time down 24.1% (complaint), 32.1% (contract), 21.1% (handbook), 11.8% (memo); grade gain significant only for the contract."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0667-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Time down 24.1% (complaint), 32.1% (contract), 21.1% (handbook), 11.8% (memo); grade gain significant only for the contract.","inferred":false}},"evaluations[].exposure":{"state":"value_present","value":{"kind":"text","text":"GPT-4 access after several hours of training."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0667-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"GPT-4 access after several hours of training.","inferred":false}},"evaluations[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0667-evaluation-1"],"role":"Evaluations of the benefit"},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0667-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Randomized.","inferred":false}},"evaluations[].measurement_method":{"state":"value_present","value":{"kind":"text","text":"Self-tracked time on task and blind grading on a 4.0 scale."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0667-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Self-tracked time on task and blind grading on a 4.0 scale.","inferred":false}},"evaluations[].metric":{"state":"value_present","value":{"kind":"text","text":"Minutes per task and grade."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0667-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Minutes per task and grade.","inferred":false}},"evaluations[].reproduction_materials":{"state":"value_present","value":{"kind":"text","text":"Task materials reproduced in the article appendices."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0667-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Task materials reproduced in the article appendices.","inferred":false}},"evaluations[].setting_sample":{"state":"value_present","value":{"kind":"text","text":"60 University of Minnesota law students, 240 task completions."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0667-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"60 University of Minnesota law students, 240 task completions.","inferred":false}},"evaluations[].uncertainty_limitations":{"state":"value_present","value":{"kind":"text","text":"Students in place of practicing lawyers; small sample; designed tasks."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0667-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Students in place of practicing lawyers; small sample; designed tasks.","inferred":false}},"external_references[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"paired_incident":{"database":"AI Incident Database","identifier":"541","title":"ChatGPT Reportedly Produced False Court Case Law Presented by Legal Counsel in Court","url":"https://incidentdatabase.ai/cite/541","group":"personal","basis":"Both cases concern a person who uses ChatGPT or GPT-4 to draft legal documents, where the drafter is held responsible for fabricated or wrong content in the result."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["paired_incident"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"follow_up[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-0667"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Reduced Task Time"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"Jonathan Choi of the University of Southern California and Amy Monahan and Daniel Schwarcz of the University of Minnesota recruited students from the University of Minnesota Law School in April 2023. They split 60 students at random into two groups of thirty, and each group used GPT-4 on two of four tasks typical of junior lawyers and no AI on the other two: drafting a complaint, a contract, a section of an employee handbook and a client memo. Fifty-nine students finished. The students first took about two hours of online training in GPT-4 and then worked remotely from June to early August 2023, and graders scored the work blind. Access to GPT-4 cut time on every task: by 24.1% on the complaint, 32.1% on the contract, 21.1% on the handbook and 11.8% on the memo. Quality rose only slightly and inconsistently, with a significant gain on the contract task, and where the tool helped, the lowest-skilled participants gained most. On the tasks where GPT-4 helped most, participants reported more satisfaction from using it. The Minnesota Law Review published the study in November 2024. The participants were students, the sample was small, and the tasks were written for the study. A University of Minnesota Law School faculty fund paid for the work.","benefit_type":"Reduced Task Time"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":1,"reason":"Students with GPT-4 finished each legal task faster, saving 11.8% to 32.1% of their time."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"reported"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"GPT-4 is a general-purpose language model. The study tested its use for legal drafting and analysis."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"University of Minnesota Law School, Minneapolis, United States","developer_country":"United States","deployer_country":"United States"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","developer_country","deployer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"GPT-4 is a general-purpose language model. The study tested its use for legal drafting and analysis."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2023-08-01","date_basis":"source information","date_precision":"month","revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://doi.org/10.24926/265535.4225","editor":"AIBD"},{"date":"2026-10-29","change":"Linked a comparable incident in the AI Incident Database (incident 541).","source":"https://incidentdatabase.ai/cite/541","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["Students took about two hours of online training in using GPT-4 for legal work.","Tasks mirrored the work of junior associates.","Graders scored the work blind to the assignment."],"must_hold":["The user must check the model's legal output for errors.","The task must be one where the model helps; it helped least on the client memo."],"echoes":[{"date":"2025-03-02","setting":"Upper-level law students using o1-preview or Vincent AI on six legal tasks","outcome":"recurred","source":"https://web.archive.org/web/2025/https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5162111"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["Students took about two hours of online training in using GPT-4 for legal work.","Tasks mirrored the work of junior associates.","Graders scored the work blind to the assignment."],"must_hold":["The user must check the model's legal output for errors.","The task must be one where the model helps; it helped least on the client memo."],"echoes":[{"date":"2025-03-02","setting":"Upper-level law students using o1-preview or Vincent AI on six legal tasks","outcome":"recurred","source":"https://web.archive.org/web/2025/https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5162111"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://doi.org/10.24926/265535.4225","editor":"AIBD"},{"date":"2026-10-29","change":"Linked a comparable incident in the AI Incident Database (incident 541).","source":"https://incidentdatabase.ai/cite/541","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Legal services"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"report","citation":"Choi JH, Monahan AB, Schwarcz D. Lawyering in the Age of Artificial Intelligence. Minnesota Law Review. 2024;109:147. doi:10.24926/265535.4225 (student-edited law review).","title":"Lawyering in the Age of Artificial Intelligence","url":"https://doi.org/10.24926/265535.4225","date":"2024-11-01","written_by":"independent","quote":"We found that access to GPT-4 only slightly and inconsistently improved the quality of participants' legal analysis but induced large and consistent increases in speed."},{"kind":"report","citation":"Choi JH, Monahan AB, Schwarcz D. Lawyering in the Age of Artificial Intelligence. 109 Minn. L. Rev. 147 (2024), full text (archived copy).","title":"Lawyering in the Age of Artificial Intelligence (full text)","url":"https://web.archive.org/web/2025id_/https://scholarship.law.umn.edu/cgi/viewcontent.cgi?article=1002&context=minnlrev","date":"2024-11-01","written_by":"independent","quote":"We recruited students from the University of Minnesota Law School in April 2023 to participate in our study over"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0667-source-0001","AIBD-0667-source-0002"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"report","citation":"Choi JH, Monahan AB, Schwarcz D. Lawyering in the Age of Artificial Intelligence. Minnesota Law Review. 2024;109:147. doi:10.24926/265535.4225 (student-edited law review).","title":"Lawyering in the Age of Artificial Intelligence","url":"https://doi.org/10.24926/265535.4225","date":"2024-11-01","written_by":"independent","quote":"We found that access to GPT-4 only slightly and inconsistently improved the quality of participants' legal analysis but induced large and consistent increases in speed."},{"kind":"report","citation":"Choi JH, Monahan AB, Schwarcz D. Lawyering in the Age of Artificial Intelligence. 109 Minn. L. Rev. 147 (2024), full text (archived copy).","title":"Lawyering in the Age of Artificial Intelligence (full text)","url":"https://web.archive.org/web/2025id_/https://scholarship.law.umn.edu/cgi/viewcontent.cgi?article=1002&context=minnlrev","date":"2024-11-01","written_by":"independent","quote":"We recruited students from the University of Minnesota Law School in April 2023 to participate in our study over"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"In a randomized trial with 60 Minnesota law students, GPT-4 cut time on four legal tasks by 11.8% to 32.1%, with slight and uneven gains in quality.","account":"Jonathan Choi of the University of Southern California and Amy Monahan and Daniel Schwarcz of the University of Minnesota recruited students from the University of Minnesota Law School in April 2023. They split 60 students at random into two groups of thirty, and each group used GPT-4 on two of four tasks typical of junior lawyers and no AI on the other two: drafting a complaint, a contract, a section of an employee handbook and a client memo. Fifty-nine students finished. The students first took about two hours of online training in GPT-4 and then worked remotely from June to early August 2023, and graders scored the work blind. Access to GPT-4 cut time on every task: by 24.1% on the complaint, 32.1% on the contract, 21.1% on the handbook and 11.8% on the memo. Quality rose only slightly and inconsistently, with a significant gain on the contract task, and where the tool helped, the lowest-skilled participants gained most. On the tasks where GPT-4 helped most, participants reported more satisfaction from using it. The Minnesota Law Review published the study in November 2024. The participants were students, the sample was small, and the tasks were written for the study. A University of Minnesota Law School faculty fund paid for the work."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"GPT-4"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"GPT-4 speeds law students' drafting of complaints, contracts, handbooks and memos by 12% to 32% in a randomized trial at the University of Minnesota","headline":"*GPT-4* speeds up legal drafting for law students"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["2461df1227b1679e3d6c9108797f72097d7a814c02c415b46de411ad2b37ec2a"],"role":"SHA-256 of the complete public case and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public case, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"Jonathan Choi of the University of Southern California and Amy Monahan and Daniel Schwarcz of the University of Minnesota recruited students from the University of Minnesota Law School in April 2023. They split 60 students at random into two groups of thirty, and each group used GPT-4 on two of four tasks typical of junior lawyers and no AI on the other two: drafting a complaint, a contract, a section of an employee handbook and a client memo. Fifty-nine students finished. The students first took about two hours of online training in GPT-4 and then worked remotely from June to early August 2023, and graders scored the work blind. Access to GPT-4 cut time on every task: by 24.1% on the complaint, 32.1% on the contract, 21.1% on the handbook and 11.8% on the memo. Quality rose only slightly and inconsistently, with a significant gain on the contract task, and where the tool helped, the lowest-skilled participants gained most. On the tasks where GPT-4 helped most, participants reported more satisfaction from using it. The Minnesota Law Review published the study in November 2024. The participants were students, the sample was small, and the tasks were written for the study. A University of Minnesota Law School faculty fund paid for the work.","replication":{"produced_by":["Students took about two hours of online training in using GPT-4 for legal work.","Tasks mirrored the work of junior associates.","Graders scored the work blind to the assignment."],"must_hold":["The user must check the model's legal output for errors.","The task must be one where the model helps; it helped least on the client memo."],"echoes":[{"date":"2025-03-02","setting":"Upper-level law students using o1-preview or Vincent AI on six legal tasks","outcome":"recurred","source":"https://web.archive.org/web/2025/https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5162111"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":44,"reviewed":15,"reviewed_complete":15,"reviewed_partial":0,"record_mapped":29,"review_not_assessed":47,"not_assessed":0,"complete":false,"by_state":{"value_present":44,"not_reported":0,"unresolved":47,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"7e9007475738649377850117e61dacce04a5906744d0e484dc883ed7d935aa61"}