{"id":"AIBD-0665","title":"BCG consultants using GPT-4 finish 12.2% more tasks 25.1% faster with higher quality in a preregistered experiment (jagged frontier study)","summary":"In a preregistered experiment with 758 BCG consultants, GPT-4 users completed 12.2% more tasks 25.1% faster with higher quality, but erred more on a task beyond its reach.","date_observed":"2023-09-22","date_basis":"source publication","date_precision":"day","benefit_status":"reported","source_collection":null,"source_entry":null,"account":"Researchers from Harvard Business School, Wharton, Warwick Business School, MIT Sloan and the Boston Consulting Group Henderson Institute ran a preregistered experiment with 758 BCG consultants, about 7% of the firm's individual contributor-level consultants. After a baseline task, the consultants were randomly assigned to work without AI, with GPT-4, or with GPT-4 and an overview of prompt engineering; all used GPT-4 as available at the end of April 2023 through a company platform built on the OpenAI API. On 18 realistic consulting tasks that the authors judged within the reach of the model, the consultants using AI completed 12.2% more tasks, finished 25.1% more quickly and produced work that human graders rated more than 40% higher in quality. Consultants below the average baseline score improved by 43% and those above it by 17%. On a business problem chosen to lie outside the model's reach, consultants using AI were 19 percentage points less likely to reach the correct answer. Harvard Business School released the working paper on 22 September 2023, and Organization Science published the study in March 2026. The tasks were designed for the experiment. Three of the nine authors worked for BCG, another advises BCG, and Harvard Business School paid for part of the research.","category":"general benefit","unexpected":false,"purpose":"OpenAI built GPT-4 as a general-purpose language model. The experiment tested its use on consulting tasks.","system":"GPT-4 (via a ChatGPT-style interface with default system prompt and temperature)","model_version":"GPT-4 as available at the end of April 2023","developer":"OpenAI","deployer":"Boston Consulting Group, with the Harvard Business School Digital Data Design Institute research team","location":"Boston Consulting Group offices (the sources do not list countries)","domain":"Management consulting","segment":null,"benefit_type":"Reduced Task Time","taxonomy_note":null,"registry_ids":{},"ratings":[{"group":"personal","depth":2,"reason":"Consultants below the average baseline score raised their task performance by 43%, and those above it by 17%."},{"group":"organizational","depth":null,"reason":"The tasks were designed for the experiment, and the sources report no realized outcome for Boston Consulting Group as a firm."}],"ai_role":{"status":"assessed","statement":"Consultants were randomly assigned to work with or without GPT-4, so the differences in output isolate the effect of access to the model."},"evidence":[{"kind":"peer-reviewed study","citation":"Dell'Acqua F, McFowland E III, Mollick E, Lifshitz H, Kellogg KC, Rajendran S, Krayer L, Candelon F, Lakhani KR. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality. Organization Science. 2026;37(2):403-423. doi:10.1287/orsc.2025.21838 (publisher abstract as deposited with Crossref).","title":"Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality","url":"https://doi.org/10.1287/orsc.2025.21838","date":"2026-03-01","written_by":"deployer","quote":"subjects using AI outperformed those not using AI, completing 12.2% more tasks and completing them 25.1% more quickly on average while also delivering solutions of significantly improved quality"},{"kind":"preprint","citation":"Dell'Acqua F, et al. Navigating the Jagged Technological Frontier. HBS Working Paper 24-013 (PDF, archived copy), 22 September 2023.","title":"Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality","url":"https://web.archive.org/web/20250102233043id_/https://www.hbs.edu/ris/Publication%20Files/24-013_d9b45b68-9e74-42d6-a1c6-c72fb70c7282.pdf","date":"2023-09-22","written_by":"deployer","quote":"Consultants across the skills distribution benefited significantly from having AI augmentation, with those below the average performance threshold increasing by 43% and those above increasing by 17% compared to their own scores."}],"replication":{"produced_by":["Consultants used GPT-4 on tasks that experts had chosen as within its reach.","Some consultants also received a short overview of prompt engineering.","Human graders scored the quality of the work."],"must_hold":["The task must lie within what the model can do well.","Workers must check output on tasks where the model is weak."],"echoes":[{"date":"2024-07-01","setting":"776 Procter & Gamble professionals on product development tasks with a GPT-4-based tool","model_version":null,"outcome":"recurred","source":"https://www.nber.org/system/files/working_papers/w33641/w33641.pdf"}]},"paired_incident":{"database":"AI Incident Database","identifier":"1193","title":"Purportedly Taxpayer-Funded Deloitte Report for Australian Government Contains Alleged AI-Generated Citations and Fabricated Legal Quote","url":"https://incidentdatabase.ai/cite/1193","severity":null,"group":"organizational","basis":"Both cases concern a consulting firm that uses a large language model to produce client work, where errors in the output fall on the firm."},"headline":"Consultants with *GPT-4* finish more tasks in less time","metric":"completed 12.2% more tasks, finished 25.1% more quickly","comparator":{"against":"Consultants randomly assigned to work without AI","before":"Control group task completion, speed and quality","after":"12.2% more tasks, 25.1% faster, quality rated more than 40% higher","basis":"measured","quote":"they completed 12.2% more tasks on average, and completed tasks 25.1% more quickly), and produced significantly higher quality results (more than 40% higher quality compared to a control group)","url":"https://web.archive.org/web/20250102233043id_/https://www.hbs.edu/ris/Publication%20Files/24-013_d9b45b68-9e74-42d6-a1c6-c72fb70c7282.pdf"},"later":null,"value":null,"developer_country":"United States","deployer_country":"United States","unrated_because":null,"brief_notes":null,"brief":null,"revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://api.crossref.org/works/10.1287/orsc.2025.21838","editor":"AIBD"},{"date":"2026-10-29","change":"Linked a comparable incident in the AI Incident Database (incident 1193).","source":"https://incidentdatabase.ai/cite/1193","editor":"AIBD"},{"date":"2026-10-29","change":"Revised the depth ratings.","source":null,"editor":"AIBD"}],"confidence":"claimed","shade":"low","claim_status":"The deployer's account","claim_basis":"Sources: the deployer's peer-reviewed study and the deployer's preprint. The cited evidence includes no independent source and no finding by a regulator or court.","case_jurisdictions":[],"profile":{"case_id":"AIBD-0665","catalog_version":"1.0.0","source_fingerprint":"45372dfc7ec5dfde3e97e4d526d8c1054117a07edea300e8dbd701a918e0ec26","collection_refs":[{"collection_id":"aibd","version":"2026-10-29"}],"source_ids":["AIBD-0665-source-0001","AIBD-0665-source-0002"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0665-evaluation-1"],"role":"Evaluations of the benefit"},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":[],"source_refs":["AIBD-0665-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"partial","record_ids":["AIBD-0665-evaluation-1"]},"monetary observations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"OpenAI","deployer":"Boston Consulting Group, with the Harvard Business School Digital Data Design Institute research team"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer","deployer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"value_present","value":{"kind":"record_fields","fields":{"ai_role":{"status":"assessed","statement":"Consultants were randomly assigned to work with or without GPT-4, so the differences in output isolate the effect of access to the model."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"general benefit","ai_role":{"status":"assessed","statement":"Consultants were randomly assigned to work with or without GPT-4, so the differences in output isolate the effect of access to the model."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"Consultants below the average baseline score raised their task performance by 43%, and those above it by 17%."},{"group":"organizational","reason":"The tasks were designed for the experiment, and the sources report no realized outcome for Boston Consulting Group as a firm."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"low","evidence_word":"claimed","claim_status":"The deployer's account","basis":"Sources: the deployer's peer-reviewed study and the deployer's preprint. The cited evidence includes no independent source and no finding by a regulator or court.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"Consultants below the average baseline score raised their task performance by 43%, and those above it by 17%."},{"group":"organizational","reason":"The tasks were designed for the experiment, and the sources report no realized outcome for Boston Consulting Group as a firm."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"Consultants below the average baseline score raised their task performance by 43%, and those above it by 17%."},{"group":"organizational","reason":"The tasks were designed for the experiment, and the sources report no realized outcome for Boston Consulting Group as a firm."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"value_present","value":{"kind":"structured","data":{"classification":"evaluation or study","answer":"The case describes an evaluation or a study. A preregistered randomized experiment.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0665-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The case describes an evaluation or a study. A preregistered randomized experiment.","inferred":false}},"case_links[]":{"state":"value_present","value":{"kind":"structured","data":{"answer":"Both cases concern a consulting firm that uses a large language model to produce client work, where errors in the output fall on the firm.","links":[{"database":"AI Incident Database","identifier":"1193","title":"Purportedly Taxpayer-Funded Deloitte Report for Australian Government Contains Alleged AI-Generated Citations and Fabricated Legal Quote","url":"https://incidentdatabase.ai/cite/1193","group":"organizational","basis":"Both cases concern a consulting firm that uses a large language model to produce client work, where errors in the output fall on the firm.","relation":"comparable_harm"}]}},"provenance":{"kind":"curatorial","basis":"Compared with the incidents in the AI Incident Database snapshot of 26 October 2026; a second reading confirmed that the incident concerns the same kind of system and stakeholder.","record_fields":["paired_incident"],"source_refs":["https://incidentdatabase.ai/cite/1193"]},"review_state":"reviewed","coverage":"complete","display":{"answer":"Both cases concern a consulting firm that uses a large language model to produce client work, where errors in the output fall on the firm.","inferred":false}},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"The deployer's account. Sources: the deployer's peer-reviewed study and the deployer's preprint. The cited evidence includes no independent source and no finding by a regulator or court.","evidence_word":"claimed","claim_status":"The deployer's account"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0665-source-0001","AIBD-0665-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The deployer's account. Sources: the deployer's peer-reviewed study and the deployer's preprint. The cited evidence includes no independent source and no finding by a regulator or court.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"pilot","answer":"The system was used in a pilot. The model was provided for an experiment on designed tasks.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0665-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was used in a pilot. The model was provided for an experiment on designed tasks.","inferred":false}},"economics[].amount":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].costs[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].counterfactual":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].currency_price_basis":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].gross_net":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].measure_type":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].period_unit":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].realization":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].recipient_payer":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].return_payback":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].valuation_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"evaluations[].ai_uplift":{"state":"value_present","value":{"kind":"text","text":"Gains of 43% for below-average and 17% for above-average consultants."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0665-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Gains of 43% for below-average and 17% for above-average consultants.","inferred":false}},"evaluations[].comparator":{"state":"value_present","value":{"kind":"text","text":"Consultants without AI."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0665-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Consultants without AI.","inferred":false}},"evaluations[].design":{"state":"value_present","value":{"kind":"text","text":"Randomized."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0665-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Randomized.","inferred":false}},"evaluations[].effect":{"state":"value_present","value":{"kind":"text","text":"12.2% more tasks, 25.1% faster, more than 40% higher quality inside the frontier; 19 points lower correctness outside it."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0665-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"12.2% more tasks, 25.1% faster, more than 40% higher quality inside the frontier; 19 points lower correctness outside it.","inferred":false}},"evaluations[].exposure":{"state":"value_present","value":{"kind":"text","text":"GPT-4 access during the experimental task, with or without a prompt overview."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0665-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"GPT-4 access during the experimental task, with or without a prompt overview.","inferred":false}},"evaluations[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0665-evaluation-1"],"role":"Evaluations of the benefit"},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0665-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Randomized.","inferred":false}},"evaluations[].measurement_method":{"state":"value_present","value":{"kind":"text","text":"Subtasks completed, time to completion and graded quality of outputs."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0665-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Subtasks completed, time to completion and graded quality of outputs.","inferred":false}},"evaluations[].metric":{"state":"value_present","value":{"kind":"text","text":"Tasks completed, speed and quality score."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0665-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Tasks completed, speed and quality score.","inferred":false}},"evaluations[].reproduction_materials":{"state":"value_present","value":{"kind":"text","text":"Preregistered on OSF; data not public."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0665-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Preregistered on OSF; data not public.","inferred":false}},"evaluations[].setting_sample":{"state":"value_present","value":{"kind":"text","text":"758 BCG consultants, two experiments, 18 tasks inside the frontier and one outside."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0665-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"758 BCG consultants, two experiments, 18 tasks inside the frontier and one outside.","inferred":false}},"evaluations[].uncertainty_limitations":{"state":"value_present","value":{"kind":"text","text":"Designed tasks; one firm; one model version; BCG co-authors; lower correctness on the task outside the frontier."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0665-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Designed tasks; one firm; one model version; BCG co-authors; lower correctness on the task outside the frontier.","inferred":false}},"external_references[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"paired_incident":{"database":"AI Incident Database","identifier":"1193","title":"Purportedly Taxpayer-Funded Deloitte Report for Australian Government Contains Alleged AI-Generated Citations and Fabricated Legal Quote","url":"https://incidentdatabase.ai/cite/1193","group":"organizational","basis":"Both cases concern a consulting firm that uses a large language model to produce client work, where errors in the output fall on the firm."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["paired_incident"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"follow_up[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-0665"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Reduced Task Time"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"Researchers from Harvard Business School, Wharton, Warwick Business School, MIT Sloan and the Boston Consulting Group Henderson Institute ran a preregistered experiment with 758 BCG consultants, about 7% of the firm's individual contributor-level consultants. After a baseline task, the consultants were randomly assigned to work without AI, with GPT-4, or with GPT-4 and an overview of prompt engineering; all used GPT-4 as available at the end of April 2023 through a company platform built on the OpenAI API. On 18 realistic consulting tasks that the authors judged within the reach of the model, the consultants using AI completed 12.2% more tasks, finished 25.1% more quickly and produced work that human graders rated more than 40% higher in quality. Consultants below the average baseline score improved by 43% and those above it by 17%. On a business problem chosen to lie outside the model's reach, consultants using AI were 19 percentage points less likely to reach the correct answer. Harvard Business School released the working paper on 22 September 2023, and Organization Science published the study in March 2026. The tasks were designed for the experiment. Three of the nine authors worked for BCG, another advises BCG, and Harvard Business School paid for part of the research.","benefit_type":"Reduced Task Time"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"Consultants below the average baseline score raised their task performance by 43%, and those above it by 17%."},{"group":"organizational","reason":"The tasks were designed for the experiment, and the sources report no realized outcome for Boston Consulting Group as a firm."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"reported"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"OpenAI built GPT-4 as a general-purpose language model. The experiment tested its use on consulting tasks."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"Boston Consulting Group offices (the sources do not list countries)","developer_country":"United States","deployer_country":"United States"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","developer_country","deployer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"OpenAI built GPT-4 as a general-purpose language model. The experiment tested its use on consulting tasks."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2023-09-22","date_basis":"source publication","date_precision":"day","revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://api.crossref.org/works/10.1287/orsc.2025.21838","editor":"AIBD"},{"date":"2026-10-29","change":"Linked a comparable incident in the AI Incident Database (incident 1193).","source":"https://incidentdatabase.ai/cite/1193","editor":"AIBD"},{"date":"2026-10-29","change":"Revised the depth ratings.","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["Consultants used GPT-4 on tasks that experts had chosen as within its reach.","Some consultants also received a short overview of prompt engineering.","Human graders scored the quality of the work."],"must_hold":["The task must lie within what the model can do well.","Workers must check output on tasks where the model is weak."],"echoes":[{"date":"2024-07-01","setting":"776 Procter & Gamble professionals on product development tasks with a GPT-4-based tool","outcome":"recurred","source":"https://www.nber.org/system/files/working_papers/w33641/w33641.pdf"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["Consultants used GPT-4 on tasks that experts had chosen as within its reach.","Some consultants also received a short overview of prompt engineering.","Human graders scored the quality of the work."],"must_hold":["The task must lie within what the model can do well.","Workers must check output on tasks where the model is weak."],"echoes":[{"date":"2024-07-01","setting":"776 Procter & Gamble professionals on product development tasks with a GPT-4-based tool","outcome":"recurred","source":"https://www.nber.org/system/files/working_papers/w33641/w33641.pdf"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://api.crossref.org/works/10.1287/orsc.2025.21838","editor":"AIBD"},{"date":"2026-10-29","change":"Linked a comparable incident in the AI Incident Database (incident 1193).","source":"https://incidentdatabase.ai/cite/1193","editor":"AIBD"},{"date":"2026-10-29","change":"Revised the depth ratings.","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Management consulting"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"peer-reviewed study","citation":"Dell'Acqua F, McFowland E III, Mollick E, Lifshitz H, Kellogg KC, Rajendran S, Krayer L, Candelon F, Lakhani KR. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality. Organization Science. 2026;37(2):403-423. doi:10.1287/orsc.2025.21838 (publisher abstract as deposited with Crossref).","title":"Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality","url":"https://doi.org/10.1287/orsc.2025.21838","date":"2026-03-01","written_by":"deployer","quote":"subjects using AI outperformed those not using AI, completing 12.2% more tasks and completing them 25.1% more quickly on average while also delivering solutions of significantly improved quality"},{"kind":"preprint","citation":"Dell'Acqua F, et al. Navigating the Jagged Technological Frontier. HBS Working Paper 24-013 (PDF, archived copy), 22 September 2023.","title":"Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality","url":"https://web.archive.org/web/20250102233043id_/https://www.hbs.edu/ris/Publication%20Files/24-013_d9b45b68-9e74-42d6-a1c6-c72fb70c7282.pdf","date":"2023-09-22","written_by":"deployer","quote":"Consultants across the skills distribution benefited significantly from having AI augmentation, with those below the average performance threshold increasing by 43% and those above increasing by 17% compared to their own scores."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0665-source-0001","AIBD-0665-source-0002"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"peer-reviewed study","citation":"Dell'Acqua F, McFowland E III, Mollick E, Lifshitz H, Kellogg KC, Rajendran S, Krayer L, Candelon F, Lakhani KR. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality. Organization Science. 2026;37(2):403-423. doi:10.1287/orsc.2025.21838 (publisher abstract as deposited with Crossref).","title":"Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality","url":"https://doi.org/10.1287/orsc.2025.21838","date":"2026-03-01","written_by":"deployer","quote":"subjects using AI outperformed those not using AI, completing 12.2% more tasks and completing them 25.1% more quickly on average while also delivering solutions of significantly improved quality"},{"kind":"preprint","citation":"Dell'Acqua F, et al. Navigating the Jagged Technological Frontier. HBS Working Paper 24-013 (PDF, archived copy), 22 September 2023.","title":"Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality","url":"https://web.archive.org/web/20250102233043id_/https://www.hbs.edu/ris/Publication%20Files/24-013_d9b45b68-9e74-42d6-a1c6-c72fb70c7282.pdf","date":"2023-09-22","written_by":"deployer","quote":"Consultants across the skills distribution benefited significantly from having AI augmentation, with those below the average performance threshold increasing by 43% and those above increasing by 17% compared to their own scores."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"In a preregistered experiment with 758 BCG consultants, GPT-4 users completed 12.2% more tasks 25.1% faster with higher quality, but erred more on a task beyond its reach.","account":"Researchers from Harvard Business School, Wharton, Warwick Business School, MIT Sloan and the Boston Consulting Group Henderson Institute ran a preregistered experiment with 758 BCG consultants, about 7% of the firm's individual contributor-level consultants. After a baseline task, the consultants were randomly assigned to work without AI, with GPT-4, or with GPT-4 and an overview of prompt engineering; all used GPT-4 as available at the end of April 2023 through a company platform built on the OpenAI API. On 18 realistic consulting tasks that the authors judged within the reach of the model, the consultants using AI completed 12.2% more tasks, finished 25.1% more quickly and produced work that human graders rated more than 40% higher in quality. Consultants below the average baseline score improved by 43% and those above it by 17%. On a business problem chosen to lie outside the model's reach, consultants using AI were 19 percentage points less likely to reach the correct answer. Harvard Business School released the working paper on 22 September 2023, and Organization Science published the study in March 2026. The tasks were designed for the experiment. Three of the nine authors worked for BCG, another advises BCG, and Harvard Business School paid for part of the research."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"value_present","value":{"kind":"record_fields","fields":{"model_version":"GPT-4 as available at the end of April 2023"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["model_version"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"GPT-4 (via a ChatGPT-style interface with default system prompt and temperature)"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"BCG consultants using GPT-4 finish 12.2% more tasks 25.1% faster with higher quality in a preregistered experiment (jagged frontier study)","headline":"Consultants with *GPT-4* finish more tasks in less time"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["c38b2d53e6f500e42f2d67fabaa051d757ad43b39b648a5f63ddc5cb1c7b9ce7"],"role":"SHA-256 of the complete public case and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public case, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"Researchers from Harvard Business School, Wharton, Warwick Business School, MIT Sloan and the Boston Consulting Group Henderson Institute ran a preregistered experiment with 758 BCG consultants, about 7% of the firm's individual contributor-level consultants. After a baseline task, the consultants were randomly assigned to work without AI, with GPT-4, or with GPT-4 and an overview of prompt engineering; all used GPT-4 as available at the end of April 2023 through a company platform built on the OpenAI API. On 18 realistic consulting tasks that the authors judged within the reach of the model, the consultants using AI completed 12.2% more tasks, finished 25.1% more quickly and produced work that human graders rated more than 40% higher in quality. Consultants below the average baseline score improved by 43% and those above it by 17%. On a business problem chosen to lie outside the model's reach, consultants using AI were 19 percentage points less likely to reach the correct answer. Harvard Business School released the working paper on 22 September 2023, and Organization Science published the study in March 2026. The tasks were designed for the experiment. Three of the nine authors worked for BCG, another advises BCG, and Harvard Business School paid for part of the research.","replication":{"produced_by":["Consultants used GPT-4 on tasks that experts had chosen as within its reach.","Some consultants also received a short overview of prompt engineering.","Human graders scored the quality of the work."],"must_hold":["The task must lie within what the model can do well.","Workers must check output on tasks where the model is weak."],"echoes":[{"date":"2024-07-01","setting":"776 Procter & Gamble professionals on product development tasks with a GPT-4-based tool","outcome":"recurred","source":"https://www.nber.org/system/files/working_papers/w33641/w33641.pdf"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":45,"reviewed":15,"reviewed_complete":15,"reviewed_partial":0,"record_mapped":30,"review_not_assessed":46,"not_assessed":0,"complete":false,"by_state":{"value_present":45,"not_reported":0,"unresolved":46,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"dd71cfe1e85981c2bab39a32155da690e140be0d1ae5150c9f95c862166e1713"}