{"id":"AIBD-0664","title":"Generative AI assistant raises customer support agents' resolutions per hour by 15% at a Fortune 500 software firm (Brynjolfsson, Li and Raymond)","summary":"A GPT-based chat assistant raised issues resolved per hour by 15% on average among 5,172 support agents, with the largest gains for novice and low-skilled agents.","date_observed":"2020-11-01","date_basis":"source information","date_precision":"month","benefit_status":"reported","source_collection":null,"source_entry":null,"account":"Erik Brynjolfsson of Stanford and Danielle Li and Lindsey Raymond of MIT studied the staggered rollout of a generative AI chat assistant to the technical support agents of a Fortune 500 enterprise software company that sells business process software to small and medium-sized businesses in the United States. Most of the agents worked from offices in the Philippines, either for the firm or for outside contractors. An AI customer-service software company that the authors do not name built the tool on OpenAI's GPT models, and the tool suggested replies to agents in real time. A seven-week randomized pilot with 50 agents came first, and most agents gained access between November 2020 and February 2021. In the 2023 working paper, which covers about 3 million chats by 5,179 agents, the authors report that access raised issues resolved per hour by 14% on average and by 34% for novice and low-skilled agents. The version that the Quarterly Journal of Economics published in 2025 covers 5,172 agents and reports a gain of 15% on average. Customers were more polite and asked less often to speak to a manager, and attrition among agents with less than six months of tenure fell by about 10 percentage points. The most experienced and highest-skilled agents saw small gains in speed and small declines in quality. The study covers one firm, and the authors write that the attrition results call for more caution than the productivity results. The Stanford Digital Economy Lab funded the work.","category":"general benefit","unexpected":false,"purpose":"The assistant was built to suggest responses to customer support agents during live chats. Faster and better resolution of customer issues was that purpose.","system":"Generative AI conversational assistant for customer support agents, built on OpenAI's GPT family with additional machine-learning models trained on customer-service conversations","model_version":null,"developer":"An AI customer-service software company that the authors do not name (the 'AI firm'), building on OpenAI's GPT models","deployer":"A Fortune 500 enterprise software company (unnamed) and its outsourced support contractors","location":"Support offices mainly in the Philippines, with smaller groups in the United States and other countries","domain":"Customer service","segment":null,"benefit_type":"Improved Organizational Efficiency","taxonomy_note":null,"registry_ids":{},"ratings":[{"group":"personal","depth":2,"reason":"Novice and low-skilled agents resolved 34% more issues per hour in the working paper estimate, and attrition among agents with under six months of tenure fell by about 10 percentage points."},{"group":"organizational","depth":2,"reason":"The firm's agents resolved 15% more issues per hour on average after gaining access."}],"ai_role":{"status":"assessed","statement":"The authors compare agents before and after access with agents who had not yet received the tool, in a staggered rollout that began with a small randomized pilot, so the estimate isolates the effect of access to the assistant."},"evidence":[{"kind":"peer-reviewed study","citation":"Brynjolfsson E, Li D, Raymond L. Generative AI at Work. Quarterly Journal of Economics. 2025;140(2):889-942. doi:10.1093/qje/qjae044 (abstract as listed on RePEc IDEAS).","title":"Generative AI at Work","url":"https://ideas.repec.org/a/oup/qjecon/v140y2025i2p889-942..html","date":"2025-02-04","written_by":"independent","quote":"Access to AI assistance increases worker productivity, as measured by issues resolved per hour, by 15% on average, with substantial heterogeneity across workers."},{"kind":"preprint","citation":"Brynjolfsson E, Li D, Raymond LR. Generative AI at Work. NBER Working Paper 31161, April 2023, revised November 2023.","title":"Generative AI at Work (NBER Working Paper 31161)","url":"https://www.nber.org/system/files/working_papers/w31161/w31161.pdf","date":"2023-11-01","written_by":"independent","quote":"Access to the tool increases productivity, as measured by issues resolved per hour, by 14% on average, including a 34% improvement for novice and low-skilled workers but with minimal impact on experienced and highly skilled workers."}],"replication":{"produced_by":["Agents received real-time reply suggestions inside live text chats.","The model was trained on a large set of past chats labeled with outcomes.","Agents stayed free to ignore the suggestions."],"must_hold":["The work must consist of repeated text conversations on recurring problems.","The firm must hold enough past conversations to train the model.","Gains depend on a supply of less experienced agents."],"echoes":[{"date":"2022-09-01","setting":"Software developers at Microsoft, Accenture and an anonymous Fortune 100 company given GitHub Copilot","model_version":null,"outcome":"recurred","source":"https://economics.mit.edu/sites/default/files/inline-files/draft_copilot_experiments.pdf"}]},"paired_incident":{"database":"AI Incident Database","identifier":"631","title":"Chatbot for DPD Malfunctioned and Swore at Customers and Criticized Its Own Company","url":"https://incidentdatabase.ai/cite/631","severity":null,"group":"organizational","basis":"Both cases concern generative AI that writes replies in a company's customer service chats, where faulty replies fall on the company that provides the support."},"headline":"A chat assistant helps new support agents resolve more issues","metric":"raised issues resolved per hour by 15% on average","comparator":{"against":"Agents without access to the assistant","before":"Issues resolved per hour without access","after":"15% more issues resolved per hour on average","basis":"measured","quote":"Access to AI assistance increases worker productivity, as measured by issues resolved per hour, by 15% on average, with substantial heterogeneity across workers.","url":"https://ideas.repec.org/a/oup/qjecon/v140y2025i2p889-942..html"},"later":null,"value":null,"developer_country":"Not stated","deployer_country":"United States","unrated_because":null,"brief_notes":null,"brief":null,"revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://ideas.repec.org/a/oup/qjecon/v140y2025i2p889-942..html","editor":"AIBD"},{"date":"2026-10-29","change":"Linked a comparable incident in the AI Incident Database (incident 631).","source":"https://incidentdatabase.ai/cite/631","editor":"AIBD"}],"confidence":"studied","shade":"high","claim_status":"Independently studied","claim_basis":"Sources: one independent peer-reviewed study and one independent preprint. The cited evidence includes no finding by a regulator or court.","case_jurisdictions":["Philippines","United States"],"profile":{"case_id":"AIBD-0664","catalog_version":"1.0.0","source_fingerprint":"de5188b34fc5487c992507db18f03d6f6b3bb5cc90db8c8e7f0aed4e1dfc4076","collection_refs":[{"collection_id":"aibd","version":"2026-10-29"}],"source_ids":["AIBD-0664-source-0001","AIBD-0664-source-0002"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0664-evaluation-1"],"role":"Evaluations of the benefit"},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":[],"source_refs":["AIBD-0664-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"partial","record_ids":["AIBD-0664-evaluation-1"]},"monetary observations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"An AI customer-service software company that the authors do not name (the 'AI firm'), building on OpenAI's GPT models","deployer":"A Fortune 500 enterprise software company (unnamed) and its outsourced support contractors"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer","deployer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"value_present","value":{"kind":"record_fields","fields":{"ai_role":{"status":"assessed","statement":"The authors compare agents before and after access with agents who had not yet received the tool, in a staggered rollout that began with a small randomized pilot, so the estimate isolates the effect of access to the assistant."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"general benefit","ai_role":{"status":"assessed","statement":"The authors compare agents before and after access with agents who had not yet received the tool, in a staggered rollout that began with a small randomized pilot, so the estimate isolates the effect of access to the assistant."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"Novice and low-skilled agents resolved 34% more issues per hour in the working paper estimate, and attrition among agents with under six months of tenure fell by about 10 percentage points."},{"group":"organizational","depth":2,"reason":"The firm's agents resolved 15% more issues per hour on average after gaining access."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"high","evidence_word":"studied","claim_status":"Independently studied","basis":"Sources: one independent peer-reviewed study and one independent preprint. The cited evidence includes no finding by a regulator or court.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"Novice and low-skilled agents resolved 34% more issues per hour in the working paper estimate, and attrition among agents with under six months of tenure fell by about 10 percentage points."},{"group":"organizational","depth":2,"reason":"The firm's agents resolved 15% more issues per hour on average after gaining access."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"Novice and low-skilled agents resolved 34% more issues per hour in the working paper estimate, and attrition among agents with under six months of tenure fell by about 10 percentage points."},{"group":"organizational","depth":2,"reason":"The firm's agents resolved 15% more issues per hour on average after gaining access."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"value_present","value":{"kind":"structured","data":{"classification":"evaluation or study","answer":"The case describes an evaluation or a study. An academic study of a workplace rollout.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0664-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The case describes an evaluation or a study. An academic study of a workplace rollout.","inferred":false}},"case_links[]":{"state":"value_present","value":{"kind":"structured","data":{"answer":"Both cases concern generative AI that writes replies in a company's customer service chats, where faulty replies fall on the company that provides the support.","links":[{"database":"AI Incident Database","identifier":"631","title":"Chatbot for DPD Malfunctioned and Swore at Customers and Criticized Its Own Company","url":"https://incidentdatabase.ai/cite/631","group":"organizational","basis":"Both cases concern generative AI that writes replies in a company's customer service chats, where faulty replies fall on the company that provides the support.","relation":"comparable_harm"}]}},"provenance":{"kind":"curatorial","basis":"Compared with the incidents in the AI Incident Database snapshot of 26 October 2026; a second reading confirmed that the incident concerns the same kind of system and stakeholder.","record_fields":["paired_incident"],"source_refs":["https://incidentdatabase.ai/cite/631"]},"review_state":"reviewed","coverage":"complete","display":{"answer":"Both cases concern generative AI that writes replies in a company's customer service chats, where faulty replies fall on the company that provides the support.","inferred":false}},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"Independently studied. Sources: one independent peer-reviewed study and one independent preprint. The cited evidence includes no finding by a regulator or court.","evidence_word":"studied","claim_status":"Independently studied"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0664-source-0001","AIBD-0664-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Independently studied. Sources: one independent peer-reviewed study and one independent preprint. The cited evidence includes no finding by a regulator or court.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"routine use","answer":"The system was in routine use. The firm rolled the tool out to its own and outsourced agents after a pilot.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0664-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was in routine use. The firm rolled the tool out to its own and outsourced agents after a pilot.","inferred":false}},"economics[].amount":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].costs[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].counterfactual":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].currency_price_basis":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].gross_net":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].measure_type":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].period_unit":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].realization":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].recipient_payer":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].return_payback":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].valuation_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"evaluations[].ai_uplift":{"state":"value_present","value":{"kind":"text","text":"About 14% to 15% more resolutions per hour."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0664-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"About 14% to 15% more resolutions per hour.","inferred":false}},"evaluations[].comparator":{"state":"value_present","value":{"kind":"text","text":"Agents not yet given access, with agent and time fixed effects."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0664-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Agents not yet given access, with agent and time fixed effects.","inferred":false}},"evaluations[].design":{"state":"value_present","value":{"kind":"text","text":"Quasi-experimental."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0664-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Quasi-experimental.","inferred":false}},"evaluations[].effect":{"state":"value_present","value":{"kind":"text","text":"14% on average and 34% for novice and low-skilled agents (working paper); 15% on average (published)."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0664-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"14% on average and 34% for novice and low-skilled agents (working paper); 15% on average (published).","inferred":false}},"evaluations[].exposure":{"state":"value_present","value":{"kind":"text","text":"Access to real-time AI suggestions during chats."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0664-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Access to real-time AI suggestions during chats.","inferred":false}},"evaluations[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0664-evaluation-1"],"role":"Evaluations of the benefit"},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0664-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Quasi-experimental.","inferred":false}},"evaluations[].measurement_method":{"state":"value_present","value":{"kind":"text","text":"Issues resolved per hour, resolution rate, handle time, net promoter score and attrition from the firm's systems."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0664-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Issues resolved per hour, resolution rate, handle time, net promoter score and attrition from the firm's systems.","inferred":false}},"evaluations[].metric":{"state":"value_present","value":{"kind":"text","text":"Resolutions per hour."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0664-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Resolutions per hour.","inferred":false}},"evaluations[].reproduction_materials":{"state":"value_present","value":{"kind":"text","text":"Firm data not public."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0664-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Firm data not public.","inferred":false}},"evaluations[].setting_sample":{"state":"value_present","value":{"kind":"text","text":"5,179 agents and about 3 million chats in the working paper; 5,172 agents in the published version."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0664-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"5,179 agents and about 3 million chats in the working paper; 5,172 agents in the published version.","inferred":false}},"evaluations[].uncertainty_limitations":{"state":"value_present","value":{"kind":"text","text":"One firm; staggered rollout chosen by the firm; attrition estimates lack agent fixed effects; small quality declines for the most skilled agents."},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0664-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"One firm; staggered rollout chosen by the firm; attrition estimates lack agent fixed effects; small quality declines for the most skilled agents.","inferred":false}},"external_references[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"paired_incident":{"database":"AI Incident Database","identifier":"631","title":"Chatbot for DPD Malfunctioned and Swore at Customers and Criticized Its Own Company","url":"https://incidentdatabase.ai/cite/631","group":"organizational","basis":"Both cases concern generative AI that writes replies in a company's customer service chats, where faulty replies fall on the company that provides the support."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["paired_incident"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"follow_up[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-0664"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Improved Organizational Efficiency"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"Erik Brynjolfsson of Stanford and Danielle Li and Lindsey Raymond of MIT studied the staggered rollout of a generative AI chat assistant to the technical support agents of a Fortune 500 enterprise software company that sells business process software to small and medium-sized businesses in the United States. Most of the agents worked from offices in the Philippines, either for the firm or for outside contractors. An AI customer-service software company that the authors do not name built the tool on OpenAI's GPT models, and the tool suggested replies to agents in real time. A seven-week randomized pilot with 50 agents came first, and most agents gained access between November 2020 and February 2021. In the 2023 working paper, which covers about 3 million chats by 5,179 agents, the authors report that access raised issues resolved per hour by 14% on average and by 34% for novice and low-skilled agents. The version that the Quarterly Journal of Economics published in 2025 covers 5,172 agents and reports a gain of 15% on average. Customers were more polite and asked less often to speak to a manager, and attrition among agents with less than six months of tenure fell by about 10 percentage points. The most experienced and highest-skilled agents saw small gains in speed and small declines in quality. The study covers one firm, and the authors write that the attrition results call for more caution than the productivity results. The Stanford Digital Economy Lab funded the work.","benefit_type":"Improved Organizational Efficiency"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"Novice and low-skilled agents resolved 34% more issues per hour in the working paper estimate, and attrition among agents with under six months of tenure fell by about 10 percentage points."},{"group":"organizational","depth":2,"reason":"The firm's agents resolved 15% more issues per hour on average after gaining access."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"reported"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"The assistant was built to suggest responses to customer support agents during live chats. Faster and better resolution of customer issues was that purpose."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"Support offices mainly in the Philippines, with smaller groups in the United States and other countries","developer_country":"Not stated","deployer_country":"United States"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","developer_country","deployer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"The assistant was built to suggest responses to customer support agents during live chats. Faster and better resolution of customer issues was that purpose."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2020-11-01","date_basis":"source information","date_precision":"month","revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://ideas.repec.org/a/oup/qjecon/v140y2025i2p889-942..html","editor":"AIBD"},{"date":"2026-10-29","change":"Linked a comparable incident in the AI Incident Database (incident 631).","source":"https://incidentdatabase.ai/cite/631","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["Agents received real-time reply suggestions inside live text chats.","The model was trained on a large set of past chats labeled with outcomes.","Agents stayed free to ignore the suggestions."],"must_hold":["The work must consist of repeated text conversations on recurring problems.","The firm must hold enough past conversations to train the model.","Gains depend on a supply of less experienced agents."],"echoes":[{"date":"2022-09-01","setting":"Software developers at Microsoft, Accenture and an anonymous Fortune 100 company given GitHub Copilot","outcome":"recurred","source":"https://economics.mit.edu/sites/default/files/inline-files/draft_copilot_experiments.pdf"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["Agents received real-time reply suggestions inside live text chats.","The model was trained on a large set of past chats labeled with outcomes.","Agents stayed free to ignore the suggestions."],"must_hold":["The work must consist of repeated text conversations on recurring problems.","The firm must hold enough past conversations to train the model.","Gains depend on a supply of less experienced agents."],"echoes":[{"date":"2022-09-01","setting":"Software developers at Microsoft, Accenture and an anonymous Fortune 100 company given GitHub Copilot","outcome":"recurred","source":"https://economics.mit.edu/sites/default/files/inline-files/draft_copilot_experiments.pdf"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-10-29","change":"Added the case.","source":"https://ideas.repec.org/a/oup/qjecon/v140y2025i2p889-942..html","editor":"AIBD"},{"date":"2026-10-29","change":"Linked a comparable incident in the AI Incident Database (incident 631).","source":"https://incidentdatabase.ai/cite/631","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Customer service"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"peer-reviewed study","citation":"Brynjolfsson E, Li D, Raymond L. Generative AI at Work. Quarterly Journal of Economics. 2025;140(2):889-942. doi:10.1093/qje/qjae044 (abstract as listed on RePEc IDEAS).","title":"Generative AI at Work","url":"https://ideas.repec.org/a/oup/qjecon/v140y2025i2p889-942..html","date":"2025-02-04","written_by":"independent","quote":"Access to AI assistance increases worker productivity, as measured by issues resolved per hour, by 15% on average, with substantial heterogeneity across workers."},{"kind":"preprint","citation":"Brynjolfsson E, Li D, Raymond LR. Generative AI at Work. NBER Working Paper 31161, April 2023, revised November 2023.","title":"Generative AI at Work (NBER Working Paper 31161)","url":"https://www.nber.org/system/files/working_papers/w31161/w31161.pdf","date":"2023-11-01","written_by":"independent","quote":"Access to the tool increases productivity, as measured by issues resolved per hour, by 14% on average, including a 34% improvement for novice and low-skilled workers but with minimal impact on experienced and highly skilled workers."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0664-source-0001","AIBD-0664-source-0002"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"peer-reviewed study","citation":"Brynjolfsson E, Li D, Raymond L. Generative AI at Work. Quarterly Journal of Economics. 2025;140(2):889-942. doi:10.1093/qje/qjae044 (abstract as listed on RePEc IDEAS).","title":"Generative AI at Work","url":"https://ideas.repec.org/a/oup/qjecon/v140y2025i2p889-942..html","date":"2025-02-04","written_by":"independent","quote":"Access to AI assistance increases worker productivity, as measured by issues resolved per hour, by 15% on average, with substantial heterogeneity across workers."},{"kind":"preprint","citation":"Brynjolfsson E, Li D, Raymond LR. Generative AI at Work. NBER Working Paper 31161, April 2023, revised November 2023.","title":"Generative AI at Work (NBER Working Paper 31161)","url":"https://www.nber.org/system/files/working_papers/w31161/w31161.pdf","date":"2023-11-01","written_by":"independent","quote":"Access to the tool increases productivity, as measured by issues resolved per hour, by 14% on average, including a 34% improvement for novice and low-skilled workers but with minimal impact on experienced and highly skilled workers."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"A GPT-based chat assistant raised issues resolved per hour by 15% on average among 5,172 support agents, with the largest gains for novice and low-skilled agents.","account":"Erik Brynjolfsson of Stanford and Danielle Li and Lindsey Raymond of MIT studied the staggered rollout of a generative AI chat assistant to the technical support agents of a Fortune 500 enterprise software company that sells business process software to small and medium-sized businesses in the United States. Most of the agents worked from offices in the Philippines, either for the firm or for outside contractors. An AI customer-service software company that the authors do not name built the tool on OpenAI's GPT models, and the tool suggested replies to agents in real time. A seven-week randomized pilot with 50 agents came first, and most agents gained access between November 2020 and February 2021. In the 2023 working paper, which covers about 3 million chats by 5,179 agents, the authors report that access raised issues resolved per hour by 14% on average and by 34% for novice and low-skilled agents. The version that the Quarterly Journal of Economics published in 2025 covers 5,172 agents and reports a gain of 15% on average. Customers were more polite and asked less often to speak to a manager, and attrition among agents with less than six months of tenure fell by about 10 percentage points. The most experienced and highest-skilled agents saw small gains in speed and small declines in quality. The study covers one firm, and the authors write that the attrition results call for more caution than the productivity results. The Stanford Digital Economy Lab funded the work."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"Generative AI conversational assistant for customer support agents, built on OpenAI's GPT family with additional machine-learning models trained on customer-service conversations"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"Generative AI assistant raises customer support agents' resolutions per hour by 15% at a Fortune 500 software firm (Brynjolfsson, Li and Raymond)","headline":"A chat assistant helps new support agents resolve more issues"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["9ef1083d747e1e3afffcfa082221301f938269c412db2bfe58800bea1ca04e24"],"role":"SHA-256 of the complete public case and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public case, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"Erik Brynjolfsson of Stanford and Danielle Li and Lindsey Raymond of MIT studied the staggered rollout of a generative AI chat assistant to the technical support agents of a Fortune 500 enterprise software company that sells business process software to small and medium-sized businesses in the United States. Most of the agents worked from offices in the Philippines, either for the firm or for outside contractors. An AI customer-service software company that the authors do not name built the tool on OpenAI's GPT models, and the tool suggested replies to agents in real time. A seven-week randomized pilot with 50 agents came first, and most agents gained access between November 2020 and February 2021. In the 2023 working paper, which covers about 3 million chats by 5,179 agents, the authors report that access raised issues resolved per hour by 14% on average and by 34% for novice and low-skilled agents. The version that the Quarterly Journal of Economics published in 2025 covers 5,172 agents and reports a gain of 15% on average. Customers were more polite and asked less often to speak to a manager, and attrition among agents with less than six months of tenure fell by about 10 percentage points. The most experienced and highest-skilled agents saw small gains in speed and small declines in quality. The study covers one firm, and the authors write that the attrition results call for more caution than the productivity results. The Stanford Digital Economy Lab funded the work.","replication":{"produced_by":["Agents received real-time reply suggestions inside live text chats.","The model was trained on a large set of past chats labeled with outcomes.","Agents stayed free to ignore the suggestions."],"must_hold":["The work must consist of repeated text conversations on recurring problems.","The firm must hold enough past conversations to train the model.","Gains depend on a supply of less experienced agents."],"echoes":[{"date":"2022-09-01","setting":"Software developers at Microsoft, Accenture and an anonymous Fortune 100 company given GitHub Copilot","outcome":"recurred","source":"https://economics.mit.edu/sites/default/files/inline-files/draft_copilot_experiments.pdf"}]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":44,"reviewed":15,"reviewed_complete":15,"reviewed_partial":0,"record_mapped":29,"review_not_assessed":47,"not_assessed":0,"complete":false,"by_state":{"value_present":44,"not_reported":0,"unresolved":47,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"45107e4a7aedda1bfd651372e9c55a74710fe8cad6d385a927c49d87832ecf92"}