{"id":"AIBD-1171","title":"The World Bank's Independent Evaluation Group finds that an AI-enhanced chatbot helped UNDP monitor the Yemen Emergency Crisis Response Project","summary":"The World Bank's evaluation office found that UNDP used WhatsApp and an AI-enhanced chatbot for remote, real-time monitoring in Yemen, which improved transparency and complaint handling.","date_observed":"2022-03-31","date_basis":"event","date_precision":"day","benefit_status":"reported","source_collection":null,"source_entry":null,"account":"The Yemen Emergency Crisis Response Project ran from July 2016 to March 2022 with US$887.4 million of World Bank financing, under agreements with UNDP and UNICEF. The World Bank's Independent Evaluation Group reviewed the project's completion report on 29 August 2023 and rated the outcome highly satisfactory. In its assessment of monitoring, the evaluators report that UNDP's project management unit carried out 336 field visits. The unit used common social media platforms, such as WhatsApp and a chatbot enhanced by artificial intelligence. The evaluators write that these tools enhanced remote and real-time monitoring, transparency and communication, including the submission of complaints and responses to queries. They rated the quality of monitoring and evaluation substantial. The Bank's own completion report of 3 March 2023 describes the chatbot in the same words. Neither document names the chatbot, describes how artificial intelligence enhanced it or separates its contribution from that of WhatsApp, the field visits and the other monitoring channels.","category":"benefit created","unexpected":false,"purpose":"An unnamed developer built the chatbot, which UNDP's project management unit used with WhatsApp to monitor the project remotely and in real time, receive complaints and answer queries.","system":"Chatbot enhanced by artificial intelligence, used by UNDP's Yemen project management unit with WhatsApp (not named in the review)","model_version":null,"developer":"Not named in the review","deployer":"UNDP project management unit for the Yemen Emergency Crisis Response Project","location":"Yemen","domain":"Humanitarian assistance","segment":null,"benefit_type":"Improved Organizational Efficiency","work_activities":["4.A.1.a.2","4.A.4.a.3","4.A.4.a.7"],"taxonomy_note":"The gain is in remote monitoring, transparency and complaint handling for a project in a conflict zone, which the taxonomy covers only loosely.","registry_ids":{},"ratings":[{"group":"organizational","depth":2,"reason":"The evaluators credit the chatbot and WhatsApp with enhanced remote and real-time monitoring, transparency and communication across 336 field visits, without separating the chatbot's share."},{"group":"personal","depth":1,"reason":"Project participants could submit complaints and get answers to queries through the chatbot and WhatsApp, a convenience whose uptake the review does not count."}],"ai_role":{"status":"unassessed","statement":"The evaluators call the chatbot 'enhanced by artificial intelligence' and credit it, with WhatsApp, for enhanced remote monitoring, transparency and complaint handling. They do not name the chatbot, describe what the artificial intelligence does or compare monitoring with and without it. Neither document shows that the system learns from data."},"evidence":[{"kind":"evaluation report","citation":"World Bank Independent Evaluation Group. Implementation Completion Report (ICR) Review: RY: Yemen Emergency Crisis Response Proj (P159053), ICRR0023450, 29 August 2023.","title":"Implementation Completion Report (ICR) Review: RY: Yemen Emergency Crisis Response Proj (P159053)","url":"http://documents.worldbank.org/curated/en/099082923164015722/pdf/P159053154192c08f1a9281213e9202e4c1.pdf","date":"2023-08-29","written_by":"independent","quote":"UNDP’s PMU carried out 336 field visits using common social media platforms, such as WhatsApp and a chatbot enhanced by artificial intelligence, to enhance remote and real-time monitoring, which enhanced transparency and communication, including enabling submission of complaints, and responding to queries."},{"kind":"report","citation":"World Bank. Implementation Completion and Results Report: Yemen Emergency Crisis Response Project (P159053). ICR00005809, 3 March 2023. The Bank's own completion report, which the Independent Evaluation Group reviewed.","title":"Implementation Completion and Results Report: Yemen Emergency Crisis Response Project (P159053)","url":"http://documents.worldbank.org/curated/en/099212503092327158/pdf/BOSIB041a7349a0480b905060df01a9c4cc.pdf","date":"2023-03-03","written_by":"evaluator","quote":"UNDP used common social media platforms, such as WhatsApp and a chatbot enhanced by artificial intelligence, to enhance remote and real-time monitoring, which enhanced transparency and communication, including enabling submission of complaints, and responding to queries."}],"replication":{"produced_by":["UNDP's project management unit used WhatsApp and a chatbot enhanced by artificial intelligence alongside 336 field visits.","Implementing partners, third-party monitors and field consultants set up real-time channels to report security threats on the ground."],"must_hold":["Project staff and communities must have mobile phones and connectivity for the social media platforms.","Complaints and queries received through the chatbot must reach staff who act on them."],"echoes":[]},"paired_incident":null,"headline":"AI chatbot and WhatsApp help UNDP monitor a Yemen project remotely","metric":"336 field visits","comparator":null,"later":null,"benefit_dimensions":["functional","societal"],"benefit_categories":["operational_improvement","social_improvement"],"scsp_example":"Faster data management, decision-making","size":{"reach":{"band":1,"fact":"The monitoring gain went to one organization, UNDP's project management unit, and complainants are not counted.","quote":"UNDP’s PMU carried out 336 field visits using common social media platforms, such as WhatsApp and a chatbot enhanced by artificial intelligence, to enhance remote and real-time monitoring, which enhanced transparency and communication, including enabling submission of complaints, and responding to queries.","url":"http://documents.worldbank.org/curated/en/099082923164015722/pdf/P159053154192c08f1a9281213e9202e4c1.pdf","reason":null},"duration":{"band":null,"fact":null,"quote":null,"url":null,"reason":"The reviews do not say when or for how long the chatbot was used."},"distribution":{"band":null,"fact":null,"quote":null,"url":null,"reason":"The reviews give no results by group and do not count who used the chatbot."},"frequency":{"band":null,"fact":null,"quote":null,"url":null,"reason":"The reviews do not say how often the chatbot was used or over what period the field visits ran."}},"value":null,"developer_country":null,"deployer_country":"Yemen","unrated_because":null,"brief_notes":null,"brief":null,"revisions":[{"date":"2026-11-01","change":"Added the record from the finding of the World Bank Independent Evaluation Group.","source":"http://documents.worldbank.org/curated/en/099082923164015722/pdf/P159053154192c08f1a9281213e9202e4c1.pdf","editor":"AIBD"},{"date":"2026-11-01","change":"Placed the benefit under its broad categories and recorded its reach, duration, distribution and frequency from its sources.","source":null,"editor":"AIBD"}],"confidence":"found","shade":"high","claim_status":"Found by an evaluation office","claim_basis":"Sources: one independent evaluation report and the evaluators' report. The cited evidence includes no independent study or research report.","case_jurisdictions":["Yemen"],"profile":{"case_id":"AIBD-1171","catalog_version":"1.0.0","source_fingerprint":"b6781fb19641d2fe33d8248f022d9526bab89e95f8f6a957058642a1ce2213cc","collection_refs":[{"collection_id":"aibd","version":"2026-10-31"},{"collection_id":"querist","version":"2026-10-25"}],"source_ids":["AIBD-1171-source-0001","AIBD-1171-source-0002"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"monetary observations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"Not named in the review","deployer":"UNDP project management unit for the Yemen Emergency Crisis Response Project"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer","deployer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"benefit created","ai_role":{"status":"unassessed","statement":"The evaluators call the chatbot 'enhanced by artificial intelligence' and credit it, with WhatsApp, for enhanced remote monitoring, transparency and complaint handling. They do not name the chatbot, describe what the artificial intelligence does or compare monitoring with and without it. Neither document shows that the system learns from data."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"The evaluators credit the chatbot and WhatsApp with enhanced remote and real-time monitoring, transparency and communication across 336 field visits, without separating the chatbot's share."},{"group":"personal","depth":1,"reason":"Project participants could submit complaints and get answers to queries through the chatbot and WhatsApp, a convenience whose uptake the review does not count."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"high","evidence_word":"found","claim_status":"Found by an evaluation office","basis":"Sources: one independent evaluation report and the evaluators' report. The cited evidence includes no independent study or research report.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"The evaluators credit the chatbot and WhatsApp with enhanced remote and real-time monitoring, transparency and communication across 336 field visits, without separating the chatbot's share."},{"group":"personal","depth":1,"reason":"Project participants could submit complaints and get answers to queries through the chatbot and WhatsApp, a convenience whose uptake the review does not count."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"The evaluators credit the chatbot and WhatsApp with enhanced remote and real-time monitoring, transparency and communication across 336 field visits, without separating the chatbot's share."},{"group":"personal","depth":1,"reason":"Project participants could submit complaints and get answers to queries through the chatbot and WhatsApp, a convenience whose uptake the review does not count."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_links[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"Found by an evaluation office. Sources: one independent evaluation report and the evaluators' report. The cited evidence includes no independent study or report.","evidence_word":"found","claim_status":"Found by an evaluation office"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-1171-source-0001","AIBD-1171-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Found by an evaluation office. Sources: one independent evaluation report and the evaluators' report. The cited evidence includes no independent study or report.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"routine use","answer":"The system was in routine use. The chatbot was in use for project monitoring across the implementation period, according to the review.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-1171-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was in routine use. The chatbot was in use for project monitoring across the implementation period, according to the review.","inferred":false}},"economics[].amount":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].costs[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].counterfactual":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].currency_price_basis":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].gross_net":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].measure_type":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].period_unit":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].realization":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].recipient_payer":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].return_payback":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].valuation_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"evaluations[].ai_uplift":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].comparator":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].design":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].effect":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].exposure":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].id":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].measurement_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].metric":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].reproduction_materials":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].setting_sample":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].uncertainty_limitations":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"external_references[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"follow_up[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-1171"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Improved Organizational Efficiency","taxonomy_note":"The gain is in remote monitoring, transparency and complaint handling for a project in a conflict zone, which the taxonomy covers only loosely."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type","taxonomy_note"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"The Yemen Emergency Crisis Response Project ran from July 2016 to March 2022 with US$887.4 million of World Bank financing, under agreements with UNDP and UNICEF. The World Bank's Independent Evaluation Group reviewed the project's completion report on 29 August 2023 and rated the outcome highly satisfactory. In its assessment of monitoring, the evaluators report that UNDP's project management unit carried out 336 field visits. The unit used common social media platforms, such as WhatsApp and a chatbot enhanced by artificial intelligence. The evaluators write that these tools enhanced remote and real-time monitoring, transparency and communication, including the submission of complaints and responses to queries. They rated the quality of monitoring and evaluation substantial. The Bank's own completion report of 3 March 2023 describes the chatbot in the same words. Neither document names the chatbot, describes how artificial intelligence enhanced it or separates its contribution from that of WhatsApp, the field visits and the other monitoring channels.","benefit_type":"Improved Organizational Efficiency"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"The evaluators credit the chatbot and WhatsApp with enhanced remote and real-time monitoring, transparency and communication across 336 field visits, without separating the chatbot's share."},{"group":"personal","depth":1,"reason":"Project participants could submit complaints and get answers to queries through the chatbot and WhatsApp, a convenience whose uptake the review does not count."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"reported"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"An unnamed developer built the chatbot, which UNDP's project management unit used with WhatsApp to monitor the project remotely and in real time, receive complaints and answer queries."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"Yemen","deployer_country":"Yemen"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","deployer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"An unnamed developer built the chatbot, which UNDP's project management unit used with WhatsApp to monitor the project remotely and in real time, receive complaints and answer queries."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2022-03-31","date_basis":"event","date_precision":"day","revisions":[{"date":"2026-11-01","change":"Added the record from the finding of the World Bank Independent Evaluation Group.","source":"http://documents.worldbank.org/curated/en/099082923164015722/pdf/P159053154192c08f1a9281213e9202e4c1.pdf","editor":"AIBD"},{"date":"2026-11-01","change":"Placed the benefit under its broad categories and recorded its reach, duration, distribution and frequency from its sources.","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["UNDP's project management unit used WhatsApp and a chatbot enhanced by artificial intelligence alongside 336 field visits.","Implementing partners, third-party monitors and field consultants set up real-time channels to report security threats on the ground."],"must_hold":["Project staff and communities must have mobile phones and connectivity for the social media platforms.","Complaints and queries received through the chatbot must reach staff who act on them."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-11-01","change":"Added the record from the finding of the World Bank Independent Evaluation Group.","source":"http://documents.worldbank.org/curated/en/099082923164015722/pdf/P159053154192c08f1a9281213e9202e4c1.pdf","editor":"AIBD"},{"date":"2026-11-01","change":"Placed the benefit under its broad categories and recorded its reach, duration, distribution and frequency from its sources.","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Humanitarian assistance"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"evaluation report","citation":"World Bank Independent Evaluation Group. Implementation Completion Report (ICR) Review: RY: Yemen Emergency Crisis Response Proj (P159053), ICRR0023450, 29 August 2023.","title":"Implementation Completion Report (ICR) Review: RY: Yemen Emergency Crisis Response Proj (P159053)","url":"http://documents.worldbank.org/curated/en/099082923164015722/pdf/P159053154192c08f1a9281213e9202e4c1.pdf","date":"2023-08-29","written_by":"independent","quote":"UNDP’s PMU carried out 336 field visits using common social media platforms, such as WhatsApp and a chatbot enhanced by artificial intelligence, to enhance remote and real-time monitoring, which enhanced transparency and communication, including enabling submission of complaints, and responding to queries."},{"kind":"report","citation":"World Bank. Implementation Completion and Results Report: Yemen Emergency Crisis Response Project (P159053). ICR00005809, 3 March 2023. The Bank's own completion report, which the Independent Evaluation Group reviewed.","title":"Implementation Completion and Results Report: Yemen Emergency Crisis Response Project (P159053)","url":"http://documents.worldbank.org/curated/en/099212503092327158/pdf/BOSIB041a7349a0480b905060df01a9c4cc.pdf","date":"2023-03-03","written_by":"evaluator","quote":"UNDP used common social media platforms, such as WhatsApp and a chatbot enhanced by artificial intelligence, to enhance remote and real-time monitoring, which enhanced transparency and communication, including enabling submission of complaints, and responding to queries."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-1171-source-0001","AIBD-1171-source-0002"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"evaluation report","citation":"World Bank Independent Evaluation Group. Implementation Completion Report (ICR) Review: RY: Yemen Emergency Crisis Response Proj (P159053), ICRR0023450, 29 August 2023.","title":"Implementation Completion Report (ICR) Review: RY: Yemen Emergency Crisis Response Proj (P159053)","url":"http://documents.worldbank.org/curated/en/099082923164015722/pdf/P159053154192c08f1a9281213e9202e4c1.pdf","date":"2023-08-29","written_by":"independent","quote":"UNDP’s PMU carried out 336 field visits using common social media platforms, such as WhatsApp and a chatbot enhanced by artificial intelligence, to enhance remote and real-time monitoring, which enhanced transparency and communication, including enabling submission of complaints, and responding to queries."},{"kind":"report","citation":"World Bank. Implementation Completion and Results Report: Yemen Emergency Crisis Response Project (P159053). ICR00005809, 3 March 2023. The Bank's own completion report, which the Independent Evaluation Group reviewed.","title":"Implementation Completion and Results Report: Yemen Emergency Crisis Response Project (P159053)","url":"http://documents.worldbank.org/curated/en/099212503092327158/pdf/BOSIB041a7349a0480b905060df01a9c4cc.pdf","date":"2023-03-03","written_by":"evaluator","quote":"UNDP used common social media platforms, such as WhatsApp and a chatbot enhanced by artificial intelligence, to enhance remote and real-time monitoring, which enhanced transparency and communication, including enabling submission of complaints, and responding to queries."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"The World Bank's evaluation office found that UNDP used WhatsApp and an AI-enhanced chatbot for remote, real-time monitoring in Yemen, which improved transparency and complaint handling.","account":"The Yemen Emergency Crisis Response Project ran from July 2016 to March 2022 with US$887.4 million of World Bank financing, under agreements with UNDP and UNICEF. The World Bank's Independent Evaluation Group reviewed the project's completion report on 29 August 2023 and rated the outcome highly satisfactory. In its assessment of monitoring, the evaluators report that UNDP's project management unit carried out 336 field visits. The unit used common social media platforms, such as WhatsApp and a chatbot enhanced by artificial intelligence. The evaluators write that these tools enhanced remote and real-time monitoring, transparency and communication, including the submission of complaints and responses to queries. They rated the quality of monitoring and evaluation substantial. The Bank's own completion report of 3 March 2023 describes the chatbot in the same words. Neither document names the chatbot, describes how artificial intelligence enhanced it or separates its contribution from that of WhatsApp, the field visits and the other monitoring channels."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"Chatbot enhanced by artificial intelligence, used by UNDP's Yemen project management unit with WhatsApp (not named in the review)"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"The World Bank's Independent Evaluation Group finds that an AI-enhanced chatbot helped UNDP monitor the Yemen Emergency Crisis Response Project","headline":"AI chatbot and WhatsApp help UNDP monitor a Yemen project remotely"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["6cb0e84c85c249c7e7045f74881efff8e8d75dfba937950bd629e97dd39de935"],"role":"SHA-256 of the complete public record and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public record, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"The Yemen Emergency Crisis Response Project ran from July 2016 to March 2022 with US$887.4 million of World Bank financing, under agreements with UNDP and UNICEF. The World Bank's Independent Evaluation Group reviewed the project's completion report on 29 August 2023 and rated the outcome highly satisfactory. In its assessment of monitoring, the evaluators report that UNDP's project management unit carried out 336 field visits. The unit used common social media platforms, such as WhatsApp and a chatbot enhanced by artificial intelligence. The evaluators write that these tools enhanced remote and real-time monitoring, transparency and communication, including the submission of complaints and responses to queries. They rated the quality of monitoring and evaluation substantial. The Bank's own completion report of 3 March 2023 describes the chatbot in the same words. Neither document names the chatbot, describes how artificial intelligence enhanced it or separates its contribution from that of WhatsApp, the field visits and the other monitoring channels.","replication":{"produced_by":["UNDP's project management unit used WhatsApp and a chatbot enhanced by artificial intelligence alongside 336 field visits.","Implementing partners, third-party monitors and field consultants set up real-time channels to report security threats on the ground."],"must_hold":["Project staff and communities must have mobile phones and connectivity for the social media platforms.","Complaints and queries received through the chatbot must reach staff who act on them."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":28,"reviewed":2,"reviewed_complete":2,"reviewed_partial":0,"record_mapped":26,"review_not_assessed":63,"not_assessed":0,"complete":false,"by_state":{"value_present":28,"not_reported":0,"unresolved":63,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"ce87e3e3bf8aa8dc7155721edb6d7fe90e3a4513f8aaa125604caf49ab756773"}