{"id":"AIBD-1167","title":"A UNITAR evaluation relays UNOSAT's report that analysts did not need to correct 86 percent of the images that its Flood AI model processed","summary":"A UNITAR evaluation relays UNOSAT's own figures that Flood AI served seven emergency activations in a year and that analysts did not need to correct 86 percent of its images.","date_observed":"2024-05-01","date_basis":"source publication","date_precision":"day","benefit_status":"reported","source_collection":null,"source_entry":null,"account":"UNOSAT, the United Nations Satellite Centre within UNITAR, runs an emergency service that maps disasters and crises from satellite imagery. With research partners on CERN's OpenLab platform, it developed Flood AI, a model for mapping floods. UNITAR's evaluation unit issued a midterm evaluation of the Norway-funded service on 1 May 2024, carried out by an independent consultant. The evaluator relays figures from the project's first annual report, for 2022 to 2023. By that report, Flood AI was deployed in seven activations, and 86 percent of the images it processed needed no correction by analysts. These are the deployer's own figures, and the evaluator did not check them. The evaluator writes that the project exceeded its expected use of Flood AI in the first year, against a target of four activations. The evaluator also writes that the potential for AI models to speed up satellite image analysis is large. The evaluation gives no time saved and no comparison with manual mapping. It also found an apparent reduction in research on other AI models, as funding fell short of the request.","category":"aggregate result","unexpected":false,"purpose":"UNOSAT built Flood AI to map flooded areas in satellite imagery faster during emergencies.","system":"Flood AI","model_version":null,"developer":"UNOSAT (United Nations Satellite Centre, UNITAR), with research partners on CERN's OpenLab platform","deployer":"UNOSAT Emergency Mapping Service (UNITAR)","location":"Global (seven emergency activations, sites not named in the passage)","domain":"Humanitarian disaster mapping","segment":null,"benefit_type":"Repetitive Task Automation","work_activities":["4.A.1.b.1","4.A.2.a.4"],"taxonomy_note":null,"registry_ids":{},"ratings":[{"group":"organizational","depth":1,"reason":"UNOSAT reports that analysts did not need to correct 86 percent of the images that Flood AI processed."},{"group":"societal","depth":null,"reason":"The evaluation does not report whether the flood maps reached responders sooner or changed any response."}],"ai_role":{"status":"unassessed","statement":"The evaluation relays the share of the model's images that analysts left uncorrected. It does not compare mapping time or quality with and without the model. For Flood AI it cites a related 2020 journal article on a fully convolutional neural network for flood segmentation in radar imagery, a model that learns from data."},"evidence":[{"kind":"descriptive report","citation":"Douthwaite, B. Midterm Evaluation of UNOSAT Mapping Service: Evidence-Based Information Support to Humanitarian Assistance, Peace and Security Using Satellite Imagery and Geospatial Techniques Project. UNITAR Planning, Performance Monitoring and Evaluation Unit, 1 May 2024. Published in the UN Evaluation Group repository.","title":"Midterm Evaluation of UNOSAT Mapping Service - Evidence-Based Information Support to Humanitarian Assistance, Peace and Security Using Satellite Imagery and Geospatial Techniques Project","url":"https://unevaluation.org/sites/default/files/member_publications/UNOSATEmergencyMappingservice-MTEfinalversion_150524_3615_11727447531762.pdf","date":"2024-05-01","written_by":"evaluator","quote":"The project's first annual report states that the Flood AI model was deployed as a response to seven activations and that 86 per cent of the images processed by the model did not require correction or adjustment by the analysts."}],"replication":{"produced_by":["UNOSAT ran Flood AI on satellite images in seven emergency activations in the project's first year.","Analysts validated the model's outputs through a human-in-the-loop protocol before products were released.","UNOSAT developed the model with research partners on CERN's OpenLab platform."],"must_hold":["Analysts must validate the model's outputs before products are released.","Research on AI models needs funding, which fell short of the project's request."],"echoes":[]},"paired_incident":null,"headline":"UNOSAT reports that analysts left 86% of *Flood AI* images uncorrected","metric":"86 percent of the images it processed needed no correction by analysts","comparator":null,"later":null,"value":null,"developer_country":"Switzerland","deployer_country":"Switzerland","unrated_because":null,"brief_notes":null,"brief":null,"revisions":[{"date":"2026-11-01","change":"Added the record from the report of the evaluator for UNITAR.","source":"https://unevaluation.org/sites/default/files/member_publications/UNOSATEmergencyMappingservice-MTEfinalversion_150524_3615_11727447531762.pdf","editor":"AIBD"}],"confidence":"claimed","shade":"low","claim_status":"The evaluator's account","claim_basis":"Sources: the evaluators' descriptive report. The cited evidence includes no independent source and no finding by an external body.","case_jurisdictions":[],"profile":{"case_id":"AIBD-1167","catalog_version":"1.0.0","source_fingerprint":"f5a6a134a098430ef739496401ef1ae3c35db6a1547bd3bd455f7f1853be4789","collection_refs":[{"collection_id":"aibd","version":"2026-10-31"}],"source_ids":["AIBD-1167-source-0001"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"monetary observations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"UNOSAT (United Nations Satellite Centre, UNITAR), with research partners on CERN's OpenLab platform","deployer":"UNOSAT Emergency Mapping Service (UNITAR)"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer","deployer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"aggregate result","ai_role":{"status":"unassessed","statement":"The evaluation relays the share of the model's images that analysts left uncorrected. It does not compare mapping time or quality with and without the model. For Flood AI it cites a related 2020 journal article on a fully convolutional neural network for flood segmentation in radar imagery, a model that learns from data."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":1,"reason":"UNOSAT reports that analysts did not need to correct 86 percent of the images that Flood AI processed."},{"group":"societal","reason":"The evaluation does not report whether the flood maps reached responders sooner or changed any response."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"low","evidence_word":"claimed","claim_status":"The evaluator's account","basis":"Sources: the evaluators' descriptive report. The cited evidence includes no independent source and no finding by an external body.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":1,"reason":"UNOSAT reports that analysts did not need to correct 86 percent of the images that Flood AI processed."},{"group":"societal","reason":"The evaluation does not report whether the flood maps reached responders sooner or changed any response."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":1,"reason":"UNOSAT reports that analysts did not need to correct 86 percent of the images that Flood AI processed."},{"group":"societal","reason":"The evaluation does not report whether the flood maps reached responders sooner or changed any response."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_links[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"The evaluator's account. Sources: the evaluators' descriptive report. The cited evidence includes no independent source and no finding by an external body.","evidence_word":"claimed","claim_status":"The evaluator's account"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-1167-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The evaluator's account. Sources: the evaluators' descriptive report. The cited evidence includes no independent source and no finding by an external body.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"routine use","answer":"The system was in routine use. Flood AI was deployed in seven emergency activations in the project's first year, against a target of four.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-1167-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was in routine use. Flood AI was deployed in seven emergency activations in the project's first year, against a target of four.","inferred":false}},"economics[].amount":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].costs[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].counterfactual":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].currency_price_basis":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].gross_net":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].measure_type":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].period_unit":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].realization":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].recipient_payer":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].return_payback":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].valuation_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"evaluations[].ai_uplift":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].comparator":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].design":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].effect":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].exposure":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].id":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].measurement_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].metric":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].reproduction_materials":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].setting_sample":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].uncertainty_limitations":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"external_references[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"follow_up[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-1167"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Repetitive Task Automation"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"UNOSAT, the United Nations Satellite Centre within UNITAR, runs an emergency service that maps disasters and crises from satellite imagery. With research partners on CERN's OpenLab platform, it developed Flood AI, a model for mapping floods. UNITAR's evaluation unit issued a midterm evaluation of the Norway-funded service on 1 May 2024, carried out by an independent consultant. The evaluator relays figures from the project's first annual report, for 2022 to 2023. By that report, Flood AI was deployed in seven activations, and 86 percent of the images it processed needed no correction by analysts. These are the deployer's own figures, and the evaluator did not check them. The evaluator writes that the project exceeded its expected use of Flood AI in the first year, against a target of four activations. The evaluator also writes that the potential for AI models to speed up satellite image analysis is large. The evaluation gives no time saved and no comparison with manual mapping. It also found an apparent reduction in research on other AI models, as funding fell short of the request.","benefit_type":"Repetitive Task Automation"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":1,"reason":"UNOSAT reports that analysts did not need to correct 86 percent of the images that Flood AI processed."},{"group":"societal","reason":"The evaluation does not report whether the flood maps reached responders sooner or changed any response."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"reported"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"UNOSAT built Flood AI to map flooded areas in satellite imagery faster during emergencies."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"Global (seven emergency activations, sites not named in the passage)","developer_country":"Switzerland","deployer_country":"Switzerland"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","developer_country","deployer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"UNOSAT built Flood AI to map flooded areas in satellite imagery faster during emergencies."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2024-05-01","date_basis":"source publication","date_precision":"day","revisions":[{"date":"2026-11-01","change":"Added the record from the report of the evaluator for UNITAR.","source":"https://unevaluation.org/sites/default/files/member_publications/UNOSATEmergencyMappingservice-MTEfinalversion_150524_3615_11727447531762.pdf","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["UNOSAT ran Flood AI on satellite images in seven emergency activations in the project's first year.","Analysts validated the model's outputs through a human-in-the-loop protocol before products were released.","UNOSAT developed the model with research partners on CERN's OpenLab platform."],"must_hold":["Analysts must validate the model's outputs before products are released.","Research on AI models needs funding, which fell short of the project's request."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-11-01","change":"Added the record from the report of the evaluator for UNITAR.","source":"https://unevaluation.org/sites/default/files/member_publications/UNOSATEmergencyMappingservice-MTEfinalversion_150524_3615_11727447531762.pdf","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Humanitarian disaster mapping"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"descriptive report","citation":"Douthwaite, B. Midterm Evaluation of UNOSAT Mapping Service: Evidence-Based Information Support to Humanitarian Assistance, Peace and Security Using Satellite Imagery and Geospatial Techniques Project. UNITAR Planning, Performance Monitoring and Evaluation Unit, 1 May 2024. Published in the UN Evaluation Group repository.","title":"Midterm Evaluation of UNOSAT Mapping Service - Evidence-Based Information Support to Humanitarian Assistance, Peace and Security Using Satellite Imagery and Geospatial Techniques Project","url":"https://unevaluation.org/sites/default/files/member_publications/UNOSATEmergencyMappingservice-MTEfinalversion_150524_3615_11727447531762.pdf","date":"2024-05-01","written_by":"evaluator","quote":"The project's first annual report states that the Flood AI model was deployed as a response to seven activations and that 86 per cent of the images processed by the model did not require correction or adjustment by the analysts."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-1167-source-0001"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"descriptive report","citation":"Douthwaite, B. Midterm Evaluation of UNOSAT Mapping Service: Evidence-Based Information Support to Humanitarian Assistance, Peace and Security Using Satellite Imagery and Geospatial Techniques Project. UNITAR Planning, Performance Monitoring and Evaluation Unit, 1 May 2024. Published in the UN Evaluation Group repository.","title":"Midterm Evaluation of UNOSAT Mapping Service - Evidence-Based Information Support to Humanitarian Assistance, Peace and Security Using Satellite Imagery and Geospatial Techniques Project","url":"https://unevaluation.org/sites/default/files/member_publications/UNOSATEmergencyMappingservice-MTEfinalversion_150524_3615_11727447531762.pdf","date":"2024-05-01","written_by":"evaluator","quote":"The project's first annual report states that the Flood AI model was deployed as a response to seven activations and that 86 per cent of the images processed by the model did not require correction or adjustment by the analysts."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"A UNITAR evaluation relays UNOSAT's own figures that Flood AI served seven emergency activations in a year and that analysts did not need to correct 86 percent of its images.","account":"UNOSAT, the United Nations Satellite Centre within UNITAR, runs an emergency service that maps disasters and crises from satellite imagery. With research partners on CERN's OpenLab platform, it developed Flood AI, a model for mapping floods. UNITAR's evaluation unit issued a midterm evaluation of the Norway-funded service on 1 May 2024, carried out by an independent consultant. The evaluator relays figures from the project's first annual report, for 2022 to 2023. By that report, Flood AI was deployed in seven activations, and 86 percent of the images it processed needed no correction by analysts. These are the deployer's own figures, and the evaluator did not check them. The evaluator writes that the project exceeded its expected use of Flood AI in the first year, against a target of four activations. The evaluator also writes that the potential for AI models to speed up satellite image analysis is large. The evaluation gives no time saved and no comparison with manual mapping. It also found an apparent reduction in research on other AI models, as funding fell short of the request."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"Flood AI"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"A UNITAR evaluation relays UNOSAT's report that analysts did not need to correct 86 percent of the images that its Flood AI model processed","headline":"UNOSAT reports that analysts left 86% of *Flood AI* images uncorrected"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["a521aed844de1eee8ef0388129957bd42abe1a24493798e51285a4ae7bf62c6c"],"role":"SHA-256 of the complete public record and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public record, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"UNOSAT, the United Nations Satellite Centre within UNITAR, runs an emergency service that maps disasters and crises from satellite imagery. With research partners on CERN's OpenLab platform, it developed Flood AI, a model for mapping floods. UNITAR's evaluation unit issued a midterm evaluation of the Norway-funded service on 1 May 2024, carried out by an independent consultant. The evaluator relays figures from the project's first annual report, for 2022 to 2023. By that report, Flood AI was deployed in seven activations, and 86 percent of the images it processed needed no correction by analysts. These are the deployer's own figures, and the evaluator did not check them. The evaluator writes that the project exceeded its expected use of Flood AI in the first year, against a target of four activations. The evaluator also writes that the potential for AI models to speed up satellite image analysis is large. The evaluation gives no time saved and no comparison with manual mapping. It also found an apparent reduction in research on other AI models, as funding fell short of the request.","replication":{"produced_by":["UNOSAT ran Flood AI on satellite images in seven emergency activations in the project's first year.","Analysts validated the model's outputs through a human-in-the-loop protocol before products were released.","UNOSAT developed the model with research partners on CERN's OpenLab platform."],"must_hold":["Analysts must validate the model's outputs before products are released.","Research on AI models needs funding, which fell short of the project's request."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":28,"reviewed":2,"reviewed_complete":2,"reviewed_partial":0,"record_mapped":26,"review_not_assessed":63,"not_assessed":0,"complete":false,"by_state":{"value_present":28,"not_reported":0,"unresolved":63,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"a1eeb0b9ea979b6aef68c69b840549a690a5af7fb052697dfad85ffbf3734069"}