{"id":"AIBD-1127","title":"GAO reports that NOAA's ProbSevere machine learning model has improved forecasters' ability to warn of severe thunderstorms and tornadoes","summary":"GAO reports that forecasters who evaluated NOAA's ProbSevere said that it increased their confidence in issuing warnings 95 percent of the time and lead time 76 percent of the time.","date_observed":"2023-12-14","date_basis":"source publication","date_precision":"day","benefit_status":"reported","source_collection":null,"source_entry":null,"account":"NOAA and the Cooperative Institute for Meteorological Satellite Studies in Wisconsin developed ProbSevere. Its second version uses naive Bayesian classifiers to estimate the probability that a storm will produce severe hail, wind or a tornado. In a technology assessment dated 14 December 2023, the U.S. Government Accountability Office (GAO) states that the model has improved forecasters' ability to warn of severe thunderstorms and tornadoes. GAO writes that forecaster feedback suggests that warnings come about 5 to 10 minutes earlier. It reports that forecasters who evaluated the model said that it increased their confidence in issuing warnings 95 percent of the time and lead time 76 percent of the time. GAO did not test the model or analyze survey data. It relays the developers' 2020 paper and gives no source for the two percentages. In that paper, 68 forecasters in NOAA's Hazardous Weather Testbed answered daily surveys from 2017 to 2019. In about 80% to 90% of their answers, forecasters said that ProbSevere increased confidence. GAO and the developers write that more research is needed to quantify the effect on lead time.","category":"aggregate result","unexpected":false,"purpose":"NOAA and the Cooperative Institute for Meteorological Satellite Studies built ProbSevere to estimate the probability that a developing storm will become severe in about the next hour.","system":"NOAA ProbSevere v2.0 (ProbHail, ProbWind and ProbTor)","model_version":"version 2","developer":"NOAA (NESDIS Center for Satellite Applications and Research) with the Cooperative Institute for Meteorological Satellite Studies (CIMSS), University of Wisconsin-Madison","deployer":"NOAA National Weather Service","location":"United States (national)","domain":"Weather forecasting / severe storm warnings","segment":null,"benefit_type":"Improved Task Accuracy","work_activities":["4.A.1.b.3","4.A.1.a.2"],"taxonomy_note":null,"registry_ids":{},"ratings":[{"group":"organizational","depth":2,"reason":"Forecasters who evaluated the model said that it increased their confidence in warning decisions and their lead time to severe hazards."},{"group":"societal","depth":null,"reason":"No source measures how much earlier the public was warned, and GAO says that more research is needed to quantify the effect on lead time."}],"ai_role":{"status":"unassessed","statement":"No source compares warnings issued with and without ProbSevere. The evidence is forecasters' own survey answers, and the developers write that ProbSevere is one information source among several in warning decisions. They add that National Weather Service forecasters decide which storms warrant warnings more accurately than the model does. The model uses naive Bayesian classifiers trained to specific hazards."},"evidence":[{"kind":"descriptive report","citation":"U.S. Government Accountability Office. Artificial Intelligence in Natural Hazard Modeling: Severe Storms, Hurricanes, Floods, and Wildfires. Technology assessment GAO-24-106213, 14 December 2023, p. 24.","title":"Artificial Intelligence in Natural Hazard Modeling: Severe Storms, Hurricanes, Floods, and Wildfires","url":"https://web.archive.org/web/2026id_/https://www.gao.gov/assets/d24106213.pdf","date":"2023-12-14","written_by":"independent","quote":"One exception is ProbSevere (version 2), which is a machine learning model that has improved forecasters' ability to issue warnings of severe thunderstorms and tornadoes."},{"kind":"peer-reviewed study","citation":"Cintineo, J. L., M. J. Pavolonis, J. M. Sieglaff, L. Cronce and J. Brunner. NOAA ProbSevere v2.0—ProbHail, ProbWind, and ProbTor. Weather and Forecasting 35(4): 1523–1543, 2020.","title":"NOAA ProbSevere v2.0—ProbHail, ProbWind, and ProbTor","url":"https://doi.org/10.1175/WAF-D-19-0242.1","date":"2020-08-01","written_by":"makers","quote":"From the forecaster feedback, ~80%–90% of forecaster responses said that ProbSevere increases confidence in their warning decision making, while ~60%–70% indicated that ProbSevere increases lead time to severe hazards."}],"replication":{"produced_by":["ProbSevere combined radar, satellite, lightning and weather model data for each storm that it tracked.","Naive Bayesian classifiers trained to each hazard gave the probability of severe hail, wind and tornadoes.","National Weather Service forecasters used ProbSevere as an additional tool when predicting severe weather."],"must_hold":["Forecasters must use the model alongside radar and other information, since it is one source among several.","Probability thresholds must suit the region, since the best threshold varies across the United States."],"echoes":[]},"paired_incident":null,"headline":"*ProbSevere* helps US forecasters warn of severe storms, GAO reports","metric":"increased their confidence in issuing warnings 95 percent of the time","comparator":{"against":"Warning decisions made without ProbSevere, as forecasters judged them","before":null,"after":"Warnings about 5 to 10 minutes earlier","basis":"assumed","quote":"Forecaster feedback suggests that ProbSevere allows them to warn of severe weather about 5 to 10 minutes earlier than they otherwise would have, providing more time to prepare for these hazards","url":"https://web.archive.org/web/2026id_/https://www.gao.gov/assets/d24106213.pdf"},"later":null,"value":null,"developer_country":"United States","deployer_country":"United States","unrated_because":null,"brief_notes":null,"brief":null,"revisions":[{"date":"2026-11-01","change":"Added the record from the report of the U.S. Government Accountability Office.","source":"https://web.archive.org/web/2026id_/https://www.gao.gov/assets/d24106213.pdf","editor":"AIBD"}],"confidence":"reported","shade":"medium","claim_status":"Independently reported","claim_basis":"Sources: one independent descriptive report and the makers' peer-reviewed study. The cited evidence includes no independent study of the benefit and no finding by an external body.","case_jurisdictions":["United States"],"profile":{"case_id":"AIBD-1127","catalog_version":"1.0.0","source_fingerprint":"022042b6f70927aabaa02622020d3e7a55f5bfc8dce1e9ae9e81b454a6492077","collection_refs":[{"collection_id":"aibd","version":"2026-10-31"}],"source_ids":["AIBD-1127-source-0001","AIBD-1127-source-0002"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"monetary observations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"NOAA (NESDIS Center for Satellite Applications and Research) with the Cooperative Institute for Meteorological Satellite Studies (CIMSS), University of Wisconsin-Madison","deployer":"NOAA National Weather Service"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer","deployer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"aggregate result","ai_role":{"status":"unassessed","statement":"No source compares warnings issued with and without ProbSevere. The evidence is forecasters' own survey answers, and the developers write that ProbSevere is one information source among several in warning decisions. They add that National Weather Service forecasters decide which storms warrant warnings more accurately than the model does. The model uses naive Bayesian classifiers trained to specific hazards."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"Forecasters who evaluated the model said that it increased their confidence in warning decisions and their lead time to severe hazards."},{"group":"societal","reason":"No source measures how much earlier the public was warned, and GAO says that more research is needed to quantify the effect on lead time."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"medium","evidence_word":"reported","claim_status":"Independently reported","basis":"Sources: one independent descriptive report and the makers' peer-reviewed study. The cited evidence includes no independent study of the benefit and no finding by an external body.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"Forecasters who evaluated the model said that it increased their confidence in warning decisions and their lead time to severe hazards."},{"group":"societal","reason":"No source measures how much earlier the public was warned, and GAO says that more research is needed to quantify the effect on lead time."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"Forecasters who evaluated the model said that it increased their confidence in warning decisions and their lead time to severe hazards."},{"group":"societal","reason":"No source measures how much earlier the public was warned, and GAO says that more research is needed to quantify the effect on lead time."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_links[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"Independently reported. Sources: one independent descriptive report and the makers' peer-reviewed study. The cited evidence includes no independent study of the benefit and no finding by an external body.","evidence_word":"reported","claim_status":"Independently reported"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-1127-source-0001","AIBD-1127-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Independently reported. Sources: one independent descriptive report and the makers' peer-reviewed study. The cited evidence includes no independent study of the benefit and no finding by an external body.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"routine use","answer":"The system was in routine use. GAO states that forecasters at the National Weather Service currently use ProbSevere as an additional tool when predicting severe weather.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-1127-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was in routine use. GAO states that forecasters at the National Weather Service currently use ProbSevere as an additional tool when predicting severe weather.","inferred":false}},"economics[].amount":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].costs[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].counterfactual":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].currency_price_basis":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].gross_net":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].measure_type":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].period_unit":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].realization":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].recipient_payer":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].return_payback":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].valuation_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"evaluations[].ai_uplift":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].comparator":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].design":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].effect":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].exposure":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].id":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].measurement_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].metric":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].reproduction_materials":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].setting_sample":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].uncertainty_limitations":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"external_references[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"follow_up[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-1127"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Improved Task Accuracy"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"NOAA and the Cooperative Institute for Meteorological Satellite Studies in Wisconsin developed ProbSevere. Its second version uses naive Bayesian classifiers to estimate the probability that a storm will produce severe hail, wind or a tornado. In a technology assessment dated 14 December 2023, the U.S. Government Accountability Office (GAO) states that the model has improved forecasters' ability to warn of severe thunderstorms and tornadoes. GAO writes that forecaster feedback suggests that warnings come about 5 to 10 minutes earlier. It reports that forecasters who evaluated the model said that it increased their confidence in issuing warnings 95 percent of the time and lead time 76 percent of the time. GAO did not test the model or analyze survey data. It relays the developers' 2020 paper and gives no source for the two percentages. In that paper, 68 forecasters in NOAA's Hazardous Weather Testbed answered daily surveys from 2017 to 2019. In about 80% to 90% of their answers, forecasters said that ProbSevere increased confidence. GAO and the developers write that more research is needed to quantify the effect on lead time.","benefit_type":"Improved Task Accuracy"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"Forecasters who evaluated the model said that it increased their confidence in warning decisions and their lead time to severe hazards."},{"group":"societal","reason":"No source measures how much earlier the public was warned, and GAO says that more research is needed to quantify the effect on lead time."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"reported"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"NOAA and the Cooperative Institute for Meteorological Satellite Studies built ProbSevere to estimate the probability that a developing storm will become severe in about the next hour."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"United States (national)","developer_country":"United States","deployer_country":"United States"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","developer_country","deployer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"NOAA and the Cooperative Institute for Meteorological Satellite Studies built ProbSevere to estimate the probability that a developing storm will become severe in about the next hour."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2023-12-14","date_basis":"source publication","date_precision":"day","revisions":[{"date":"2026-11-01","change":"Added the record from the report of the U.S. Government Accountability Office.","source":"https://web.archive.org/web/2026id_/https://www.gao.gov/assets/d24106213.pdf","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["ProbSevere combined radar, satellite, lightning and weather model data for each storm that it tracked.","Naive Bayesian classifiers trained to each hazard gave the probability of severe hail, wind and tornadoes.","National Weather Service forecasters used ProbSevere as an additional tool when predicting severe weather."],"must_hold":["Forecasters must use the model alongside radar and other information, since it is one source among several.","Probability thresholds must suit the region, since the best threshold varies across the United States."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-11-01","change":"Added the record from the report of the U.S. Government Accountability Office.","source":"https://web.archive.org/web/2026id_/https://www.gao.gov/assets/d24106213.pdf","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Weather forecasting / severe storm warnings"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"descriptive report","citation":"U.S. Government Accountability Office. Artificial Intelligence in Natural Hazard Modeling: Severe Storms, Hurricanes, Floods, and Wildfires. Technology assessment GAO-24-106213, 14 December 2023, p. 24.","title":"Artificial Intelligence in Natural Hazard Modeling: Severe Storms, Hurricanes, Floods, and Wildfires","url":"https://web.archive.org/web/2026id_/https://www.gao.gov/assets/d24106213.pdf","date":"2023-12-14","written_by":"independent","quote":"One exception is ProbSevere (version 2), which is a machine learning model that has improved forecasters' ability to issue warnings of severe thunderstorms and tornadoes."},{"kind":"peer-reviewed study","citation":"Cintineo, J. L., M. J. Pavolonis, J. M. Sieglaff, L. Cronce and J. Brunner. NOAA ProbSevere v2.0—ProbHail, ProbWind, and ProbTor. Weather and Forecasting 35(4): 1523–1543, 2020.","title":"NOAA ProbSevere v2.0—ProbHail, ProbWind, and ProbTor","url":"https://doi.org/10.1175/WAF-D-19-0242.1","date":"2020-08-01","written_by":"makers","quote":"From the forecaster feedback, ~80%–90% of forecaster responses said that ProbSevere increases confidence in their warning decision making, while ~60%–70% indicated that ProbSevere increases lead time to severe hazards."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-1127-source-0001","AIBD-1127-source-0002"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"descriptive report","citation":"U.S. Government Accountability Office. Artificial Intelligence in Natural Hazard Modeling: Severe Storms, Hurricanes, Floods, and Wildfires. Technology assessment GAO-24-106213, 14 December 2023, p. 24.","title":"Artificial Intelligence in Natural Hazard Modeling: Severe Storms, Hurricanes, Floods, and Wildfires","url":"https://web.archive.org/web/2026id_/https://www.gao.gov/assets/d24106213.pdf","date":"2023-12-14","written_by":"independent","quote":"One exception is ProbSevere (version 2), which is a machine learning model that has improved forecasters' ability to issue warnings of severe thunderstorms and tornadoes."},{"kind":"peer-reviewed study","citation":"Cintineo, J. L., M. J. Pavolonis, J. M. Sieglaff, L. Cronce and J. Brunner. NOAA ProbSevere v2.0—ProbHail, ProbWind, and ProbTor. Weather and Forecasting 35(4): 1523–1543, 2020.","title":"NOAA ProbSevere v2.0—ProbHail, ProbWind, and ProbTor","url":"https://doi.org/10.1175/WAF-D-19-0242.1","date":"2020-08-01","written_by":"makers","quote":"From the forecaster feedback, ~80%–90% of forecaster responses said that ProbSevere increases confidence in their warning decision making, while ~60%–70% indicated that ProbSevere increases lead time to severe hazards."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"GAO reports that forecasters who evaluated NOAA's ProbSevere said that it increased their confidence in issuing warnings 95 percent of the time and lead time 76 percent of the time.","account":"NOAA and the Cooperative Institute for Meteorological Satellite Studies in Wisconsin developed ProbSevere. Its second version uses naive Bayesian classifiers to estimate the probability that a storm will produce severe hail, wind or a tornado. In a technology assessment dated 14 December 2023, the U.S. Government Accountability Office (GAO) states that the model has improved forecasters' ability to warn of severe thunderstorms and tornadoes. GAO writes that forecaster feedback suggests that warnings come about 5 to 10 minutes earlier. It reports that forecasters who evaluated the model said that it increased their confidence in issuing warnings 95 percent of the time and lead time 76 percent of the time. GAO did not test the model or analyze survey data. It relays the developers' 2020 paper and gives no source for the two percentages. In that paper, 68 forecasters in NOAA's Hazardous Weather Testbed answered daily surveys from 2017 to 2019. In about 80% to 90% of their answers, forecasters said that ProbSevere increased confidence. GAO and the developers write that more research is needed to quantify the effect on lead time."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"value_present","value":{"kind":"record_fields","fields":{"model_version":"version 2"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["model_version"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"NOAA ProbSevere v2.0 (ProbHail, ProbWind and ProbTor)"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"GAO reports that NOAA's ProbSevere machine learning model has improved forecasters' ability to warn of severe thunderstorms and tornadoes","headline":"*ProbSevere* helps US forecasters warn of severe storms, GAO reports"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["29560e511489f2426f92cd0b9c16cd88a7c58378e0b283c2346a351350a38ce1"],"role":"SHA-256 of the complete public record and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public record, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"NOAA and the Cooperative Institute for Meteorological Satellite Studies in Wisconsin developed ProbSevere. Its second version uses naive Bayesian classifiers to estimate the probability that a storm will produce severe hail, wind or a tornado. In a technology assessment dated 14 December 2023, the U.S. Government Accountability Office (GAO) states that the model has improved forecasters' ability to warn of severe thunderstorms and tornadoes. GAO writes that forecaster feedback suggests that warnings come about 5 to 10 minutes earlier. It reports that forecasters who evaluated the model said that it increased their confidence in issuing warnings 95 percent of the time and lead time 76 percent of the time. GAO did not test the model or analyze survey data. It relays the developers' 2020 paper and gives no source for the two percentages. In that paper, 68 forecasters in NOAA's Hazardous Weather Testbed answered daily surveys from 2017 to 2019. In about 80% to 90% of their answers, forecasters said that ProbSevere increased confidence. GAO and the developers write that more research is needed to quantify the effect on lead time.","replication":{"produced_by":["ProbSevere combined radar, satellite, lightning and weather model data for each storm that it tracked.","Naive Bayesian classifiers trained to each hazard gave the probability of severe hail, wind and tornadoes.","National Weather Service forecasters used ProbSevere as an additional tool when predicting severe weather."],"must_hold":["Forecasters must use the model alongside radar and other information, since it is one source among several.","Probability thresholds must suit the region, since the best threshold varies across the United States."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":29,"reviewed":2,"reviewed_complete":2,"reviewed_partial":0,"record_mapped":27,"review_not_assessed":62,"not_assessed":0,"complete":false,"by_state":{"value_present":29,"not_reported":0,"unresolved":62,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"1c8f8b7c53f20cfbf4fda3cccf669930d11461e270aeea2a85456608ae3829bb"}