{"id":"AIBD-0761","title":"University of Reading researchers find that four AI models captured Storm Ciarán's structure and track but underestimated its peak winds","summary":"Reading researchers found that FourCastNet, Pangu-Weather, GraphCast and FourCastNet-v2 forecast Storm Ciarán's structure and track as physics models did. All four underestimated its peak winds.","date_observed":"2024-04-22","date_basis":"source publication","date_precision":"day","benefit_status":"mixed","source_collection":null,"source_entry":null,"account":"Researchers at the University of Reading compared forecasts of Storm Ciarán, which caused sixteen deaths in Northern Europe in November 2023. They ran FourCastNet, Pangu-Weather, GraphCast and FourCastNet-v2 through ECMWF's ai-models toolbox. All four started from the same operational ECMWF analysis as the IFS high-resolution forecast. The storm was an out-of-sample test, since the models' training data end before 2023. The authors report that the four models captured the cloud head, the warm sector and the jet of the warm conveyor belt. More than 48 hours ahead, their forecasts of the storm's rapid deepening and track matched an ensemble of physics-based models. All four underestimated the peak winds, at 25-26 m/s against 30 m/s in ERA5 and 34 m/s in the IFS analysis. None captured the sharp bent-back warm front. The journal npj Climate and Atmospheric Science published the study on 22 April 2024. The authors caution that one case cannot rank the models.","category":"aggregate result","unexpected":false,"purpose":"NVIDIA built FourCastNet and FourCastNet-v2, Huawei built Pangu-Weather and Google DeepMind built GraphCast to forecast global weather with deep learning trained on ERA5 reanalysis.","system":"FourCastNet, Pangu-Weather, GraphCast and FourCastNet-v2","model_version":null,"developer":"NVIDIA (FourCastNet, FourCastNet-v2), Huawei (Pangu-Weather) and Google DeepMind (GraphCast)","deployer":null,"location":"Northern Europe and the English Channel (Storm Ciarán); University of Reading, United Kingdom","domain":"Weather forecasting / windstorms","segment":null,"benefit_type":"Improved Task Accuracy","work_activities":["4.A.1.b.3"],"taxonomy_note":"The AI forecasts matched physics-based forecasts on the storm's track and deepening and fell short on its peak winds.","registry_ids":{},"ratings":[{"group":"organizational","depth":null,"reason":"The forecasts were run after the storm for comparison, and no forecaster used them in warnings."}],"ai_role":{"status":"assessed","statement":"All four models started from the same ECMWF analysis as the IFS forecast, so the authors compare each model's own output with physics-based forecasts of the same storm."},"evidence":[{"kind":"peer-reviewed study","citation":"Charlton-Perez AJ, Dacre HF, Driscoll S, et al. Do AI models produce better weather forecasts than physics-based models? A quantitative evaluation case study of Storm Ciarán. npj Climate and Atmospheric Science 7, 93 (2024).","title":"Do AI models produce better weather forecasts than physics-based models? A quantitative evaluation case study of Storm Ciarán","url":"https://www.nature.com/articles/s41612-024-00638-w","date":"2024-04-22","written_by":"independent","quote":"The four machine learning models considered (FourCastNet, Pangu-Weather, GraphCast and FourCastNet-v2) produce forecasts that accurately capture the synoptic-scale structure of the cyclone including the position of the cloud head, shape of the warm sector and location of the warm conveyor belt jet"}],"replication":{"produced_by":["The models ran through ECMWF's ai-models toolbox from the operational ECMWF analysis.","Forecasts were compared at lead times when forecasters were issuing warnings."],"must_hold":["The storm must develop like storms in the training record, with similar drivers such as the upper-level jet.","Warnings of damaging winds need physics-based guidance, since all four models underestimated the peak winds."],"echoes":[]},"paired_incident":null,"headline":"*GraphCast* and three other AI models capture Storm Ciarán's structure","metric":"at 25-26 m/s against 30 m/s in ERA5","comparator":{"against":"The IFS high-resolution forecast and control forecasts from four physics-based ensembles","before":"Physics-based forecasts of the storm's deepening and track","after":"AI forecasts of deepening and track that matched them more than 48 hours ahead","basis":"measured","quote":"forecasts of the rapid MSLP deepening and track of the storms produced by the ML models were essentially indistinguishable from forecasts from an ensemble of conventional NWP models","url":"https://www.nature.com/articles/s41612-024-00638-w"},"later":null,"value":null,"developer_country":"Multiple: United States, China, United Kingdom","deployer_country":null,"unrated_because":"The models were run after the storm, so no gain to forecasters or the public is reported.","brief_notes":null,"brief":null,"revisions":[{"date":"2026-10-31","change":"Added the record from the report of the University of Reading researchers.","source":"https://www.nature.com/articles/s41612-024-00638-w","editor":"AIBD"}],"confidence":"studied","shade":"high","claim_status":"Independently studied","claim_basis":"Sources: one independent peer-reviewed study. The cited evidence includes no finding by an external body.","case_jurisdictions":["United Kingdom"],"profile":{"case_id":"AIBD-0761","catalog_version":"1.0.0","source_fingerprint":"c2dc1b07494595526365798013d03efb796c7e618da127558938c4db72182e41","collection_refs":[{"collection_id":"aibd","version":"2026-10-31"}],"source_ids":["AIBD-0761-source-0001"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"monetary observations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"NVIDIA (FourCastNet, FourCastNet-v2), Huawei (Pangu-Weather) and Google DeepMind (GraphCast)"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"value_present","value":{"kind":"record_fields","fields":{"ai_role":{"status":"assessed","statement":"All four models started from the same ECMWF analysis as the IFS forecast, so the authors compare each model's own output with physics-based forecasts of the same storm."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"aggregate result","ai_role":{"status":"assessed","statement":"All four models started from the same ECMWF analysis as the IFS forecast, so the authors compare each model's own output with physics-based forecasts of the same storm."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"high","evidence_word":"studied","claim_status":"Independently studied","basis":"Sources: one independent peer-reviewed study. The cited evidence includes no finding by an external body.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","reason":"The forecasts were run after the storm for comparison, and no forecaster used them in warnings."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","reason":"The forecasts were run after the storm for comparison, and no forecaster used them in warnings."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_links[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"Independently studied. Sources: one independent peer-reviewed study. The cited evidence includes no finding by an external body.","evidence_word":"studied","claim_status":"Independently studied"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0761-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Independently studied. Sources: one independent peer-reviewed study. The cited evidence includes no finding by an external body.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"laboratory test","answer":"The system was tested in a laboratory. The researchers ran the models after the storm to compare them with operational forecasts.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0761-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was tested in a laboratory. The researchers ran the models after the storm to compare them with operational forecasts.","inferred":false}},"economics[].amount":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].costs[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].counterfactual":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].currency_price_basis":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].gross_net":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].measure_type":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].period_unit":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].realization":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].recipient_payer":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].return_payback":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].valuation_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"evaluations[].ai_uplift":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].comparator":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].design":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].effect":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].exposure":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].id":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].measurement_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].metric":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].reproduction_materials":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].setting_sample":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].uncertainty_limitations":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"external_references[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"follow_up[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-0761"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Improved Task Accuracy","taxonomy_note":"The AI forecasts matched physics-based forecasts on the storm's track and deepening and fell short on its peak winds."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type","taxonomy_note"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"Researchers at the University of Reading compared forecasts of Storm Ciarán, which caused sixteen deaths in Northern Europe in November 2023. They ran FourCastNet, Pangu-Weather, GraphCast and FourCastNet-v2 through ECMWF's ai-models toolbox. All four started from the same operational ECMWF analysis as the IFS high-resolution forecast. The storm was an out-of-sample test, since the models' training data end before 2023. The authors report that the four models captured the cloud head, the warm sector and the jet of the warm conveyor belt. More than 48 hours ahead, their forecasts of the storm's rapid deepening and track matched an ensemble of physics-based models. All four underestimated the peak winds, at 25-26 m/s against 30 m/s in ERA5 and 34 m/s in the IFS analysis. None captured the sharp bent-back warm front. The journal npj Climate and Atmospheric Science published the study on 22 April 2024. The authors caution that one case cannot rank the models.","benefit_type":"Improved Task Accuracy"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","reason":"The forecasts were run after the storm for comparison, and no forecaster used them in warnings."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"mixed"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"NVIDIA built FourCastNet and FourCastNet-v2, Huawei built Pangu-Weather and Google DeepMind built GraphCast to forecast global weather with deep learning trained on ERA5 reanalysis."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"Northern Europe and the English Channel (Storm Ciarán); University of Reading, United Kingdom","developer_country":"Multiple: United States, China, United Kingdom"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","developer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"NVIDIA built FourCastNet and FourCastNet-v2, Huawei built Pangu-Weather and Google DeepMind built GraphCast to forecast global weather with deep learning trained on ERA5 reanalysis."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2024-04-22","date_basis":"source publication","date_precision":"day","revisions":[{"date":"2026-10-31","change":"Added the record from the report of the University of Reading researchers.","source":"https://www.nature.com/articles/s41612-024-00638-w","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["The models ran through ECMWF's ai-models toolbox from the operational ECMWF analysis.","Forecasts were compared at lead times when forecasters were issuing warnings."],"must_hold":["The storm must develop like storms in the training record, with similar drivers such as the upper-level jet.","Warnings of damaging winds need physics-based guidance, since all four models underestimated the peak winds."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-10-31","change":"Added the record from the report of the University of Reading researchers.","source":"https://www.nature.com/articles/s41612-024-00638-w","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Weather forecasting / windstorms"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"peer-reviewed study","citation":"Charlton-Perez AJ, Dacre HF, Driscoll S, et al. Do AI models produce better weather forecasts than physics-based models? A quantitative evaluation case study of Storm Ciarán. npj Climate and Atmospheric Science 7, 93 (2024).","title":"Do AI models produce better weather forecasts than physics-based models? A quantitative evaluation case study of Storm Ciarán","url":"https://www.nature.com/articles/s41612-024-00638-w","date":"2024-04-22","written_by":"independent","quote":"The four machine learning models considered (FourCastNet, Pangu-Weather, GraphCast and FourCastNet-v2) produce forecasts that accurately capture the synoptic-scale structure of the cyclone including the position of the cloud head, shape of the warm sector and location of the warm conveyor belt jet"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0761-source-0001"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"peer-reviewed study","citation":"Charlton-Perez AJ, Dacre HF, Driscoll S, et al. Do AI models produce better weather forecasts than physics-based models? A quantitative evaluation case study of Storm Ciarán. npj Climate and Atmospheric Science 7, 93 (2024).","title":"Do AI models produce better weather forecasts than physics-based models? A quantitative evaluation case study of Storm Ciarán","url":"https://www.nature.com/articles/s41612-024-00638-w","date":"2024-04-22","written_by":"independent","quote":"The four machine learning models considered (FourCastNet, Pangu-Weather, GraphCast and FourCastNet-v2) produce forecasts that accurately capture the synoptic-scale structure of the cyclone including the position of the cloud head, shape of the warm sector and location of the warm conveyor belt jet"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"Reading researchers found that FourCastNet, Pangu-Weather, GraphCast and FourCastNet-v2 forecast Storm Ciarán's structure and track as physics models did. All four underestimated its peak winds.","account":"Researchers at the University of Reading compared forecasts of Storm Ciarán, which caused sixteen deaths in Northern Europe in November 2023. They ran FourCastNet, Pangu-Weather, GraphCast and FourCastNet-v2 through ECMWF's ai-models toolbox. All four started from the same operational ECMWF analysis as the IFS high-resolution forecast. The storm was an out-of-sample test, since the models' training data end before 2023. The authors report that the four models captured the cloud head, the warm sector and the jet of the warm conveyor belt. More than 48 hours ahead, their forecasts of the storm's rapid deepening and track matched an ensemble of physics-based models. All four underestimated the peak winds, at 25-26 m/s against 30 m/s in ERA5 and 34 m/s in the IFS analysis. None captured the sharp bent-back warm front. The journal npj Climate and Atmospheric Science published the study on 22 April 2024. The authors caution that one case cannot rank the models."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"FourCastNet, Pangu-Weather, GraphCast and FourCastNet-v2"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"University of Reading researchers find that four AI models captured Storm Ciarán's structure and track but underestimated its peak winds","headline":"*GraphCast* and three other AI models capture Storm Ciarán's structure"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["0bff5853a8c1f4ec2663b7a694a013b48e89debaf41eb11d653f023b4141c6e5"],"role":"SHA-256 of the complete public record and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public record, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"Researchers at the University of Reading compared forecasts of Storm Ciarán, which caused sixteen deaths in Northern Europe in November 2023. They ran FourCastNet, Pangu-Weather, GraphCast and FourCastNet-v2 through ECMWF's ai-models toolbox. All four started from the same operational ECMWF analysis as the IFS high-resolution forecast. The storm was an out-of-sample test, since the models' training data end before 2023. The authors report that the four models captured the cloud head, the warm sector and the jet of the warm conveyor belt. More than 48 hours ahead, their forecasts of the storm's rapid deepening and track matched an ensemble of physics-based models. All four underestimated the peak winds, at 25-26 m/s against 30 m/s in ERA5 and 34 m/s in the IFS analysis. None captured the sharp bent-back warm front. The journal npj Climate and Atmospheric Science published the study on 22 April 2024. The authors caution that one case cannot rank the models.","replication":{"produced_by":["The models ran through ECMWF's ai-models toolbox from the operational ECMWF analysis.","Forecasts were compared at lead times when forecasters were issuing warnings."],"must_hold":["The storm must develop like storms in the training record, with similar drivers such as the upper-level jet.","Warnings of damaging winds need physics-based guidance, since all four models underestimated the peak winds."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":28,"reviewed":2,"reviewed_complete":2,"reviewed_partial":0,"record_mapped":26,"review_not_assessed":63,"not_assessed":0,"complete":false,"by_state":{"value_present":28,"not_reported":0,"unresolved":63,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"37e9eb1051ae4bd17962beda2222539d7f3cc0cac30f5b31bf15a8ad781caaac"}