{"id":"AIBD-0758","title":"CMA's Shanghai Typhoon Institute finds that AI models had smaller worst-case errors in typhoon tracks than physics models at long lead times","summary":"In CMA's official verification of 2025 typhoons, AI models had smaller worst-case errors in tracks than physics-based models at long lead times. ECMWF's AIFS gave the best tracks at every lead time.","date_observed":"2026-10-01","date_basis":"source publication","date_precision":"month","benefit_status":"mixed","source_collection":null,"source_entry":null,"account":"Staff of CMA's Shanghai Typhoon Institute and National Meteorological Centre verify typhoon forecasts each year under CMA's regulations on typhoon operations and services. Their verification of the 27 named typhoons of 2025 appeared in Meteorological Monthly in October 2026. It covers seven AI weather models, with forecasts supplied by the National Meteorological Centre. They were FengQing, FengWu, FuXi, Pangu, AIFS, FourCastNet and GraphCast. At short lead times, the 95th-percentile track errors of the AI models were generally larger than those of physics-based models. At long lead times they were smaller. In a same-sample check, ECMWF's AIFS gave the best track forecasts at every lead time. The physics-based models' 24-hour mean track errors were 12% to 55% larger than those of AIFS. FourCastNet's errors were much larger than those of the other AI models. The authors add that correcting the AI models' intensity forecasts with an adaptation model reduces their underestimation of typhoon intensity.","category":"aggregate result","unexpected":false,"purpose":"ECMWF, the Shanghai Artificial Intelligence Laboratory, Fudan University, Huawei, NVIDIA, Google DeepMind and CMA built these AI weather models to forecast global weather from data.","system":"Seven AI weather models verified by CMA, including AIFS, FengWu, FuXi, Pangu, FourCastNet, GraphCast and CMA's FengQing","model_version":null,"developer":"ECMWF (AIFS), Shanghai Artificial Intelligence Laboratory (FengWu), Fudan University (FuXi), Huawei (Pangu), NVIDIA (FourCastNet), Google DeepMind (GraphCast) and CMA (FengQing)","deployer":"CMA National Meteorological Centre, which supplied the AI models' forecasts","location":"Beijing and Shanghai, China (western North Pacific and South China Sea)","domain":"Weather forecasting / tropical cyclones","segment":null,"benefit_type":"Improved Task Accuracy","work_activities":["4.A.1.b.3"],"taxonomy_note":null,"registry_ids":{},"ratings":[{"group":"organizational","depth":2,"reason":"In CMA's same-sample check, the physics-based models' 24-hour mean track errors were 12% to 55% larger than those of AIFS."}],"ai_role":{"status":"assessed","statement":"CMA verified each AI model's own forecasts against the best track and against physics-based models, which isolates the models' skill. CMA runs the models and built one of them, FengQing, so its verification of FengQing is not outside the maker. The other six come from outside makers."},"evidence":[{"kind":"peer-reviewed study","citation":"YANG Mengqi, et al. 2025年西北太平洋和南海台风预报精度评定 (Accuracy Verification of Typhoon Forecasts over the Northwest Pacific and the South China Sea in 2025). Meteorological Monthly (气象) 52(10): 1257-1270, October 2026. DOI 10.7519/j.issn.1000-0526.2026.062602 (Chinese abstract).","title":"2025年西北太平洋和南海台风预报精度评定 (Accuracy Verification of Typhoon Forecasts over the Northwest Pacific and the South China Sea in 2025)","url":"http://qxqk.nmc.cn/qx/ch/reader/view_abstract.aspx?file_no=20261008&flag=1","date":null,"written_by":"deployer","quote":"人工智能气象模型在短预报时效的路径预报误差第95%分位数普遍高于数值预报模式，长预报时效则低于数值预报模式。"},{"kind":"peer-reviewed study","citation":"YANG Mengqi, et al. Accuracy Verification of Typhoon Forecasts over the Northwest Pacific and the South China Sea in 2025. Meteorological Monthly 52(10): 1257-1270, October 2026 (English abstract).","title":"Accuracy Verification of Typhoon Forecasts over the Northwest Pacific and the South China Sea in 2025","url":"http://qxqk.nmc.cn/qx/ch/reader/view_abstract.aspx?file_no=20261008&flag=1","date":null,"written_by":"deployer","quote":"For AI-based weather prediction (AIWP) models, however, the 95th percentile track forecast errors were generally larger than those of numerical weather prediction (NWP) models at short lead times, but smaller than those of NWP models at long lead times."}],"replication":{"produced_by":["The National Meteorological Centre supplied the forecasts of the seven AI models.","The authors verified all forecast methods against best-track data for the named typhoons of 2025."],"must_hold":["At short lead times, the AI models' worst errors exceed those of physics-based models.","AI intensity forecasts need correction for their underestimation of typhoon intensity."],"echoes":[]},"paired_incident":null,"headline":"*AIFS* gave the best typhoon tracks in China's 2025 verification","metric":"12% to 55% larger than those of AIFS","comparator":{"against":"Physics-based numerical weather prediction models verified for the same 2025 typhoons","before":null,"after":null,"basis":"measured","quote":"For AI-based weather prediction (AIWP) models, however, the 95th percentile track forecast errors were generally larger than those of numerical weather prediction (NWP) models at short lead times, but smaller than those of NWP models at long lead times.","url":"http://qxqk.nmc.cn/qx/ch/reader/view_abstract.aspx?file_no=20261008&flag=1"},"later":null,"value":null,"developer_country":"Multiple: United Kingdom, China, United States","deployer_country":"China","unrated_because":null,"brief_notes":null,"brief":null,"revisions":[{"date":"2026-10-31","change":"Added the record from the report of the China Meteorological Administration.","source":"http://qxqk.nmc.cn/qx/ch/reader/view_abstract.aspx?file_no=20261008&flag=1","editor":"AIBD"}],"confidence":"claimed","shade":"low","claim_status":"The deployer's account","claim_basis":"Sources: the deployer's peer-reviewed study. The cited evidence includes no independent source and no finding by an external body.","case_jurisdictions":["China"],"profile":{"case_id":"AIBD-0758","catalog_version":"1.0.0","source_fingerprint":"0ba0c413eab13a682f97ebac7649c1d0dab5ae4144760b4d37e72a555c60f877","collection_refs":[{"collection_id":"aibd","version":"2026-10-31"}],"source_ids":["AIBD-0758-source-0001","AIBD-0758-source-0002"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"monetary observations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"ECMWF (AIFS), Shanghai Artificial Intelligence Laboratory (FengWu), Fudan University (FuXi), Huawei (Pangu), NVIDIA (FourCastNet), Google DeepMind (GraphCast) and CMA (FengQing)","deployer":"CMA National Meteorological Centre, which supplied the AI models' forecasts"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer","deployer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"value_present","value":{"kind":"record_fields","fields":{"ai_role":{"status":"assessed","statement":"CMA verified each AI model's own forecasts against the best track and against physics-based models, which isolates the models' skill. CMA runs the models and built one of them, FengQing, so its verification of FengQing is not outside the maker. The other six come from outside makers."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"aggregate result","ai_role":{"status":"assessed","statement":"CMA verified each AI model's own forecasts against the best track and against physics-based models, which isolates the models' skill. CMA runs the models and built one of them, FengQing, so its verification of FengQing is not outside the maker. The other six come from outside makers."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"In CMA's same-sample check, the physics-based models' 24-hour mean track errors were 12% to 55% larger than those of AIFS."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"low","evidence_word":"claimed","claim_status":"The deployer's account","basis":"Sources: the deployer's peer-reviewed study. The cited evidence includes no independent source and no finding by an external body.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"In CMA's same-sample check, the physics-based models' 24-hour mean track errors were 12% to 55% larger than those of AIFS."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"In CMA's same-sample check, the physics-based models' 24-hour mean track errors were 12% to 55% larger than those of AIFS."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_links[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"The deployer's account. Sources: the deployer's peer-reviewed study. The cited evidence includes no independent source and no finding by an external body.","evidence_word":"claimed","claim_status":"The deployer's account"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0758-source-0001","AIBD-0758-source-0002"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The deployer's account. Sources: the deployer's peer-reviewed study. The cited evidence includes no independent source and no finding by an external body.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"routine use","answer":"The system was in routine use. The authors state that several AI weather models have run operationally at CMA's National Meteorological Centre since 2023.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0758-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was in routine use. The authors state that several AI weather models have run operationally at CMA's National Meteorological Centre since 2023.","inferred":false}},"economics[].amount":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].costs[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].counterfactual":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].currency_price_basis":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].gross_net":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].measure_type":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].period_unit":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].realization":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].recipient_payer":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].return_payback":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].valuation_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"evaluations[].ai_uplift":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].comparator":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].design":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].effect":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].exposure":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].id":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].measurement_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].metric":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].reproduction_materials":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].setting_sample":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].uncertainty_limitations":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"external_references[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"follow_up[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-0758"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Improved Task Accuracy"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"Staff of CMA's Shanghai Typhoon Institute and National Meteorological Centre verify typhoon forecasts each year under CMA's regulations on typhoon operations and services. Their verification of the 27 named typhoons of 2025 appeared in Meteorological Monthly in October 2026. It covers seven AI weather models, with forecasts supplied by the National Meteorological Centre. They were FengQing, FengWu, FuXi, Pangu, AIFS, FourCastNet and GraphCast. At short lead times, the 95th-percentile track errors of the AI models were generally larger than those of physics-based models. At long lead times they were smaller. In a same-sample check, ECMWF's AIFS gave the best track forecasts at every lead time. The physics-based models' 24-hour mean track errors were 12% to 55% larger than those of AIFS. FourCastNet's errors were much larger than those of the other AI models. The authors add that correcting the AI models' intensity forecasts with an adaptation model reduces their underestimation of typhoon intensity.","benefit_type":"Improved Task Accuracy"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"In CMA's same-sample check, the physics-based models' 24-hour mean track errors were 12% to 55% larger than those of AIFS."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"mixed"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"ECMWF, the Shanghai Artificial Intelligence Laboratory, Fudan University, Huawei, NVIDIA, Google DeepMind and CMA built these AI weather models to forecast global weather from data."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"Beijing and Shanghai, China (western North Pacific and South China Sea)","developer_country":"Multiple: United Kingdom, China, United States","deployer_country":"China"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","developer_country","deployer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"ECMWF, the Shanghai Artificial Intelligence Laboratory, Fudan University, Huawei, NVIDIA, Google DeepMind and CMA built these AI weather models to forecast global weather from data."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2026-10-01","date_basis":"source publication","date_precision":"month","revisions":[{"date":"2026-10-31","change":"Added the record from the report of the China Meteorological Administration.","source":"http://qxqk.nmc.cn/qx/ch/reader/view_abstract.aspx?file_no=20261008&flag=1","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["The National Meteorological Centre supplied the forecasts of the seven AI models.","The authors verified all forecast methods against best-track data for the named typhoons of 2025."],"must_hold":["At short lead times, the AI models' worst errors exceed those of physics-based models.","AI intensity forecasts need correction for their underestimation of typhoon intensity."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-10-31","change":"Added the record from the report of the China Meteorological Administration.","source":"http://qxqk.nmc.cn/qx/ch/reader/view_abstract.aspx?file_no=20261008&flag=1","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Weather forecasting / tropical cyclones"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"peer-reviewed study","citation":"YANG Mengqi, et al. 2025年西北太平洋和南海台风预报精度评定 (Accuracy Verification of Typhoon Forecasts over the Northwest Pacific and the South China Sea in 2025). Meteorological Monthly (气象) 52(10): 1257-1270, October 2026. DOI 10.7519/j.issn.1000-0526.2026.062602 (Chinese abstract).","title":"2025年西北太平洋和南海台风预报精度评定 (Accuracy Verification of Typhoon Forecasts over the Northwest Pacific and the South China Sea in 2025)","url":"http://qxqk.nmc.cn/qx/ch/reader/view_abstract.aspx?file_no=20261008&flag=1","written_by":"deployer","quote":"人工智能气象模型在短预报时效的路径预报误差第95%分位数普遍高于数值预报模式，长预报时效则低于数值预报模式。"},{"kind":"peer-reviewed study","citation":"YANG Mengqi, et al. Accuracy Verification of Typhoon Forecasts over the Northwest Pacific and the South China Sea in 2025. Meteorological Monthly 52(10): 1257-1270, October 2026 (English abstract).","title":"Accuracy Verification of Typhoon Forecasts over the Northwest Pacific and the South China Sea in 2025","url":"http://qxqk.nmc.cn/qx/ch/reader/view_abstract.aspx?file_no=20261008&flag=1","written_by":"deployer","quote":"For AI-based weather prediction (AIWP) models, however, the 95th percentile track forecast errors were generally larger than those of numerical weather prediction (NWP) models at short lead times, but smaller than those of NWP models at long lead times."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0758-source-0001","AIBD-0758-source-0002"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"peer-reviewed study","citation":"YANG Mengqi, et al. 2025年西北太平洋和南海台风预报精度评定 (Accuracy Verification of Typhoon Forecasts over the Northwest Pacific and the South China Sea in 2025). Meteorological Monthly (气象) 52(10): 1257-1270, October 2026. DOI 10.7519/j.issn.1000-0526.2026.062602 (Chinese abstract).","title":"2025年西北太平洋和南海台风预报精度评定 (Accuracy Verification of Typhoon Forecasts over the Northwest Pacific and the South China Sea in 2025)","url":"http://qxqk.nmc.cn/qx/ch/reader/view_abstract.aspx?file_no=20261008&flag=1","written_by":"deployer","quote":"人工智能气象模型在短预报时效的路径预报误差第95%分位数普遍高于数值预报模式，长预报时效则低于数值预报模式。"},{"kind":"peer-reviewed study","citation":"YANG Mengqi, et al. Accuracy Verification of Typhoon Forecasts over the Northwest Pacific and the South China Sea in 2025. Meteorological Monthly 52(10): 1257-1270, October 2026 (English abstract).","title":"Accuracy Verification of Typhoon Forecasts over the Northwest Pacific and the South China Sea in 2025","url":"http://qxqk.nmc.cn/qx/ch/reader/view_abstract.aspx?file_no=20261008&flag=1","written_by":"deployer","quote":"For AI-based weather prediction (AIWP) models, however, the 95th percentile track forecast errors were generally larger than those of numerical weather prediction (NWP) models at short lead times, but smaller than those of NWP models at long lead times."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"In CMA's official verification of 2025 typhoons, AI models had smaller worst-case errors in tracks than physics-based models at long lead times. ECMWF's AIFS gave the best tracks at every lead time.","account":"Staff of CMA's Shanghai Typhoon Institute and National Meteorological Centre verify typhoon forecasts each year under CMA's regulations on typhoon operations and services. Their verification of the 27 named typhoons of 2025 appeared in Meteorological Monthly in October 2026. It covers seven AI weather models, with forecasts supplied by the National Meteorological Centre. They were FengQing, FengWu, FuXi, Pangu, AIFS, FourCastNet and GraphCast. At short lead times, the 95th-percentile track errors of the AI models were generally larger than those of physics-based models. At long lead times they were smaller. In a same-sample check, ECMWF's AIFS gave the best track forecasts at every lead time. The physics-based models' 24-hour mean track errors were 12% to 55% larger than those of AIFS. FourCastNet's errors were much larger than those of the other AI models. The authors add that correcting the AI models' intensity forecasts with an adaptation model reduces their underestimation of typhoon intensity."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"Seven AI weather models verified by CMA, including AIFS, FengWu, FuXi, Pangu, FourCastNet, GraphCast and CMA's FengQing"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"CMA's Shanghai Typhoon Institute finds that AI models had smaller worst-case errors in typhoon tracks than physics models at long lead times","headline":"*AIFS* gave the best typhoon tracks in China's 2025 verification"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["033c81dac86b23a968737dea0aba981cd60fbfc43b17ad6405d6377a165d3c27"],"role":"SHA-256 of the complete public record and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public record, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"Staff of CMA's Shanghai Typhoon Institute and National Meteorological Centre verify typhoon forecasts each year under CMA's regulations on typhoon operations and services. Their verification of the 27 named typhoons of 2025 appeared in Meteorological Monthly in October 2026. It covers seven AI weather models, with forecasts supplied by the National Meteorological Centre. They were FengQing, FengWu, FuXi, Pangu, AIFS, FourCastNet and GraphCast. At short lead times, the 95th-percentile track errors of the AI models were generally larger than those of physics-based models. At long lead times they were smaller. In a same-sample check, ECMWF's AIFS gave the best track forecasts at every lead time. The physics-based models' 24-hour mean track errors were 12% to 55% larger than those of AIFS. FourCastNet's errors were much larger than those of the other AI models. The authors add that correcting the AI models' intensity forecasts with an adaptation model reduces their underestimation of typhoon intensity.","replication":{"produced_by":["The National Meteorological Centre supplied the forecasts of the seven AI models.","The authors verified all forecast methods against best-track data for the named typhoons of 2025."],"must_hold":["At short lead times, the AI models' worst errors exceed those of physics-based models.","AI intensity forecasts need correction for their underestimation of typhoon intensity."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":29,"reviewed":2,"reviewed_complete":2,"reviewed_partial":0,"record_mapped":27,"review_not_assessed":62,"not_assessed":0,"complete":false,"by_state":{"value_present":29,"not_reported":0,"unresolved":62,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"cdc0ba0df1b8b30e22479a68feb6fa034472438bb59b6319ced0e6b439ae164b"}