{"id":"AIBD-0755","title":"China's National Meteorological Centre finds that Pangu-Weather and FengWu beat ECMWF's model on 2023 typhoon tracks beyond 36 hours","summary":"CMA's National Meteorological Centre found that Pangu-Weather and FengWu beat ECMWF's model on typhoon tracks beyond 36 hours in 2023, though not within 24 hours.","date_observed":"2024-02-27","date_basis":"source publication","date_precision":"day","benefit_status":"mixed","source_collection":null,"source_entry":null,"account":"The National Meteorological Centre (NMC) of the China Meteorological Administration (CMA) runs AI weather models in real time. It has run Pangu-Weather since December 2022 and FengWu and FuXi since July 2023. On 27 February 2024, the NMC presented its verification of 2023 typhoon forecasts to the 56th session of the ESCAP/WMO Typhoon Committee. Within 24 hours, Pangu-Weather and FengWu were comparable to or slightly worse than ECMWF's deterministic model. For 36 hours and beyond, both had smaller errors than the ECMWF model. Averaged over all lead times, mean absolute track errors were 140.8 for Pangu-Weather and 141.1 for FengWu, against 169.7 for ECMWF. FuXi had the largest errors, at 176.7. All three AI models underestimated typhoon intensity and were clearly worse than ECMWF on it. The results come from slides that cover one season.","category":"aggregate result","unexpected":false,"purpose":"Huawei, the Shanghai Artificial Intelligence Laboratory and Fudan University built Pangu-Weather, FengWu and FuXi to forecast global weather in the medium range with models trained on past weather data.","system":"Pangu-Weather, FengWu and FuXi, run in real time at CMA's National Meteorological Centre","model_version":null,"developer":"Huawei (Pangu-Weather), Shanghai Artificial Intelligence Laboratory (FengWu) and Fudan University (FuXi)","deployer":"CMA National Meteorological Centre","location":"Beijing, China (western North Pacific and South China Sea)","domain":"Weather forecasting / tropical cyclones","segment":null,"benefit_type":"Improved Task Accuracy","work_activities":["4.A.1.b.3"],"taxonomy_note":null,"registry_ids":{},"ratings":[{"group":"organizational","depth":2,"reason":"Averaged over all lead times, Pangu-Weather's track error was 140.8 against 169.7 for ECMWF's model in CMA's 2023 verification."}],"ai_role":{"status":"assessed","statement":"CMA compared each AI model's own track forecasts with ECMWF's operational model for the same 2023 typhoons, which isolates the models' skill. CMA describes them as data-driven models built on transformer architectures. CMA runs the models itself, and the makers are outside CMA."},"evidence":[{"kind":"report","citation":"China Meteorological Administration, National Meteorological Centre. Preliminary Explorations on Data-driven AI-Models in Tropical Cyclone Prediction. Presentation by QIAN Qifeng to the 56th session of the ESCAP/WMO Typhoon Committee, 27 February 2024, slide 9.","title":"Preliminary Explorations on Data-driven AI-Models in Tropical Cyclone Prediction","url":"https://typhooncommittee.org/56th/docs/item%205/Technical%20PresentationsTC56/10.%20Preliminary%20Explorations%20on%20Data-driven%20AI%20Models%20in%20Tropical%20Cyclone%20Prediction.pdf","date":"2024-02-27","written_by":"deployer","quote":"For forecast periods of 36 hours and beyond, Pangu-Weather and FengWu have errors smaller than the ECMWF-IFS."}],"replication":{"produced_by":["CMA ran the three models in real time and made operational products from them on its forecasting platform.","CMA verified each model's typhoon tracks for the 2023 season."],"must_hold":["The advantage holds at lead times of 36 hours and beyond.","Intensity comes from other models, since the AI models underestimate it."],"echoes":[]},"paired_incident":null,"headline":"*Pangu-Weather* beats ECMWF on 2023 typhoon tracks in China's check","metric":"140.8 for Pangu-Weather and 141.1 for FengWu, against 169.7 for ECMWF","comparator":{"against":"ECMWF's operational deterministic model (ECMWF-IFS) for the same 2023 typhoons","before":"169.7 (ECMWF, mean absolute track error averaged over all lead times)","after":"140.8 (Pangu-Weather) and 141.1 (FengWu)","basis":"measured","quote":"For forecast periods of 36 hours and beyond, Pangu-Weather and FengWu have errors smaller than the ECMWF-IFS.","url":"https://typhooncommittee.org/56th/docs/item%205/Technical%20PresentationsTC56/10.%20Preliminary%20Explorations%20on%20Data-driven%20AI%20Models%20in%20Tropical%20Cyclone%20Prediction.pdf"},"later":null,"value":null,"developer_country":"China","deployer_country":"China","unrated_because":null,"brief_notes":null,"brief":null,"revisions":[{"date":"2026-10-31","change":"Added the record from the report of the China Meteorological Administration.","source":"https://typhooncommittee.org/56th/docs/item%205/Technical%20PresentationsTC56/10.%20Preliminary%20Explorations%20on%20Data-driven%20AI%20Models%20in%20Tropical%20Cyclone%20Prediction.pdf","editor":"AIBD"}],"confidence":"claimed","shade":"low","claim_status":"The deployer's account","claim_basis":"Sources: the deployer's report. The cited evidence includes no independent source and no finding by an external body.","case_jurisdictions":["China"],"profile":{"case_id":"AIBD-0755","catalog_version":"1.0.0","source_fingerprint":"87f5ce08c692df485dc755ea61aac77d2719c12450b2383de91e2ec6a9ce86ae","collection_refs":[{"collection_id":"aibd","version":"2026-10-31"}],"source_ids":["AIBD-0755-source-0001"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"monetary observations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"Huawei (Pangu-Weather), Shanghai Artificial Intelligence Laboratory (FengWu) and Fudan University (FuXi)","deployer":"CMA National Meteorological Centre"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer","deployer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"value_present","value":{"kind":"record_fields","fields":{"ai_role":{"status":"assessed","statement":"CMA compared each AI model's own track forecasts with ECMWF's operational model for the same 2023 typhoons, which isolates the models' skill. CMA describes them as data-driven models built on transformer architectures. CMA runs the models itself, and the makers are outside CMA."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"aggregate result","ai_role":{"status":"assessed","statement":"CMA compared each AI model's own track forecasts with ECMWF's operational model for the same 2023 typhoons, which isolates the models' skill. CMA describes them as data-driven models built on transformer architectures. CMA runs the models itself, and the makers are outside CMA."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"Averaged over all lead times, Pangu-Weather's track error was 140.8 against 169.7 for ECMWF's model in CMA's 2023 verification."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"low","evidence_word":"claimed","claim_status":"The deployer's account","basis":"Sources: the deployer's report. The cited evidence includes no independent source and no finding by an external body.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"Averaged over all lead times, Pangu-Weather's track error was 140.8 against 169.7 for ECMWF's model in CMA's 2023 verification."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"Averaged over all lead times, Pangu-Weather's track error was 140.8 against 169.7 for ECMWF's model in CMA's 2023 verification."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_links[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"The deployer's account. Sources: the deployer's report. The cited evidence includes no independent source and no finding by an external body.","evidence_word":"claimed","claim_status":"The deployer's account"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0755-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The deployer's account. Sources: the deployer's report. The cited evidence includes no independent source and no finding by an external body.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"routine use","answer":"The system was in routine use. CMA's National Meteorological Centre has run Pangu-Weather in real time since December 2022 and FengWu and FuXi since July 2023.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0755-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was in routine use. CMA's National Meteorological Centre has run Pangu-Weather in real time since December 2022 and FengWu and FuXi since July 2023.","inferred":false}},"economics[].amount":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].costs[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].counterfactual":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].currency_price_basis":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].gross_net":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].measure_type":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].period_unit":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].realization":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].recipient_payer":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].return_payback":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].valuation_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"evaluations[].ai_uplift":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].comparator":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].design":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].effect":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].exposure":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].id":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].measurement_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].metric":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].reproduction_materials":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].setting_sample":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].uncertainty_limitations":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"external_references[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"follow_up[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-0755"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Improved Task Accuracy"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"The National Meteorological Centre (NMC) of the China Meteorological Administration (CMA) runs AI weather models in real time. It has run Pangu-Weather since December 2022 and FengWu and FuXi since July 2023. On 27 February 2024, the NMC presented its verification of 2023 typhoon forecasts to the 56th session of the ESCAP/WMO Typhoon Committee. Within 24 hours, Pangu-Weather and FengWu were comparable to or slightly worse than ECMWF's deterministic model. For 36 hours and beyond, both had smaller errors than the ECMWF model. Averaged over all lead times, mean absolute track errors were 140.8 for Pangu-Weather and 141.1 for FengWu, against 169.7 for ECMWF. FuXi had the largest errors, at 176.7. All three AI models underestimated typhoon intensity and were clearly worse than ECMWF on it. The results come from slides that cover one season.","benefit_type":"Improved Task Accuracy"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"Averaged over all lead times, Pangu-Weather's track error was 140.8 against 169.7 for ECMWF's model in CMA's 2023 verification."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"mixed"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"Huawei, the Shanghai Artificial Intelligence Laboratory and Fudan University built Pangu-Weather, FengWu and FuXi to forecast global weather in the medium range with models trained on past weather data."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"Beijing, China (western North Pacific and South China Sea)","developer_country":"China","deployer_country":"China"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","developer_country","deployer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"Huawei, the Shanghai Artificial Intelligence Laboratory and Fudan University built Pangu-Weather, FengWu and FuXi to forecast global weather in the medium range with models trained on past weather data."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2024-02-27","date_basis":"source publication","date_precision":"day","revisions":[{"date":"2026-10-31","change":"Added the record from the report of the China Meteorological Administration.","source":"https://typhooncommittee.org/56th/docs/item%205/Technical%20PresentationsTC56/10.%20Preliminary%20Explorations%20on%20Data-driven%20AI%20Models%20in%20Tropical%20Cyclone%20Prediction.pdf","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["CMA ran the three models in real time and made operational products from them on its forecasting platform.","CMA verified each model's typhoon tracks for the 2023 season."],"must_hold":["The advantage holds at lead times of 36 hours and beyond.","Intensity comes from other models, since the AI models underestimate it."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-10-31","change":"Added the record from the report of the China Meteorological Administration.","source":"https://typhooncommittee.org/56th/docs/item%205/Technical%20PresentationsTC56/10.%20Preliminary%20Explorations%20on%20Data-driven%20AI%20Models%20in%20Tropical%20Cyclone%20Prediction.pdf","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Weather forecasting / tropical cyclones"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"report","citation":"China Meteorological Administration, National Meteorological Centre. Preliminary Explorations on Data-driven AI-Models in Tropical Cyclone Prediction. Presentation by QIAN Qifeng to the 56th session of the ESCAP/WMO Typhoon Committee, 27 February 2024, slide 9.","title":"Preliminary Explorations on Data-driven AI-Models in Tropical Cyclone Prediction","url":"https://typhooncommittee.org/56th/docs/item%205/Technical%20PresentationsTC56/10.%20Preliminary%20Explorations%20on%20Data-driven%20AI%20Models%20in%20Tropical%20Cyclone%20Prediction.pdf","date":"2024-02-27","written_by":"deployer","quote":"For forecast periods of 36 hours and beyond, Pangu-Weather and FengWu have errors smaller than the ECMWF-IFS."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0755-source-0001"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"report","citation":"China Meteorological Administration, National Meteorological Centre. Preliminary Explorations on Data-driven AI-Models in Tropical Cyclone Prediction. Presentation by QIAN Qifeng to the 56th session of the ESCAP/WMO Typhoon Committee, 27 February 2024, slide 9.","title":"Preliminary Explorations on Data-driven AI-Models in Tropical Cyclone Prediction","url":"https://typhooncommittee.org/56th/docs/item%205/Technical%20PresentationsTC56/10.%20Preliminary%20Explorations%20on%20Data-driven%20AI%20Models%20in%20Tropical%20Cyclone%20Prediction.pdf","date":"2024-02-27","written_by":"deployer","quote":"For forecast periods of 36 hours and beyond, Pangu-Weather and FengWu have errors smaller than the ECMWF-IFS."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"CMA's National Meteorological Centre found that Pangu-Weather and FengWu beat ECMWF's model on typhoon tracks beyond 36 hours in 2023, though not within 24 hours.","account":"The National Meteorological Centre (NMC) of the China Meteorological Administration (CMA) runs AI weather models in real time. It has run Pangu-Weather since December 2022 and FengWu and FuXi since July 2023. On 27 February 2024, the NMC presented its verification of 2023 typhoon forecasts to the 56th session of the ESCAP/WMO Typhoon Committee. Within 24 hours, Pangu-Weather and FengWu were comparable to or slightly worse than ECMWF's deterministic model. For 36 hours and beyond, both had smaller errors than the ECMWF model. Averaged over all lead times, mean absolute track errors were 140.8 for Pangu-Weather and 141.1 for FengWu, against 169.7 for ECMWF. FuXi had the largest errors, at 176.7. All three AI models underestimated typhoon intensity and were clearly worse than ECMWF on it. The results come from slides that cover one season."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"Pangu-Weather, FengWu and FuXi, run in real time at CMA's National Meteorological Centre"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"China's National Meteorological Centre finds that Pangu-Weather and FengWu beat ECMWF's model on 2023 typhoon tracks beyond 36 hours","headline":"*Pangu-Weather* beats ECMWF on 2023 typhoon tracks in China's check"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["9a58b95b886466c3281542e9c29ba511c456b5474568cff8df33d908a216d8cc"],"role":"SHA-256 of the complete public record and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public record, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"The National Meteorological Centre (NMC) of the China Meteorological Administration (CMA) runs AI weather models in real time. It has run Pangu-Weather since December 2022 and FengWu and FuXi since July 2023. On 27 February 2024, the NMC presented its verification of 2023 typhoon forecasts to the 56th session of the ESCAP/WMO Typhoon Committee. Within 24 hours, Pangu-Weather and FengWu were comparable to or slightly worse than ECMWF's deterministic model. For 36 hours and beyond, both had smaller errors than the ECMWF model. Averaged over all lead times, mean absolute track errors were 140.8 for Pangu-Weather and 141.1 for FengWu, against 169.7 for ECMWF. FuXi had the largest errors, at 176.7. All three AI models underestimated typhoon intensity and were clearly worse than ECMWF on it. The results come from slides that cover one season.","replication":{"produced_by":["CMA ran the three models in real time and made operational products from them on its forecasting platform.","CMA verified each model's typhoon tracks for the 2023 season."],"must_hold":["The advantage holds at lead times of 36 hours and beyond.","Intensity comes from other models, since the AI models underestimate it."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":29,"reviewed":2,"reviewed_complete":2,"reviewed_partial":0,"record_mapped":27,"review_not_assessed":62,"not_assessed":0,"complete":false,"by_state":{"value_present":29,"not_reported":0,"unresolved":62,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"2fb73fc7af85e270cc8614094bebf7a50635f78e5a921354e18fbb8bcc700acb"}