{ "id": "AIBD-0336", "title": "Udbetaling Danmark's fraud-control models may have produced 135, 36 and 72 cases with benefits stopped or reduced in 2023", "summary": "UDK/ATP ML fraud models (about 60 since 2019): 2023 figures of 135, 36 and 72 cases with revenue from three models; Amnesty (Nov 2024) calls their effectiveness questionable.", "date_observed": "2024-11-12", "date_basis": "unassessed", "date_precision": "unassessed", "benefit_status": null, "source_collection": null, "source_entry": null, "account": "Udbetaling Danmark (UDK), the Danish public authority that has centralized welfare payments since 2012, has its benefits administered by ATP (Arbejdsmarkedets Tillaegspension). ATP designs the fraud-control components of UDK's Joint Data Unit (Den Faelles Dataenhed, established 2014 per the University of Copenhagen's Offentlig AI database). NNIT A/S is the supplier. As of 2019 UDK was reported to use up to 60 algorithms for fraud detection and other purposes. The registry describes profiling and machine learning (it names Naive Bayes classification and cluster analysis) run on linked register data. In a letter to Amnesty International before November 2024, UDK supplied its 2023 statistics for three data-driven controls. The “Really Single” model produced 412 requested cases, 292 established control cases and 135 cases with revenue (benefits stopped or reduced). “Model Abroad” (unreported departures, state and disability pension) gave 511, 351 and 36. The parental-leave controls including the “Fictitious Employment” model gave 491, 207 and 72. UDK cautioned that cases with revenue in a year are not necessarily a proportion of cases established that year. A UDK/ATP control official told Amnesty the team manages 5,000 to 6,000 cases a year, of which around 1,800 come from the models. Amnesty's report of 12 November 2024 concludes it is questionable whether the algorithms are effective. It finds most opened cases end with no further action and the system risks discrimination and mass surveillance. A Copenhagen Municipality control official said more often than not there is no fraud in the cases UDK sends. Background sources give earlier yields. AlgorithmWatch (2020) reports 2017 and 2018 controls unveiled about half a billion kroner in erroneous payments. The Automating Society 2020 Denmark chapter cites a July 2019 report. It counted 705 cases in 2017 and 1,094 in 2018 where payments were stopped, adjusted or repaid, out of 2.7 million screened beneficiaries. The dm.dk FOI report (October 2024) and ATP's page are not among the sources. The report said nearly all 1.2 million pensioners were screened and only 36 cases in 2023 ended with a benefit stopped or reduced. That matches the Model Abroad row. ATP's page credits 260 million kr. a year of savings to IT modernization generally. The 2023 figures carry no day.", "category": "harm prevented", "unexpected": false, "purpose": "Built to flag benefit recipients at elevated risk of receiving unentitled payments for manual control by UDK's control team. The recorded outcome, cases with benefits stopped or reduced, is that purpose.", "system": "Udbetaling Danmark / ATP Den Faelles Dataenhed fraud-control models (about 60 profiling and machine-learning models as of 2019, including 'Really Single', 'Model Abroad' and 'Fictitious Employment')", "model_version": null, "developer": "ATP (Arbejdsmarkedets Tillaegspension) with NNIT A/S as supplier, for Udbetaling Danmark", "deployer": "Udbetaling Danmark (public authority), administered by ATP", "location": "Denmark (national)", "domain": "Public administration: social benefits fraud control", "segment": "National welfare payments authority", "benefit_type": "Reduced Risk", "taxonomy_note": null, "registry_ids": {}, "ratings": [ { "group": "personal", "depth": null, "reason": "No individual beneficiary of the controls is identified. The people directly affected are those investigated. By Amnesty's reading of UDK's figures, 54% to almost 90% of opened cases per model ended with no further action." }, { "group": "organizational", "depth": 1, "reason": "UDK's own 2023 figures: 135, 36 and 72 cases with revenue from three models against 412, 511 and 491 requested cases. No amounts are given and the authority says yearly figures are not directly comparable." }, { "group": "societal", "depth": 1, "reason": "Controls cover about 2.4 million recipients paid around DKK 241 billion in 2021. The documented yield is a few hundred cases a year. Amnesty's report disputes the necessity and proportionality of the processing." } ], "ai_role": { "status": "assessed", "statement": "The models select “requested cases” that UDK's HOK control team screens and investigates. Decisions to stop or reduce benefits are taken by humans after investigation. UDK measures model quality by the share of cases the team accepts for manual processing. Every outcome figure is a UDK statement (FOI responses and a letter) as reproduced verbatim in Amnesty International's report. No official publication of them is among the sources. Amnesty contests effectiveness and alleges discrimination risks, and UDK rejected parts of Amnesty's assessment. NNIT built the models to ATP's specifications. No developer-authored source is on record. The dm.dk report and ATP's page are not among the sources." }, "evidence": [ { "kind": "report", "citation": "Amnesty International (2024). Coded Injustice: Surveillance and Discrimination in Denmark's Automated Welfare State (EUR 18/8709/2024), 12 Nov 2024, Table 5 'Statistics for data-driven controls used by UDK in 2023', p. 43.", "title": "Coded Injustice: Surveillance and Discrimination in Denmark's Automated Welfare State", "url": "https://www.amnesty.org/en/documents/eur18/8709/2024/en/", "date": "2024-11-12", "written_by": "independent", "quote": "135 Cases with revenue – the number of cases that were found to be fraudulent or erroneous after an investigation was conducted" }, { "kind": "report", "citation": "Amnesty International (2024). Coded Injustice, full report PDF, p. 30 (Fraud-control algorithms).", "title": "Coded Injustice (PDF), chapter on fraud-control algorithms", "url": "https://www.amnesty.org/en/wp-content/uploads/2024/11/EUR1887092024ENGLISH.pdf", "date": "2024-11-12", "written_by": "independent", "quote": "As of 2019, UDK were using up to 60 different algorithms for fraud detection and other purposes." }, { "kind": "report", "citation": "Udbetaling Danmark, letter to Amnesty International on 2023 performance of data-driven controls, as reproduced in Amnesty International (2024), Coded Injustice, pp. 42-43.", "title": "UDK statement on how it reports cases with revenue (reproduced in Coded Injustice)", "url": "https://www.amnesty.org/en/wp-content/uploads/2024/11/EUR1887092024ENGLISH.pdf", "date": "2024-11-12", "written_by": "deployer", "quote": "Finally, we report how many cases result in citizens having their benefits stopped or reduced (number of cases with revenue)." }, { "kind": "report", "citation": "Amnesty International (2024). Coded Injustice, p. 44, assessment of effectiveness and interview with a UDK/ATP control official (11 Jan 2024).", "title": "Coded Injustice, p. 44 (effectiveness of the fraud-control algorithms)", "url": "https://www.amnesty.org/en/wp-content/uploads/2024/11/EUR1887092024ENGLISH.pdf", "date": "2024-11-12", "written_by": "independent", "quote": "It is questionable therefore whether the use of fraud control algorithms is effective at identifying fraud in this context." }, { "kind": "press release", "citation": "Amnesty International (2024). Denmark: AI-powered welfare system fuels mass surveillance and risks discriminating against marginalized groups – report. Press release, 12 Nov 2024.", "title": "Denmark: AI-powered welfare system fuels mass surveillance and risks discriminating against marginalized groups", "url": "https://www.amnesty.org/en/latest/news/2024/11/denmark-ai-powered-welfare-system-fuels-mass-surveillance-and-risks-discriminating-against-marginalized-groups/", "date": "2024-11-12", "written_by": "independent", "quote": "UDK and ATP use a system of up to 60 algorithmic models purportedly designed to detect social benefits fraud and flag individuals for further investigations." }, { "kind": "other", "citation": "University of Copenhagen, Faculty of Law (n.d.). Offentlig AI database entry 'Kontrol med udbetaling af ydelser' (Udbetaling Danmark; status: I brug; supplier NNIT A/S; about 60 models as of 2019).", "title": "Kontrol med udbetaling af ydelser (Offentlig AI database entry)", "url": "https://offentlig-ai.dk/projekter/kontrol-med-udbetaling-ydelser", "date": null, "written_by": "independent", "quote": "Udbetaling Danmark udvikler og bruger modeller til kontrol på områder, hvor myndigheden har erfaret, at der forekommer uberettigede udbetalinger." }, { "kind": "report", "citation": "Kayser-Bril, N. (2020). In a quest to optimize welfare management, Denmark built a surveillance behemoth. AlgorithmWatch, 6 Aug 2020.", "title": "In a quest to optimize welfare management, Denmark built a surveillance behemoth", "url": "https://algorithmwatch.org/en/udbetaling-danmark/", "date": "2020-08-06", "written_by": "independent", "quote": "unveiled half a billion Danish crowns in erroneous payment (approximately 70 million euros)" }, { "kind": "report", "citation": "Alfter, B. (2020). Denmark chapter, section 'Controlling social welfare recipients'. In AlgorithmWatch and Bertelsmann Stiftung, Automating Society Report 2020.", "title": "Automating Society Report 2020: Denmark", "url": "https://automatingsociety.algorithmwatch.org/report2020/denmark/", "date": null, "written_by": "independent", "quote": "leading only to 705 cases in 2017 and 1094 cases in 2018 where the payments were stopped, adjusted, or paid back because of mistakes" }, { "kind": "news report", "citation": "Amnesty International Algorithmic Accountability Lab (2024). How We Did It: Amnesty International's Investigation of Algorithms in Denmark's Welfare System. Global Investigative Journalism Network, 14 Nov 2024.", "title": "How We Did It: Amnesty International's Investigation of Algorithms in Denmark's Welfare System", "url": "https://gijn.org/stories/amnesty-internationals-investigation-algorithms-denmarks-welfare-system/", "date": "2024-11-14", "written_by": "independent", "quote": "ATP is responsible for designing the fraud control components of UDK's Joint Data Unit." } ], "replication": { "produced_by": [ "Register linkage of benefits data with civil, housing, business, tax, income, health and employment registers under a 2015 legal authorization", "About 60 profiling and machine-learning models developed by NNIT to ATP's specifications", "A UDK/ATP control team (HOK) that manually screens model output into established control cases and investigates them", "A legal duty to also process all externally reported cases, which the control team says limits the share drawn from the models to about 30%" ], "must_hold": [ "Legal basis for register linkage and model use remains in force despite Amnesty's challenge and the dispute over classification under the EU AI Act", "Control-team capacity of about 5,000 to 6,000 cases a year", "Models continue to be monitored on their training cases, since UDK documents no evaluation metrics", "Political tolerance of the system after the November 2024 criticism" ], "echoes": [ { "date": null, "setting": "2017 and 2018 controls over 2.7 million screened beneficiaries, as reported in a July 2019 report cited by Automating Society 2020: 705 and 1,094 cases with payments stopped, adjusted or repaid", "model_version": null, "outcome": "recurred", "source": "https://automatingsociety.algorithmwatch.org/report2020/denmark/" }, { "date": null, "setting": "dm.dk FOI report, October 2024, as read by an earlier check (not re-read): nearly all 1.2 million pensioners screened, 36 cases in 2023 ending with a benefit stopped or reduced", "model_version": null, "outcome": "recurred", "source": "dm.dk FOI report, October 2024, as summarized in an earlier check of this case; consistent with the Model Abroad row of Amnesty's Table 5" } ] }, "paired_incident": null, "headline": "UDK fraud models may have yielded 135, 36 and 72 cases", "metric": "2023 figures of 135, 36 and 72 cases with revenue from three models", "comparator": null, "later": { "status": "unknown", "date": null, "note": "No later development in the controls is established. The employment minister forwarded a UDK board account to Parliament (Folketinget DIU Alm.del 2024-25 bilag 32), but its contents remain unverified.", "quote": null, "url": null }, "value": { "amount": "half a billion Danish crowns (approximately 70 million euros)", "what": "erroneous payments unveiled by UDK and municipal controls in 2017 and 2018; the source says it is unclear how much was recovered and that the figure includes averted future payments, and it does not isolate the models", "who": "Udbetaling Danmark and the municipalities (the Danish public purse)", "estimate": false, "quote": "In 2017 and 2018, controls by UDK and municipalities unveiled half a billion Danish crowns in erroneous payment (approximately 70 million euros). It is unclear what amount was recovered, what was not recovered due to the beneficiary’s incapacity to repay, and what was an accounting trick (the figure includes future payments that were averted because of the control).", "url": "https://algorithmwatch.org/en/udbetaling-danmark/" }, "developer_country": "Denmark", "deployer_country": "Denmark", "unrated_because": null, "brief_notes": null, "brief": null, "revisions": [ { "date": "2026-10-20", "change": "Record drafted from the cited sources.", "source": null, "editor": "AIBD" }, { "date": "2026-10-21", "change": "Headline reworded to state the event without naming who claims it.", "source": null, "editor": "AIBD" }, { "date": "2026-10-21", "change": "Title reworded to state the event without naming who claims it.", "source": null, "editor": "AIBD" }, { "date": "2026-10-21", "change": "Added the value as stated from the cited sources.", "source": "https://algorithmwatch.org/en/udbetaling-danmark/", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Benefit type set to Reduced Risk under Trustible's full taxonomy of 23 types; it was Improved Organizational Efficiency among the six first used.", "source": "https://trustible.ai/post/a-framework-for-measuring-the-benefits-of-ai/", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Country of the developer and the deployer added from the organization's own site or a registry.", "source": "https://www.atp.dk/en/about-atp; https://da.wikipedia.org/wiki/Udbetaling_Danmark", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Prose pass: a rating reason, account, purpose, the system's part rewritten in shorter sentences with abbreviations expanded; facts, figures and quotations unchanged.", "source": null, "editor": "AIBD" }, { "date": "2026-10-21", "change": "Notes about which sources could be read were reworded for readers; facts, figures and quotations unchanged.", "source": null, "editor": "AIBD" }, { "date": "2026-10-25", "change": "Clarified the displayed field answers and placed measurements and limitations in their respective fields.", "source": null, "editor": "AIBD" } ], "confidence": "studied", "shade": "high", "claim_status": "Independently studied", "claim_basis": "Sources: four independent reports, one independent news report, one independent press release and one other independent source. The cited evidence includes no finding by a regulator or court.", "profile": { "case_id": "AIBD-0336", "catalog_version": "1.0.0", "source_fingerprint": "f17dd05704428e7ad3e78e0e01f15067f7cee4c98a7dc6f09bc1eea7d43f4f91", "collection_refs": [ { "collection_id": "aibd", "version": "snapshot-8bf7cfec9cb185aa" }, { "collection_id": "aibd", "version": "2026-10-25" }, { "collection_id": "adyt", "version": "2026-10-25" }, { "collection_id": "chanticleer", "version": "2026-10-25" } ], "source_ids": [ "AIBD-0336-source-0001", "AIBD-0336-source-0002", "AIBD-0336-source-0003", "AIBD-0336-source-0004", "AIBD-0336-source-0005", "AIBD-0336-source-0006", "AIBD-0336-source-0007", "AIBD-0336-source-0008", "AIBD-0336-source-0009" ], "families": { "strategic assessments": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "record_ids": [] }, "evaluations": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "record_ids": [] }, "monetary observations": { "state": "value_present", "value": { "kind": "text", "text": "An existing source-stated monetary observation is present in the original value object." }, "provenance": { "kind": "legacy_record", "basis": "Existing value object only; its financial interpretation has not been reverified.", "record_fields": [ "value" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial", "record_ids": [ "AIBD-0336-legacy-value" ] }, "institutional decisions": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "record_ids": [] } }, "fields": { "actors[]": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "developer": "ATP (Arbejdsmarkedets Tillaegspension) with NNIT A/S as supplier, for Udbetaling Danmark", "deployer": "Udbetaling Danmark (public authority), administered by ATP" } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "developer", "deployer" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "actors[].ownership_control": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "ai_contribution": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "ai_role": { "status": "assessed", "statement": "The models select “requested cases” that UDK's HOK control team screens and investigates. Decisions to stop or reduce benefits are taken by humans after investigation. UDK measures model quality by the share of cases the team accepts for manual processing. Every outcome figure is a UDK statement (FOI responses and a letter) as reproduced verbatim in Amnesty International's report. No official publication of them is among the sources. Amnesty contests effectiveness and alleges discrimination risks, and UDK rejected parts of Amnesty's assessment. NNIT built the models to ATP's specifications. No developer-authored source is on record. The dm.dk report and ATP's page are not among the sources." } } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "ai_role" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial", "display": { "answer": "Udbetaling Danmark’s models selected benefit files for investigation. Staff screened the selections before deciding whether to investigate further.", "editor_note": "" } }, "ai_relationship": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "category": "harm prevented", "ai_role": { "status": "assessed", "statement": "The models select “requested cases” that UDK's HOK control team screens and investigates. Decisions to stop or reduce benefits are taken by humans after investigation. UDK measures model quality by the share of cases the team accepts for manual processing. Every outcome figure is a UDK statement (FOI responses and a letter) as reproduced verbatim in Amnesty International's report. No official publication of them is among the sources. Amnesty contests effectiveness and alleges discrimination risks, and UDK rejected parts of Amnesty's assessment. NNIT built the models to ATP's specifications. No developer-authored source is on record. The dm.dk report and ATP's page are not among the sources." } } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "ai_role", "category" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "assessments[].alternatives": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "assessments[].assessor_date": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "assessments[].depth": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "ratings": [ { "group": "personal", "reason": "No individual beneficiary of the controls is identified. The people directly affected are those investigated. By Amnesty's reading of UDK's figures, 54% to almost 90% of opened cases per model ended with no further action." }, { "group": "organizational", "depth": 1, "reason": "UDK's own 2023 figures: 135, 36 and 72 cases with revenue from three models against 412, 511 and 491 requested cases. No amounts are given and the authority says yearly figures are not directly comparable." }, { "group": "societal", "depth": 1, "reason": "Controls cover about 2.4 million recipients paid around DKK 241 billion in 2021. The documented yield is a few hundred cases a year. Amnesty's report disputes the necessity and proportionality of the processing." } ] } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "ratings" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "assessments[].display_confidence": { "state": "value_present", "value": { "kind": "structured", "data": { "confidence": "high", "evidence_word": "studied", "claim_status": "Independently studied", "basis": "Sources: four independent reports, one independent news report, one independent press release and one other independent source. The cited evidence includes no finding by a regulator or court.", "interpretation": "Existing display rule preserved; the richer outcome-specific confidence question still needs review." } }, "provenance": { "kind": "curatorial", "basis": "Derived from the existing case display properties; not a new evidentiary judgment.", "record_fields": [], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "assessments[].rationale": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "ratings": [ { "group": "personal", "reason": "No individual beneficiary of the controls is identified. The people directly affected are those investigated. By Amnesty's reading of UDK's figures, 54% to almost 90% of opened cases per model ended with no further action." }, { "group": "organizational", "depth": 1, "reason": "UDK's own 2023 figures: 135, 36 and 72 cases with revenue from three models against 412, 511 and 491 requested cases. No amounts are given and the authority says yearly figures are not directly comparable." }, { "group": "societal", "depth": 1, "reason": "Controls cover about 2.4 million recipients paid around DKK 241 billion in 2021. The documented yield is a few hundred cases a year. Amnesty's report disputes the necessity and proportionality of the processing." } ] } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "ratings" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial", "display": { "answer": "The agency reports several hundred adjustments to benefits, without amounts or a matched comparison. Amnesty’s reading suggests that most opened investigations produced no further action. No individual benefit or wider improvement is established." } }, "assessments[].rubric_version": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "assessments[].status_basis": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "assessments[].target": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "ratings": [ { "group": "personal", "reason": "No individual beneficiary of the controls is identified. The people directly affected are those investigated. By Amnesty's reading of UDK's figures, 54% to almost 90% of opened cases per model ended with no further action." }, { "group": "organizational", "depth": 1, "reason": "UDK's own 2023 figures: 135, 36 and 72 cases with revenue from three models against 412, 511 and 491 requested cases. No amounts are given and the authority says yearly figures are not directly comparable." }, { "group": "societal", "depth": 1, "reason": "Controls cover about 2.4 million recipients paid around DKK 241 billion in 2021. The documented yield is a few hundred cases a year. Amnesty's report disputes the necessity and proportionality of the processing." } ] } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "ratings" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "availability": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "brief": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "case_kind": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "case_links[]": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "claims[].evidence_links": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "claims[].evidence_strength": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "claims[].missingness": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "claims[].proposition": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "cross_border_flows[]": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "decisions[].action_outcome": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "family": "institutional decisions" }, "decisions[].grounds": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "family": "institutional decisions" }, "decisions[].institution_task": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "family": "institutional decisions" }, "decisions[].success_criterion": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "family": "institutional decisions" }, "deployment_stage": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "economics[].amount": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "value": { "amount": "half a billion Danish crowns (approximately 70 million euros)", "what": "erroneous payments unveiled by UDK and municipal controls in 2017 and 2018; the source says it is unclear how much was recovered and that the figure includes averted future payments, and it does not isolate the models", "who": "Udbetaling Danmark and the municipalities (the Danish public purse)", "estimate": false, "quote": "In 2017 and 2018, controls by UDK and municipalities unveiled half a billion Danish crowns in erroneous payment (approximately 70 million euros). It is unclear what amount was recovered, what was not recovered due to the beneficiary’s incapacity to repay, and what was an accounting trick (the figure includes future payments that were averted because of the control).", "url": "https://algorithmwatch.org/en/udbetaling-danmark/" } } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "value" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "economics[].costs[]": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "economics[].counterfactual": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "economics[].currency_price_basis": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "economics[].gross_net": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "economics[].measure_type": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "value": { "amount": "half a billion Danish crowns (approximately 70 million euros)", "what": "erroneous payments unveiled by UDK and municipal controls in 2017 and 2018; the source says it is unclear how much was recovered and that the figure includes averted future payments, and it does not isolate the models", "who": "Udbetaling Danmark and the municipalities (the Danish public purse)", "estimate": false, "quote": "In 2017 and 2018, controls by UDK and municipalities unveiled half a billion Danish crowns in erroneous payment (approximately 70 million euros). It is unclear what amount was recovered, what was not recovered due to the beneficiary’s incapacity to repay, and what was an accounting trick (the figure includes future payments that were averted because of the control).", "url": "https://algorithmwatch.org/en/udbetaling-danmark/" } } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "value" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "economics[].period_unit": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "economics[].realization": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "economics[].recipient_payer": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "value": { "amount": "half a billion Danish crowns (approximately 70 million euros)", "what": "erroneous payments unveiled by UDK and municipal controls in 2017 and 2018; the source says it is unclear how much was recovered and that the figure includes averted future payments, and it does not isolate the models", "who": "Udbetaling Danmark and the municipalities (the Danish public purse)", "estimate": false, "quote": "In 2017 and 2018, controls by UDK and municipalities unveiled half a billion Danish crowns in erroneous payment (approximately 70 million euros). It is unclear what amount was recovered, what was not recovered due to the beneficiary’s incapacity to repay, and what was an accounting trick (the figure includes future payments that were averted because of the control).", "url": "https://algorithmwatch.org/en/udbetaling-danmark/" } } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "value" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "economics[].return_payback": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "economics[].valuation_method": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "evaluations[].ai_uplift": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "family": "evaluations" }, "evaluations[].comparator": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "family": "evaluations" }, "evaluations[].design": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "family": "evaluations" }, "evaluations[].effect": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "family": "evaluations", "display": { "answer": "The three controls reported 135, 36, and 72 cases with payments stopped or reduced in 2023. The same controls requested 412, 511, and 491 cases, but the agency cautions that the annual totals do not necessarily refer to the same cases." } }, "evaluations[].exposure": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "family": "evaluations" }, "evaluations[].id": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "family": "evaluations" }, "evaluations[].measurement_method": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "family": "evaluations" }, "evaluations[].metric": { "state": "not_assessed", "provenance": { "kind": "legacy_record", "basis": "Mapped from the existing case description without a new external source review.", "record_fields": [ "metric" ], "source_refs": [] }, "review_state": "not_assessed", "coverage": "partial", "family": "evaluations", "display": { "answer": "Cases requested, cases opened for investigation, and cases in which benefits were stopped or reduced." } }, "evaluations[].reproduction_materials": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "family": "evaluations" }, "evaluations[].setting_sample": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "family": "evaluations" }, "evaluations[].uncertainty_limitations": { "state": "not_assessed", "provenance": { "kind": "legacy_record", "basis": "Mapped from the existing case description without a new external source review.", "record_fields": [ "ai_role", "account" ], "source_refs": [] }, "review_state": "not_assessed", "coverage": "partial", "family": "evaluations", "display": { "answer": "The outcome figures come from the agency’s letter reproduced by Amnesty. No monetary amounts or matched case cohorts are given. Amnesty disputes the controls’ effectiveness and necessity, while the agency rejected parts of that assessment." } }, "external_references[]": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "follow_up[]": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "later": { "status": "unknown", "note": "No later development in the controls is established. The employment minister forwarded a UDK board account to Parliament (Folketinget DIU Alm.del 2024-25 bilag 32), but its contents remain unverified." } } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "later" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "human_oversight": { "state": "value_present", "value": { "kind": "text", "text": "Staff investigated the files and decided whether benefits should be reduced or stopped." }, "provenance": { "kind": "legacy_record", "basis": "Mapped from the existing case description without a new external source review.", "record_fields": [ "ai_role", "account" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "id": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "id": "AIBD-0336" } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "id" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "infrastructure_dependencies[]": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "outcomes[].beneficiaries": { "state": "value_present", "value": { "kind": "text", "text": "Udbetaling Danmark is the intended institutional beneficiary. No individual beneficiary of the controls is identified." }, "provenance": { "kind": "legacy_record", "basis": "Moved the specific information already present in the entry into its applicable question. No new source review is implied.", "record_fields": [ "ratings", "account" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial", "display": { "answer": "Udbetaling Danmark is the intended institutional beneficiary. No individual beneficiary of the controls is identified." } }, "outcomes[].beneficiary_count": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "display": { "answer": "No count of beneficiaries is established. The screened population is not a measure of people who benefited." } }, "outcomes[].benefit_type": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "benefit_type": "Reduced Risk" } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "benefit_type" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "outcomes[].claim": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "account": "Udbetaling Danmark (UDK), the Danish public authority that has centralized welfare payments since 2012, has its benefits administered by ATP (Arbejdsmarkedets Tillaegspension). ATP designs the fraud-control components of UDK's Joint Data Unit (Den Faelles Dataenhed, established 2014 per the University of Copenhagen's Offentlig AI database). NNIT A/S is the supplier. As of 2019 UDK was reported to use up to 60 algorithms for fraud detection and other purposes. The registry describes profiling and machine learning (it names Naive Bayes classification and cluster analysis) run on linked register data. In a letter to Amnesty International before November 2024, UDK supplied its 2023 statistics for three data-driven controls. The “Really Single” model produced 412 requested cases, 292 established control cases and 135 cases with revenue (benefits stopped or reduced). “Model Abroad” (unreported departures, state and disability pension) gave 511, 351 and 36. The parental-leave controls including the “Fictitious Employment” model gave 491, 207 and 72. UDK cautioned that cases with revenue in a year are not necessarily a proportion of cases established that year. A UDK/ATP control official told Amnesty the team manages 5,000 to 6,000 cases a year, of which around 1,800 come from the models. Amnesty's report of 12 November 2024 concludes it is questionable whether the algorithms are effective. It finds most opened cases end with no further action and the system risks discrimination and mass surveillance. A Copenhagen Municipality control official said more often than not there is no fraud in the cases UDK sends. Background sources give earlier yields. AlgorithmWatch (2020) reports 2017 and 2018 controls unveiled about half a billion kroner in erroneous payments. The Automating Society 2020 Denmark chapter cites a July 2019 report. It counted 705 cases in 2017 and 1,094 in 2018 where payments were stopped, adjusted or repaid, out of 2.7 million screened beneficiaries. The dm.dk FOI report (October 2024) and ATP's page are not among the sources. The report said nearly all 1.2 million pensioners were screened and only 36 cases in 2023 ended with a benefit stopped or reduced. That matches the Model Abroad row. ATP's page credits 260 million kr. a year of savings to IT modernization generally. The 2023 figures carry no day.", "benefit_type": "Reduced Risk" } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "account", "benefit_type" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial", "display": { "answer": "Udbetaling Danmark reported 135, 36, and 72 cases with benefits stopped or reduced through three controls in 2023." } }, "outcomes[].distribution": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "outcomes[].duration": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "outcomes[].id": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "outcomes[].stakeholder_group": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "ratings": [ { "group": "personal", "reason": "No individual beneficiary of the controls is identified. The people directly affected are those investigated. By Amnesty's reading of UDK's figures, 54% to almost 90% of opened cases per model ended with no further action." }, { "group": "organizational", "depth": 1, "reason": "UDK's own 2023 figures: 135, 36 and 72 cases with revenue from three models against 412, 511 and 491 requested cases. No amounts are given and the authority says yearly figures are not directly comparable." }, { "group": "societal", "depth": 1, "reason": "Controls cover about 2.4 million recipients paid around DKK 241 billion in 2021. The documented yield is a few hundred cases a year. Amnesty's report disputes the necessity and proportionality of the processing." } ] } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "ratings" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "outcomes[].status": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "outcomes[].tradeoffs": { "state": "value_present", "value": { "kind": "text", "text": "Amnesty International alleges discrimination risks and disproportionate surveillance. Udbetaling Danmark disputed parts of its assessment." }, "provenance": { "kind": "legacy_record", "basis": "Moved the specific information already present in the entry into its applicable question. No new source review is implied.", "record_fields": [ "account", "ai_role" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial", "display": { "answer": "Amnesty International alleges discrimination risks and disproportionate surveillance. Udbetaling Danmark disputed parts of its assessment." } }, "outcomes[].unexpected": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "unexpected": false, "purpose": "Built to flag benefit recipients at elevated risk of receiving unentitled payments for manual control by UDK's control team. The recorded outcome, cases with benefits stopped or reduced, is that purpose." } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "unexpected", "purpose" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "places[]": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "location": "Denmark (national)", "developer_country": "Denmark", "deployer_country": "Denmark" } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "location", "developer_country", "deployer_country" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "purpose": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "purpose": "Built to flag benefit recipients at elevated risk of receiving unentitled payments for manual control by UDK's control team. The recorded outcome, cases with benefits stopped or reduced, is that purpose." } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "purpose" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "record_dates": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "date_observed": "2024-11-12", "revisions": [ { "date": "2026-10-20", "change": "Record drafted from the cited sources.", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Headline reworded to state the event without naming who claims it.", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Title reworded to state the event without naming who claims it.", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Added the value as stated from the cited sources.", "source": "https://algorithmwatch.org/en/udbetaling-danmark/", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Benefit type set to Reduced Risk under Trustible's full taxonomy of 23 types; it was Improved Organizational Efficiency among the six first used.", "source": "https://trustible.ai/post/a-framework-for-measuring-the-benefits-of-ai/", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Country of the developer and the deployer added from the organization's own site or a registry.", "source": "https://www.atp.dk/en/about-atp; https://da.wikipedia.org/wiki/Udbetaling_Danmark", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Prose pass: a rating reason, account, purpose, the system's part rewritten in shorter sentences with abbreviations expanded; facts, figures and quotations unchanged.", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Notes about which sources could be read were reworded for readers; facts, figures and quotations unchanged.", "editor": "AIBD" }, { "date": "2026-10-25", "change": "Clarified the displayed field answers and placed measurements and limitations in their respective fields.", "editor": "AIBD" } ] } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "date_observed", "revisions" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "replication.attempts[]": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "replication": { "produced_by": [ "Register linkage of benefits data with civil, housing, business, tax, income, health and employment registers under a 2015 legal authorization", "About 60 profiling and machine-learning models developed by NNIT to ATP's specifications", "A UDK/ATP control team (HOK) that manually screens model output into established control cases and investigates them", "A legal duty to also process all externally reported cases, which the control team says limits the share drawn from the models to about 30%" ], "must_hold": [ "Legal basis for register linkage and model use remains in force despite Amnesty's challenge and the dispute over classification under the EU AI Act", "Control-team capacity of about 5,000 to 6,000 cases a year", "Models continue to be monitored on their training cases, since UDK documents no evaluation metrics", "Political tolerance of the system after the November 2024 criticism" ], "echoes": [ { "setting": "2017 and 2018 controls over 2.7 million screened beneficiaries, as reported in a July 2019 report cited by Automating Society 2020: 705 and 1,094 cases with payments stopped, adjusted or repaid", "outcome": "recurred", "source": "https://automatingsociety.algorithmwatch.org/report2020/denmark/" }, { "setting": "dm.dk FOI report, October 2024, as read by an earlier check (not re-read): nearly all 1.2 million pensioners screened, 36 cases in 2023 ending with a benefit stopped or reduced", "outcome": "recurred", "source": "dm.dk FOI report, October 2024, as summarized in an earlier check of this case; consistent with the Model Abroad row of Amnesty's Table 5" } ] } } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "replication" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "replication.conditions": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "replication": { "produced_by": [ "Register linkage of benefits data with civil, housing, business, tax, income, health and employment registers under a 2015 legal authorization", "About 60 profiling and machine-learning models developed by NNIT to ATP's specifications", "A UDK/ATP control team (HOK) that manually screens model output into established control cases and investigates them", "A legal duty to also process all externally reported cases, which the control team says limits the share drawn from the models to about 30%" ], "must_hold": [ "Legal basis for register linkage and model use remains in force despite Amnesty's challenge and the dispute over classification under the EU AI Act", "Control-team capacity of about 5,000 to 6,000 cases a year", "Models continue to be monitored on their training cases, since UDK documents no evaluation metrics", "Political tolerance of the system after the November 2024 criticism" ], "echoes": [ { "setting": "2017 and 2018 controls over 2.7 million screened beneficiaries, as reported in a July 2019 report cited by Automating Society 2020: 705 and 1,094 cases with payments stopped, adjusted or repaid", "outcome": "recurred", "source": "https://automatingsociety.algorithmwatch.org/report2020/denmark/" }, { "setting": "dm.dk FOI report, October 2024, as read by an earlier check (not re-read): nearly all 1.2 million pensioners screened, 36 cases in 2023 ending with a benefit stopped or reduced", "outcome": "recurred", "source": "dm.dk FOI report, October 2024, as summarized in an earlier check of this case; consistent with the Model Abroad row of Amnesty's Table 5" } ] } } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "replication" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial", "display": { "answer": "Register access and model use need a lawful basis. Staff must investigate the flagged cases. The cited material does not provide evaluation measures that would establish how well the controls perform." } }, "revision_history[]": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "revisions": [ { "date": "2026-10-20", "change": "Record drafted from the cited sources.", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Headline reworded to state the event without naming who claims it.", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Title reworded to state the event without naming who claims it.", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Added the value as stated from the cited sources.", "source": "https://algorithmwatch.org/en/udbetaling-danmark/", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Benefit type set to Reduced Risk under Trustible's full taxonomy of 23 types; it was Improved Organizational Efficiency among the six first used.", "source": "https://trustible.ai/post/a-framework-for-measuring-the-benefits-of-ai/", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Country of the developer and the deployer added from the organization's own site or a registry.", "source": "https://www.atp.dk/en/about-atp; https://da.wikipedia.org/wiki/Udbetaling_Danmark", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Prose pass: a rating reason, account, purpose, the system's part rewritten in shorter sentences with abbreviations expanded; facts, figures and quotations unchanged.", "editor": "AIBD" }, { "date": "2026-10-21", "change": "Notes about which sources could be read were reworded for readers; facts, figures and quotations unchanged.", "editor": "AIBD" }, { "date": "2026-10-25", "change": "Clarified the displayed field answers and placed measurements and limitations in their respective fields.", "editor": "AIBD" } ] } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "revisions" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "sector": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "domain": "Public administration: social benefits fraud control", "segment": "National welfare payments authority" } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "domain", "segment" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "sources[].access_version": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "sources[].citation_locator": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "evidence": [ { "kind": "report", "citation": "Amnesty International (2024). Coded Injustice: Surveillance and Discrimination in Denmark's Automated Welfare State (EUR 18/8709/2024), 12 Nov 2024, Table 5 'Statistics for data-driven controls used by UDK in 2023', p. 43.", "title": "Coded Injustice: Surveillance and Discrimination in Denmark's Automated Welfare State", "url": "https://www.amnesty.org/en/documents/eur18/8709/2024/en/", "date": "2024-11-12", "written_by": "independent", "quote": "135 Cases with revenue – the number of cases that were found to be fraudulent or erroneous after an investigation was conducted" }, { "kind": "report", "citation": "Amnesty International (2024). Coded Injustice, full report PDF, p. 30 (Fraud-control algorithms).", "title": "Coded Injustice (PDF), chapter on fraud-control algorithms", "url": "https://www.amnesty.org/en/wp-content/uploads/2024/11/EUR1887092024ENGLISH.pdf", "date": "2024-11-12", "written_by": "independent", "quote": "As of 2019, UDK were using up to 60 different algorithms for fraud detection and other purposes." }, { "kind": "report", "citation": "Udbetaling Danmark, letter to Amnesty International on 2023 performance of data-driven controls, as reproduced in Amnesty International (2024), Coded Injustice, pp. 42-43.", "title": "UDK statement on how it reports cases with revenue (reproduced in Coded Injustice)", "url": "https://www.amnesty.org/en/wp-content/uploads/2024/11/EUR1887092024ENGLISH.pdf", "date": "2024-11-12", "written_by": "deployer", "quote": "Finally, we report how many cases result in citizens having their benefits stopped or reduced (number of cases with revenue)." }, { "kind": "report", "citation": "Amnesty International (2024). Coded Injustice, p. 44, assessment of effectiveness and interview with a UDK/ATP control official (11 Jan 2024).", "title": "Coded Injustice, p. 44 (effectiveness of the fraud-control algorithms)", "url": "https://www.amnesty.org/en/wp-content/uploads/2024/11/EUR1887092024ENGLISH.pdf", "date": "2024-11-12", "written_by": "independent", "quote": "It is questionable therefore whether the use of fraud control algorithms is effective at identifying fraud in this context." }, { "kind": "press release", "citation": "Amnesty International (2024). Denmark: AI-powered welfare system fuels mass surveillance and risks discriminating against marginalized groups – report. Press release, 12 Nov 2024.", "title": "Denmark: AI-powered welfare system fuels mass surveillance and risks discriminating against marginalized groups", "url": "https://www.amnesty.org/en/latest/news/2024/11/denmark-ai-powered-welfare-system-fuels-mass-surveillance-and-risks-discriminating-against-marginalized-groups/", "date": "2024-11-12", "written_by": "independent", "quote": "UDK and ATP use a system of up to 60 algorithmic models purportedly designed to detect social benefits fraud and flag individuals for further investigations." }, { "kind": "other", "citation": "University of Copenhagen, Faculty of Law (n.d.). Offentlig AI database entry 'Kontrol med udbetaling af ydelser' (Udbetaling Danmark; status: I brug; supplier NNIT A/S; about 60 models as of 2019).", "title": "Kontrol med udbetaling af ydelser (Offentlig AI database entry)", "url": "https://offentlig-ai.dk/projekter/kontrol-med-udbetaling-ydelser", "written_by": "independent", "quote": "Udbetaling Danmark udvikler og bruger modeller til kontrol på områder, hvor myndigheden har erfaret, at der forekommer uberettigede udbetalinger." }, { "kind": "report", "citation": "Kayser-Bril, N. (2020). In a quest to optimize welfare management, Denmark built a surveillance behemoth. AlgorithmWatch, 6 Aug 2020.", "title": "In a quest to optimize welfare management, Denmark built a surveillance behemoth", "url": "https://algorithmwatch.org/en/udbetaling-danmark/", "date": "2020-08-06", "written_by": "independent", "quote": "unveiled half a billion Danish crowns in erroneous payment (approximately 70 million euros)" }, { "kind": "report", "citation": "Alfter, B. (2020). Denmark chapter, section 'Controlling social welfare recipients'. In AlgorithmWatch and Bertelsmann Stiftung, Automating Society Report 2020.", "title": "Automating Society Report 2020: Denmark", "url": "https://automatingsociety.algorithmwatch.org/report2020/denmark/", "written_by": "independent", "quote": "leading only to 705 cases in 2017 and 1094 cases in 2018 where the payments were stopped, adjusted, or paid back because of mistakes" }, { "kind": "news report", "citation": "Amnesty International Algorithmic Accountability Lab (2024). How We Did It: Amnesty International's Investigation of Algorithms in Denmark's Welfare System. Global Investigative Journalism Network, 14 Nov 2024.", "title": "How We Did It: Amnesty International's Investigation of Algorithms in Denmark's Welfare System", "url": "https://gijn.org/stories/amnesty-internationals-investigation-algorithms-denmarks-welfare-system/", "date": "2024-11-14", "written_by": "independent", "quote": "ATP is responsible for designing the fraud control components of UDK's Joint Data Unit." } ] } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "evidence" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "sources[].id": { "state": "value_present", "value": { "kind": "identifiers", "identifiers": [ "AIBD-0336-source-0001", "AIBD-0336-source-0002", "AIBD-0336-source-0003", "AIBD-0336-source-0004", "AIBD-0336-source-0005", "AIBD-0336-source-0006", "AIBD-0336-source-0007", "AIBD-0336-source-0008", "AIBD-0336-source-0009" ], "role": "Existing evidence items, in their original order" }, "provenance": { "kind": "curatorial", "basis": "Stable within the fingerprinted record version; no source content or source review has been added.", "record_fields": [], "source_refs": [] }, "review_state": "record_mapped", "coverage": "complete" }, "sources[].language": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "sources[].relationship": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "evidence": [ { "kind": "report", "citation": "Amnesty International (2024). Coded Injustice: Surveillance and Discrimination in Denmark's Automated Welfare State (EUR 18/8709/2024), 12 Nov 2024, Table 5 'Statistics for data-driven controls used by UDK in 2023', p. 43.", "title": "Coded Injustice: Surveillance and Discrimination in Denmark's Automated Welfare State", "url": "https://www.amnesty.org/en/documents/eur18/8709/2024/en/", "date": "2024-11-12", "written_by": "independent", "quote": "135 Cases with revenue – the number of cases that were found to be fraudulent or erroneous after an investigation was conducted" }, { "kind": "report", "citation": "Amnesty International (2024). Coded Injustice, full report PDF, p. 30 (Fraud-control algorithms).", "title": "Coded Injustice (PDF), chapter on fraud-control algorithms", "url": "https://www.amnesty.org/en/wp-content/uploads/2024/11/EUR1887092024ENGLISH.pdf", "date": "2024-11-12", "written_by": "independent", "quote": "As of 2019, UDK were using up to 60 different algorithms for fraud detection and other purposes." }, { "kind": "report", "citation": "Udbetaling Danmark, letter to Amnesty International on 2023 performance of data-driven controls, as reproduced in Amnesty International (2024), Coded Injustice, pp. 42-43.", "title": "UDK statement on how it reports cases with revenue (reproduced in Coded Injustice)", "url": "https://www.amnesty.org/en/wp-content/uploads/2024/11/EUR1887092024ENGLISH.pdf", "date": "2024-11-12", "written_by": "deployer", "quote": "Finally, we report how many cases result in citizens having their benefits stopped or reduced (number of cases with revenue)." }, { "kind": "report", "citation": "Amnesty International (2024). Coded Injustice, p. 44, assessment of effectiveness and interview with a UDK/ATP control official (11 Jan 2024).", "title": "Coded Injustice, p. 44 (effectiveness of the fraud-control algorithms)", "url": "https://www.amnesty.org/en/wp-content/uploads/2024/11/EUR1887092024ENGLISH.pdf", "date": "2024-11-12", "written_by": "independent", "quote": "It is questionable therefore whether the use of fraud control algorithms is effective at identifying fraud in this context." }, { "kind": "press release", "citation": "Amnesty International (2024). Denmark: AI-powered welfare system fuels mass surveillance and risks discriminating against marginalized groups – report. Press release, 12 Nov 2024.", "title": "Denmark: AI-powered welfare system fuels mass surveillance and risks discriminating against marginalized groups", "url": "https://www.amnesty.org/en/latest/news/2024/11/denmark-ai-powered-welfare-system-fuels-mass-surveillance-and-risks-discriminating-against-marginalized-groups/", "date": "2024-11-12", "written_by": "independent", "quote": "UDK and ATP use a system of up to 60 algorithmic models purportedly designed to detect social benefits fraud and flag individuals for further investigations." }, { "kind": "other", "citation": "University of Copenhagen, Faculty of Law (n.d.). Offentlig AI database entry 'Kontrol med udbetaling af ydelser' (Udbetaling Danmark; status: I brug; supplier NNIT A/S; about 60 models as of 2019).", "title": "Kontrol med udbetaling af ydelser (Offentlig AI database entry)", "url": "https://offentlig-ai.dk/projekter/kontrol-med-udbetaling-ydelser", "written_by": "independent", "quote": "Udbetaling Danmark udvikler og bruger modeller til kontrol på områder, hvor myndigheden har erfaret, at der forekommer uberettigede udbetalinger." }, { "kind": "report", "citation": "Kayser-Bril, N. (2020). In a quest to optimize welfare management, Denmark built a surveillance behemoth. AlgorithmWatch, 6 Aug 2020.", "title": "In a quest to optimize welfare management, Denmark built a surveillance behemoth", "url": "https://algorithmwatch.org/en/udbetaling-danmark/", "date": "2020-08-06", "written_by": "independent", "quote": "unveiled half a billion Danish crowns in erroneous payment (approximately 70 million euros)" }, { "kind": "report", "citation": "Alfter, B. (2020). Denmark chapter, section 'Controlling social welfare recipients'. In AlgorithmWatch and Bertelsmann Stiftung, Automating Society Report 2020.", "title": "Automating Society Report 2020: Denmark", "url": "https://automatingsociety.algorithmwatch.org/report2020/denmark/", "written_by": "independent", "quote": "leading only to 705 cases in 2017 and 1094 cases in 2018 where the payments were stopped, adjusted, or paid back because of mistakes" }, { "kind": "news report", "citation": "Amnesty International Algorithmic Accountability Lab (2024). How We Did It: Amnesty International's Investigation of Algorithms in Denmark's Welfare System. Global Investigative Journalism Network, 14 Nov 2024.", "title": "How We Did It: Amnesty International's Investigation of Algorithms in Denmark's Welfare System", "url": "https://gijn.org/stories/amnesty-internationals-investigation-algorithms-denmarks-welfare-system/", "date": "2024-11-14", "written_by": "independent", "quote": "ATP is responsible for designing the fraud control components of UDK's Joint Data Unit." } ] } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "evidence" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "sources[].upstream_sources": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "strategic_assessments[]": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial", "family": "strategic assessments" }, "summary": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "summary": "UDK/ATP ML fraud models (about 60 since 2019): 2023 figures of 135, 36 and 72 cases with revenue from three models; Amnesty (Nov 2024) calls their effectiveness questionable.", "account": "Udbetaling Danmark (UDK), the Danish public authority that has centralized welfare payments since 2012, has its benefits administered by ATP (Arbejdsmarkedets Tillaegspension). ATP designs the fraud-control components of UDK's Joint Data Unit (Den Faelles Dataenhed, established 2014 per the University of Copenhagen's Offentlig AI database). NNIT A/S is the supplier. As of 2019 UDK was reported to use up to 60 algorithms for fraud detection and other purposes. The registry describes profiling and machine learning (it names Naive Bayes classification and cluster analysis) run on linked register data. In a letter to Amnesty International before November 2024, UDK supplied its 2023 statistics for three data-driven controls. The “Really Single” model produced 412 requested cases, 292 established control cases and 135 cases with revenue (benefits stopped or reduced). “Model Abroad” (unreported departures, state and disability pension) gave 511, 351 and 36. The parental-leave controls including the “Fictitious Employment” model gave 491, 207 and 72. UDK cautioned that cases with revenue in a year are not necessarily a proportion of cases established that year. A UDK/ATP control official told Amnesty the team manages 5,000 to 6,000 cases a year, of which around 1,800 come from the models. Amnesty's report of 12 November 2024 concludes it is questionable whether the algorithms are effective. It finds most opened cases end with no further action and the system risks discrimination and mass surveillance. A Copenhagen Municipality control official said more often than not there is no fraud in the cases UDK sends. Background sources give earlier yields. AlgorithmWatch (2020) reports 2017 and 2018 controls unveiled about half a billion kroner in erroneous payments. The Automating Society 2020 Denmark chapter cites a July 2019 report. It counted 705 cases in 2017 and 1,094 in 2018 where payments were stopped, adjusted or repaid, out of 2.7 million screened beneficiaries. The dm.dk FOI report (October 2024) and ATP's page are not among the sources. The report said nearly all 1.2 million pensioners were screened and only 36 cases in 2023 ended with a benefit stopped or reduced. That matches the Model Abroad row. ATP's page credits 260 million kr. a year of savings to IT modernization generally. The 2023 figures carry no day." } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "summary", "account" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "systems[].autonomy_permissions": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "systems[].component_role": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "systems[].deployment_configuration": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "systems[].inputs_outputs": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "systems[].model_name": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "systems[].model_version": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "systems[].product_name": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "system": "Udbetaling Danmark / ATP Den Faelles Dataenhed fraud-control models (about 60 profiling and machine-learning models as of 2019, including 'Really Single', 'Model Abroad' and 'Fictitious Employment')" } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "system" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "systems[].provider": { "state": "value_present", "value": { "kind": "structured", "data": { "application_developer": "ATP (Arbejdsmarkedets Tillaegspension) with NNIT A/S as supplier, for Udbetaling Danmark" } }, "provenance": { "kind": "legacy_record", "basis": "Mapped from the existing case description without a new external source review.", "record_fields": [ "developer" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "taxonomy_crosswalks[]": { "state": "not_assessed", "review_state": "not_assessed", "coverage": "partial" }, "title": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "title": "Udbetaling Danmark's fraud-control models may have produced 135, 36 and 72 cases with benefits stopped or reduced in 2023", "headline": "UDK fraud models may have yielded 135, 36 and 72 cases" } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "title", "headline" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial" }, "version": { "state": "value_present", "value": { "kind": "identifiers", "identifiers": [ "f17dd05704428e7ad3e78e0e01f15067f7cee4c98a7dc6f09bc1eea7d43f4f91" ], "role": "SHA-256 of normalized original case data" }, "provenance": { "kind": "curatorial", "basis": "Deterministic content version assigned by the local migration.", "record_fields": [], "source_refs": [] }, "review_state": "record_mapped", "coverage": "complete" }, "workflow": { "state": "value_present", "value": { "kind": "record_fields", "fields": { "account": "Udbetaling Danmark (UDK), the Danish public authority that has centralized welfare payments since 2012, has its benefits administered by ATP (Arbejdsmarkedets Tillaegspension). ATP designs the fraud-control components of UDK's Joint Data Unit (Den Faelles Dataenhed, established 2014 per the University of Copenhagen's Offentlig AI database). NNIT A/S is the supplier. As of 2019 UDK was reported to use up to 60 algorithms for fraud detection and other purposes. The registry describes profiling and machine learning (it names Naive Bayes classification and cluster analysis) run on linked register data. In a letter to Amnesty International before November 2024, UDK supplied its 2023 statistics for three data-driven controls. The “Really Single” model produced 412 requested cases, 292 established control cases and 135 cases with revenue (benefits stopped or reduced). “Model Abroad” (unreported departures, state and disability pension) gave 511, 351 and 36. The parental-leave controls including the “Fictitious Employment” model gave 491, 207 and 72. UDK cautioned that cases with revenue in a year are not necessarily a proportion of cases established that year. A UDK/ATP control official told Amnesty the team manages 5,000 to 6,000 cases a year, of which around 1,800 come from the models. Amnesty's report of 12 November 2024 concludes it is questionable whether the algorithms are effective. It finds most opened cases end with no further action and the system risks discrimination and mass surveillance. A Copenhagen Municipality control official said more often than not there is no fraud in the cases UDK sends. Background sources give earlier yields. AlgorithmWatch (2020) reports 2017 and 2018 controls unveiled about half a billion kroner in erroneous payments. The Automating Society 2020 Denmark chapter cites a July 2019 report. It counted 705 cases in 2017 and 1,094 in 2018 where payments were stopped, adjusted or repaid, out of 2.7 million screened beneficiaries. The dm.dk FOI report (October 2024) and ATP's page are not among the sources. The report said nearly all 1.2 million pensioners were screened and only 36 cases in 2023 ended with a benefit stopped or reduced. That matches the Model Abroad row. ATP's page credits 260 million kr. a year of savings to IT modernization generally. The 2023 figures carry no day.", "replication": { "produced_by": [ "Register linkage of benefits data with civil, housing, business, tax, income, health and employment registers under a 2015 legal authorization", "About 60 profiling and machine-learning models developed by NNIT to ATP's specifications", "A UDK/ATP control team (HOK) that manually screens model output into established control cases and investigates them", "A legal duty to also process all externally reported cases, which the control team says limits the share drawn from the models to about 30%" ], "must_hold": [ "Legal basis for register linkage and model use remains in force despite Amnesty's challenge and the dispute over classification under the EU AI Act", "Control-team capacity of about 5,000 to 6,000 cases a year", "Models continue to be monitored on their training cases, since UDK documents no evaluation metrics", "Political tolerance of the system after the November 2024 criticism" ], "echoes": [ { "setting": "2017 and 2018 controls over 2.7 million screened beneficiaries, as reported in a July 2019 report cited by Automating Society 2020: 705 and 1,094 cases with payments stopped, adjusted or repaid", "outcome": "recurred", "source": "https://automatingsociety.algorithmwatch.org/report2020/denmark/" }, { "setting": "dm.dk FOI report, October 2024, as read by an earlier check (not re-read): nearly all 1.2 million pensioners screened, 36 cases in 2023 ending with a benefit stopped or reduced", "outcome": "recurred", "source": "dm.dk FOI report, October 2024, as summarized in an earlier check of this case; consistent with the Model Abroad row of Amnesty's Table 5" } ] } } }, "provenance": { "kind": "legacy_record", "basis": "Preserved from the existing AIBD record; external sources were not rechecked.", "record_fields": [ "account", "replication" ], "source_refs": [] }, "review_state": "record_mapped", "coverage": "partial", "display": { "answer": "Models examine linked public registers and flag recipients for review. A UDK/ATP team that handles about 5,000 to 6,000 cases a year screens the results and investigates selected cases. Around 1,800 of its annual cases come from the models." } } }, "summary": { "total": 91, "addressed": 91, "known": 35, "reviewed": 0, "record_mapped": 35, "not_assessed": 56, "complete": false, "by_state": { "value_present": 35, "not_reported": 0, "unresolved": 0, "not_assessed": 56, "not_applicable": 0, "withheld": 0, "source_unavailable": 0 } } } }