{"id":"AIBD-0743","title":"Evaluators for the Department of Health and Social Care find that DERM safely sorted referrals for suspected skin cancer at four NHS sites","summary":"Evaluators of DERM at four NHS sites found that it triaged suspected skin cancer accurately and safely, cut unnecessary referrals and freed consultant time. Its gains in quality of life were unclear.","date_observed":"2024-01-01","date_basis":"source publication","date_precision":"year","benefit_status":"reported","source_collection":null,"source_entry":null,"account":"Skin Analytics implemented DERM, a UKCA class IIa AI device, at six NHS sites under phase IV of the AI Awards in Health and Social Care. The Department of Health and Social Care commissioned an evaluation of four of them, which Unity Insights summarized. In secondary care, DERM assessed 5,678 lesions from 4,159 referrals. The evaluators report pathway sensitivity for melanoma above 90% at every site and a secondary care sensitivity of 97.4%. A consultant dermatologist reviewed each case that DERM classified as low risk. This second read overturned 36% of potential discharges, and trust dermatologists then discharged 41% of those. Consultants found seven cancers among the 754 overturned cases, most of them at low risk. In a model of costs and benefits, the evaluators estimated that DERM saves 2,600 dermatologist appointments a year in an average trust. They estimated that it returns £1.7 for every £1 spent in secondary care. Their analysis of costs against quality of life was mostly inconclusive. Without direct baseline data, the evaluators could not fully separate DERM's effect from that of teledermatology.","category":"aggregate result","unexpected":false,"purpose":"Skin Analytics built DERM to assess photographs of skin lesions and to assign each lesion to a care pathway.","system":"DERM (Deep Ensemble for the Recognition of Malignancy)","model_version":null,"developer":"Skin Analytics","deployer":"Chelsea and Westminster, University Hospitals Bristol and Weston, Ashford and St Peter's and University Hospitals Birmingham NHS foundation trusts, with Birmingham primary care networks","location":"London, Bristol, Surrey and Birmingham, England, United Kingdom","domain":"Healthcare","segment":null,"benefit_type":"Improved Organizational Efficiency","work_activities":["4.A.1.b.1","4.A.2.b.1"],"taxonomy_note":null,"registry_ids":{},"ratings":[{"group":"organizational","depth":2,"reason":"The evaluators found that DERM cut unnecessary referrals and released consultant time, and they estimated that it saves 2,600 dermatologist appointments a year in an average trust."},{"group":"personal","depth":null,"reason":"The evaluators found no evidence of incorrect discharge, but they had too little data to measure the effect of missed cancers or quality of life."}],"ai_role":{"status":"unassessed","statement":"The evaluators measured DERM's pathway sensitivity and modeled its costs, but they could not obtain direct baseline data to compare care with and without it. Staff described a large gain in capacity, which the evaluators could not separate from the effect of teledermatology alone. They also assessed whether best practice in machine learning was followed in developing the DERM algorithm."},"evidence":[{"kind":"evaluation report","citation":"Unity Insights. Evaluation of Deep Ensemble for the Recognition of Malignancy (DERM). Summary of an evaluation commissioned by the Department of Health and Social Care under phase IV of the AI Awards in Health and Social Care. Hosted by NIHR ARC Kent, Surrey and Sussex. 2024.","title":"Evaluation of Deep Ensemble for the Recognition of Malignancy (DERM)","url":"https://www.arc-kss.nihr.ac.uk/resource-library/798-evaluation-of-deep-ensemble-for-the-recognition-of-malignancy-derm/file","date":null,"written_by":"evaluator","quote":"Findings suggest DERM performed accurately and safely across secondary care and community hub models, though the latter requires a larger sample size for validation. Unnecessary referrals were reduced and consultant time released due to DERM implementation."},{"kind":"assessment report","citation":"National Institute for Health and Care Excellence. Artificial intelligence (AI) technologies for assessing and triaging skin lesions referred to the urgent suspected skin cancer pathway: early value assessment. HealthTech guidance HTG746, section 3 (Committee discussion), paragraph 3.11. London: NICE, published 1 May 2025, last updated 17 March 2026.","title":"Artificial intelligence (AI) technologies for assessing and triaging skin lesions referred to the urgent suspected skin cancer pathway: early value assessment (HTG746), section 3, Committee discussion","url":"https://www.nice.org.uk/guidance/htg746/chapter/3-Committee-discussion","date":"2025-05-01","written_by":"independent","quote":"The Unity Insights Skin Analytics evaluation report presented data from 3 NHS sites that used DERM in a post-referral setting (that is, after primary care referral to a dermatology service). This suggested that using automated DERM would lower the number of non-cancer lesions flagged for further review compared with using teledermatology alone."},{"kind":"assessment report","citation":"Walton M, Llewellyn A, Uphoff E, Lord J, Harden M, Hodgson R, Simmonds M. Artificial Intelligence technologies for assessing skin lesions for referral on the urgent suspected cancer pathway to detect benign lesions and reduce secondary care specialist appointments: early value assessment. Health Technology Assessment 2026;30(10):1-96. doi:10.3310/GJMS0317. PubMed 41660864.","title":"Artificial Intelligence technologies for assessing skin lesions for referral on the urgent suspected cancer pathway to detect benign lesions and reduce secondary care specialist appointments: early value assessment","url":"https://doi.org/10.3310/gjms0317","date":"2026-02-01","written_by":"independent","quote":"DERM might lead to around half of all patients being discharged without assessment by a dermatologist, but a small number of malignant lesions would be missed."},{"kind":"peer-reviewed study","citation":"Anderson ADG, Lo SN, Guitera P. Commentary: Real-world post-deployment performance of a novel machine learning-based digital health technology for skin lesion assessment and suggestions for post-market surveillance. Frontiers in Medicine 2024;11:1345659. doi:10.3389/fmed.2024.1345659. A peer-reviewed commentary that re-analyses the published data.","title":"Commentary: Real-world post-deployment performance of a novel machine learning-based digital health technology for skin lesion assessment and suggestions for post-market surveillance","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC10877043/","date":"2024-02-06","written_by":"independent","quote":"Unfortunately, the lack of transparency and failure to apply adequate scientific rigor to the current description of post-market implementation of the DERM algorithm do not support the use of AI alone as a screening tool. In particular, the reported sensitivity and missed cancer rates are potentially compromised by significant selection and attrition bias.","supports_benefit":false}],"replication":{"produced_by":["Referrers sent photographs of lesions through existing teledermatology services.","DERM assigned each lesion to a pathway, and a consultant reviewed each case that it classified as low risk."],"must_hold":["Referrals are appropriate and carry high-quality photographs, with a supported and trained workforce.","Enough dermatologists are available for the second read, given national workforce shortages."],"echoes":[]},"paired_incident":null,"headline":"*DERM* triages suspected skin cancer safely at four NHS sites","metric":"2,600 dermatologist appointments a year","comparator":{"against":"Traditional teledermatology without DERM and the non-digital face-to-face pathway, in the evaluators' cost-benefit model","before":null,"after":"42% less time spent reviewing images with DERM than traditional teledermatology","basis":"assumed","quote":"The cost-benefit analysis modelled the impact of teledermatology with and without DERM on skin cancer referrals and compared each against a non-digital face-to-face pathway. All instances highlighted savings for the NHS; DERM saved more dermatologist time than traditional teledermatology.","url":"https://www.arc-kss.nihr.ac.uk/resource-library/798-evaluation-of-deep-ensemble-for-the-recognition-of-malignancy-derm/file"},"later":null,"value":{"amount":"£1.7 returned for every £1 spent","what":"Return on each pound spent on DERM in the secondary care model, from the evaluators' cost-benefit analysis","who":"The NHS","estimate":true,"quote":"£1.7 returned for every £1 spent.","url":"https://www.arc-kss.nihr.ac.uk/resource-library/798-evaluation-of-deep-ensemble-for-the-recognition-of-malignancy-derm/file"},"developer_country":"United Kingdom","deployer_country":"United Kingdom","unrated_because":null,"brief_notes":null,"brief":null,"revisions":[{"date":"2026-10-31","change":"Added the record from the report of Unity Insights.","source":"https://www.arc-kss.nihr.ac.uk/resource-library/798-evaluation-of-deep-ensemble-for-the-recognition-of-malignancy-derm/file","editor":"AIBD"},{"date":"2026-10-31","change":"Rewrote the prose in plain sentences, with its facts, figures and quotations unchanged.","source":null,"editor":"AIBD"},{"date":"2026-10-31","change":"Added an independent assessment report that supports the benefit.","source":null,"editor":"AIBD"},{"date":"2026-10-31","change":"Added an independent peer-reviewed study that questions the benefit.","source":null,"editor":"AIBD"}],"confidence":"found","shade":"high","claim_status":"Found by an assessment body","claim_basis":"Sources: two independent assessment reports, one independent peer-reviewed study and the evaluators' evaluation report. For the benefit itself, the cited sources include no independent study or report.","case_jurisdictions":["United Kingdom"],"profile":{"case_id":"AIBD-0743","catalog_version":"1.0.0","source_fingerprint":"93e3a588177836853b2ef271b577f7a700743b16cd905cf3086cff0a41c7550a","collection_refs":[{"collection_id":"aibd","version":"2026-10-31"},{"collection_id":"querist","version":"2026-10-25"}],"source_ids":["AIBD-0743-source-0001","AIBD-0743-source-0002","AIBD-0743-source-0003","AIBD-0743-source-0004"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"monetary observations":{"state":"value_present","value":{"kind":"text","text":"An existing source-stated monetary observation is present in the original value object."},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["value"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial","record_ids":["AIBD-0743-legacy-value"]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"Skin Analytics","deployer":"Chelsea and Westminster, University Hospitals Bristol and Weston, Ashford and St Peter's and University Hospitals Birmingham NHS foundation trusts, with Birmingham primary care networks"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer","deployer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"aggregate result","ai_role":{"status":"unassessed","statement":"The evaluators measured DERM's pathway sensitivity and modeled its costs, but they could not obtain direct baseline data to compare care with and without it. Staff described a large gain in capacity, which the evaluators could not separate from the effect of teledermatology alone. They also assessed whether best practice in machine learning was followed in developing the DERM algorithm."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"The evaluators found that DERM cut unnecessary referrals and released consultant time, and they estimated that it saves 2,600 dermatologist appointments a year in an average trust."},{"group":"personal","reason":"The evaluators found no evidence of incorrect discharge, but they had too little data to measure the effect of missed cancers or quality of life."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"high","evidence_word":"found","claim_status":"Found by an assessment body","basis":"Sources: two independent assessment reports, one independent peer-reviewed study and the evaluators' evaluation report. For the benefit itself, the cited sources include no independent study or report.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"The evaluators found that DERM cut unnecessary referrals and released consultant time, and they estimated that it saves 2,600 dermatologist appointments a year in an average trust."},{"group":"personal","reason":"The evaluators found no evidence of incorrect discharge, but they had too little data to measure the effect of missed cancers or quality of life."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"The evaluators found that DERM cut unnecessary referrals and released consultant time, and they estimated that it saves 2,600 dermatologist appointments a year in an average trust."},{"group":"personal","reason":"The evaluators found no evidence of incorrect discharge, but they had too little data to measure the effect of missed cancers or quality of life."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_links[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"The evaluator's account. Sources: the evaluators' evaluation report. The cited evidence includes no independent source and no finding by an external body.","evidence_word":"claimed","claim_status":"The evaluator's account"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-0743-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The evaluator's account. Sources: the evaluators' evaluation report. The cited evidence includes no independent source and no finding by an external body.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"pilot","answer":"The system was used in a pilot. Skin Analytics implemented DERM at six NHS sites under the AI Awards, and the evaluation covered four of them.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-0743-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was used in a pilot. Skin Analytics implemented DERM at six NHS sites under the AI Awards, and the evaluation covered four of them.","inferred":false}},"economics[].amount":{"state":"value_present","value":{"kind":"record_fields","fields":{"value":{"amount":"£1.7 returned for every £1 spent","what":"Return on each pound spent on DERM in the secondary care model, from the evaluators' cost-benefit analysis","who":"The NHS","estimate":true,"quote":"£1.7 returned for every £1 spent.","url":"https://www.arc-kss.nihr.ac.uk/resource-library/798-evaluation-of-deep-ensemble-for-the-recognition-of-malignancy-derm/file"}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["value"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"economics[].costs[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"economics[].counterfactual":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"economics[].currency_price_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"economics[].gross_net":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"economics[].measure_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"value":{"amount":"£1.7 returned for every £1 spent","what":"Return on each pound spent on DERM in the secondary care model, from the evaluators' cost-benefit analysis","who":"The NHS","estimate":true,"quote":"£1.7 returned for every £1 spent.","url":"https://www.arc-kss.nihr.ac.uk/resource-library/798-evaluation-of-deep-ensemble-for-the-recognition-of-malignancy-derm/file"}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["value"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"economics[].period_unit":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"economics[].realization":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"economics[].recipient_payer":{"state":"value_present","value":{"kind":"record_fields","fields":{"value":{"amount":"£1.7 returned for every £1 spent","what":"Return on each pound spent on DERM in the secondary care model, from the evaluators' cost-benefit analysis","who":"The NHS","estimate":true,"quote":"£1.7 returned for every £1 spent.","url":"https://www.arc-kss.nihr.ac.uk/resource-library/798-evaluation-of-deep-ensemble-for-the-recognition-of-malignancy-derm/file"}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["value"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"economics[].return_payback":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"economics[].valuation_method":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"evaluations[].ai_uplift":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].comparator":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].design":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].effect":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].exposure":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].id":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].measurement_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].metric":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].reproduction_materials":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].setting_sample":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].uncertainty_limitations":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"external_references[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"follow_up[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-0743"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Improved Organizational Efficiency"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"Skin Analytics implemented DERM, a UKCA class IIa AI device, at six NHS sites under phase IV of the AI Awards in Health and Social Care. The Department of Health and Social Care commissioned an evaluation of four of them, which Unity Insights summarized. In secondary care, DERM assessed 5,678 lesions from 4,159 referrals. The evaluators report pathway sensitivity for melanoma above 90% at every site and a secondary care sensitivity of 97.4%. A consultant dermatologist reviewed each case that DERM classified as low risk. This second read overturned 36% of potential discharges, and trust dermatologists then discharged 41% of those. Consultants found seven cancers among the 754 overturned cases, most of them at low risk. In a model of costs and benefits, the evaluators estimated that DERM saves 2,600 dermatologist appointments a year in an average trust. They estimated that it returns £1.7 for every £1 spent in secondary care. Their analysis of costs against quality of life was mostly inconclusive. Without direct baseline data, the evaluators could not fully separate DERM's effect from that of teledermatology.","benefit_type":"Improved Organizational Efficiency"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"organizational","depth":2,"reason":"The evaluators found that DERM cut unnecessary referrals and released consultant time, and they estimated that it saves 2,600 dermatologist appointments a year in an average trust."},{"group":"personal","reason":"The evaluators found no evidence of incorrect discharge, but they had too little data to measure the effect of missed cancers or quality of life."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"reported"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"Skin Analytics built DERM to assess photographs of skin lesions and to assign each lesion to a care pathway."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"London, Bristol, Surrey and Birmingham, England, United Kingdom","developer_country":"United Kingdom","deployer_country":"United Kingdom"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","developer_country","deployer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"Skin Analytics built DERM to assess photographs of skin lesions and to assign each lesion to a care pathway."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2024-01-01","date_basis":"source publication","date_precision":"year","revisions":[{"date":"2026-10-31","change":"Added the record from the report of Unity Insights.","source":"https://www.arc-kss.nihr.ac.uk/resource-library/798-evaluation-of-deep-ensemble-for-the-recognition-of-malignancy-derm/file","editor":"AIBD"},{"date":"2026-10-31","change":"Rewrote the prose in plain sentences, with its facts, figures and quotations unchanged.","editor":"AIBD"},{"date":"2026-10-31","change":"Added an independent assessment report that supports the benefit.","editor":"AIBD"},{"date":"2026-10-31","change":"Added an independent peer-reviewed study that questions the benefit.","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["Referrers sent photographs of lesions through existing teledermatology services.","DERM assigned each lesion to a pathway, and a consultant reviewed each case that it classified as low risk."],"must_hold":["Referrals are appropriate and carry high-quality photographs, with a supported and trained workforce.","Enough dermatologists are available for the second read, given national workforce shortages."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-10-31","change":"Added the record from the report of Unity Insights.","source":"https://www.arc-kss.nihr.ac.uk/resource-library/798-evaluation-of-deep-ensemble-for-the-recognition-of-malignancy-derm/file","editor":"AIBD"},{"date":"2026-10-31","change":"Rewrote the prose in plain sentences, with its facts, figures and quotations unchanged.","editor":"AIBD"},{"date":"2026-10-31","change":"Added an independent assessment report that supports the benefit.","editor":"AIBD"},{"date":"2026-10-31","change":"Added an independent peer-reviewed study that questions the benefit.","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Healthcare"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"evaluation report","citation":"Unity Insights. Evaluation of Deep Ensemble for the Recognition of Malignancy (DERM). Summary of an evaluation commissioned by the Department of Health and Social Care under phase IV of the AI Awards in Health and Social Care. Hosted by NIHR ARC Kent, Surrey and Sussex. 2024.","title":"Evaluation of Deep Ensemble for the Recognition of Malignancy (DERM)","url":"https://www.arc-kss.nihr.ac.uk/resource-library/798-evaluation-of-deep-ensemble-for-the-recognition-of-malignancy-derm/file","written_by":"evaluator","quote":"Findings suggest DERM performed accurately and safely across secondary care and community hub models, though the latter requires a larger sample size for validation. Unnecessary referrals were reduced and consultant time released due to DERM implementation."},{"kind":"assessment report","citation":"National Institute for Health and Care Excellence. Artificial intelligence (AI) technologies for assessing and triaging skin lesions referred to the urgent suspected skin cancer pathway: early value assessment. HealthTech guidance HTG746, section 3 (Committee discussion), paragraph 3.11. London: NICE, published 1 May 2025, last updated 17 March 2026.","title":"Artificial intelligence (AI) technologies for assessing and triaging skin lesions referred to the urgent suspected skin cancer pathway: early value assessment (HTG746), section 3, Committee discussion","url":"https://www.nice.org.uk/guidance/htg746/chapter/3-Committee-discussion","date":"2025-05-01","written_by":"independent","quote":"The Unity Insights Skin Analytics evaluation report presented data from 3 NHS sites that used DERM in a post-referral setting (that is, after primary care referral to a dermatology service). This suggested that using automated DERM would lower the number of non-cancer lesions flagged for further review compared with using teledermatology alone."},{"kind":"assessment report","citation":"Walton M, Llewellyn A, Uphoff E, Lord J, Harden M, Hodgson R, Simmonds M. Artificial Intelligence technologies for assessing skin lesions for referral on the urgent suspected cancer pathway to detect benign lesions and reduce secondary care specialist appointments: early value assessment. Health Technology Assessment 2026;30(10):1-96. doi:10.3310/GJMS0317. PubMed 41660864.","title":"Artificial Intelligence technologies for assessing skin lesions for referral on the urgent suspected cancer pathway to detect benign lesions and reduce secondary care specialist appointments: early value assessment","url":"https://doi.org/10.3310/gjms0317","date":"2026-02-01","written_by":"independent","quote":"DERM might lead to around half of all patients being discharged without assessment by a dermatologist, but a small number of malignant lesions would be missed."},{"kind":"peer-reviewed study","citation":"Anderson ADG, Lo SN, Guitera P. Commentary: Real-world post-deployment performance of a novel machine learning-based digital health technology for skin lesion assessment and suggestions for post-market surveillance. Frontiers in Medicine 2024;11:1345659. doi:10.3389/fmed.2024.1345659. A peer-reviewed commentary that re-analyses the published data.","title":"Commentary: Real-world post-deployment performance of a novel machine learning-based digital health technology for skin lesion assessment and suggestions for post-market surveillance","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC10877043/","date":"2024-02-06","written_by":"independent","quote":"Unfortunately, the lack of transparency and failure to apply adequate scientific rigor to the current description of post-market implementation of the DERM algorithm do not support the use of AI alone as a screening tool. In particular, the reported sensitivity and missed cancer rates are potentially compromised by significant selection and attrition bias.","supports_benefit":false}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-0743-source-0001","AIBD-0743-source-0002","AIBD-0743-source-0003","AIBD-0743-source-0004"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"evaluation report","citation":"Unity Insights. Evaluation of Deep Ensemble for the Recognition of Malignancy (DERM). Summary of an evaluation commissioned by the Department of Health and Social Care under phase IV of the AI Awards in Health and Social Care. Hosted by NIHR ARC Kent, Surrey and Sussex. 2024.","title":"Evaluation of Deep Ensemble for the Recognition of Malignancy (DERM)","url":"https://www.arc-kss.nihr.ac.uk/resource-library/798-evaluation-of-deep-ensemble-for-the-recognition-of-malignancy-derm/file","written_by":"evaluator","quote":"Findings suggest DERM performed accurately and safely across secondary care and community hub models, though the latter requires a larger sample size for validation. Unnecessary referrals were reduced and consultant time released due to DERM implementation."},{"kind":"assessment report","citation":"National Institute for Health and Care Excellence. Artificial intelligence (AI) technologies for assessing and triaging skin lesions referred to the urgent suspected skin cancer pathway: early value assessment. HealthTech guidance HTG746, section 3 (Committee discussion), paragraph 3.11. London: NICE, published 1 May 2025, last updated 17 March 2026.","title":"Artificial intelligence (AI) technologies for assessing and triaging skin lesions referred to the urgent suspected skin cancer pathway: early value assessment (HTG746), section 3, Committee discussion","url":"https://www.nice.org.uk/guidance/htg746/chapter/3-Committee-discussion","date":"2025-05-01","written_by":"independent","quote":"The Unity Insights Skin Analytics evaluation report presented data from 3 NHS sites that used DERM in a post-referral setting (that is, after primary care referral to a dermatology service). This suggested that using automated DERM would lower the number of non-cancer lesions flagged for further review compared with using teledermatology alone."},{"kind":"assessment report","citation":"Walton M, Llewellyn A, Uphoff E, Lord J, Harden M, Hodgson R, Simmonds M. Artificial Intelligence technologies for assessing skin lesions for referral on the urgent suspected cancer pathway to detect benign lesions and reduce secondary care specialist appointments: early value assessment. Health Technology Assessment 2026;30(10):1-96. doi:10.3310/GJMS0317. PubMed 41660864.","title":"Artificial Intelligence technologies for assessing skin lesions for referral on the urgent suspected cancer pathway to detect benign lesions and reduce secondary care specialist appointments: early value assessment","url":"https://doi.org/10.3310/gjms0317","date":"2026-02-01","written_by":"independent","quote":"DERM might lead to around half of all patients being discharged without assessment by a dermatologist, but a small number of malignant lesions would be missed."},{"kind":"peer-reviewed study","citation":"Anderson ADG, Lo SN, Guitera P. Commentary: Real-world post-deployment performance of a novel machine learning-based digital health technology for skin lesion assessment and suggestions for post-market surveillance. Frontiers in Medicine 2024;11:1345659. doi:10.3389/fmed.2024.1345659. A peer-reviewed commentary that re-analyses the published data.","title":"Commentary: Real-world post-deployment performance of a novel machine learning-based digital health technology for skin lesion assessment and suggestions for post-market surveillance","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC10877043/","date":"2024-02-06","written_by":"independent","quote":"Unfortunately, the lack of transparency and failure to apply adequate scientific rigor to the current description of post-market implementation of the DERM algorithm do not support the use of AI alone as a screening tool. In particular, the reported sensitivity and missed cancer rates are potentially compromised by significant selection and attrition bias.","supports_benefit":false}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"Evaluators of DERM at four NHS sites found that it triaged suspected skin cancer accurately and safely, cut unnecessary referrals and freed consultant time. Its gains in quality of life were unclear.","account":"Skin Analytics implemented DERM, a UKCA class IIa AI device, at six NHS sites under phase IV of the AI Awards in Health and Social Care. The Department of Health and Social Care commissioned an evaluation of four of them, which Unity Insights summarized. In secondary care, DERM assessed 5,678 lesions from 4,159 referrals. The evaluators report pathway sensitivity for melanoma above 90% at every site and a secondary care sensitivity of 97.4%. A consultant dermatologist reviewed each case that DERM classified as low risk. This second read overturned 36% of potential discharges, and trust dermatologists then discharged 41% of those. Consultants found seven cancers among the 754 overturned cases, most of them at low risk. In a model of costs and benefits, the evaluators estimated that DERM saves 2,600 dermatologist appointments a year in an average trust. They estimated that it returns £1.7 for every £1 spent in secondary care. Their analysis of costs against quality of life was mostly inconclusive. Without direct baseline data, the evaluators could not fully separate DERM's effect from that of teledermatology."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"DERM (Deep Ensemble for the Recognition of Malignancy)"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"Evaluators for the Department of Health and Social Care find that DERM safely sorted referrals for suspected skin cancer at four NHS sites","headline":"*DERM* triages suspected skin cancer safely at four NHS sites"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["6dab0093697616cd245a01cbd5a45651bdadf5cee81d607e468a792615515cd8"],"role":"SHA-256 of the complete public record and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public record, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"Skin Analytics implemented DERM, a UKCA class IIa AI device, at six NHS sites under phase IV of the AI Awards in Health and Social Care. The Department of Health and Social Care commissioned an evaluation of four of them, which Unity Insights summarized. In secondary care, DERM assessed 5,678 lesions from 4,159 referrals. The evaluators report pathway sensitivity for melanoma above 90% at every site and a secondary care sensitivity of 97.4%. A consultant dermatologist reviewed each case that DERM classified as low risk. This second read overturned 36% of potential discharges, and trust dermatologists then discharged 41% of those. Consultants found seven cancers among the 754 overturned cases, most of them at low risk. In a model of costs and benefits, the evaluators estimated that DERM saves 2,600 dermatologist appointments a year in an average trust. They estimated that it returns £1.7 for every £1 spent in secondary care. Their analysis of costs against quality of life was mostly inconclusive. Without direct baseline data, the evaluators could not fully separate DERM's effect from that of teledermatology.","replication":{"produced_by":["Referrers sent photographs of lesions through existing teledermatology services.","DERM assigned each lesion to a pathway, and a consultant reviewed each case that it classified as low risk."],"must_hold":["Referrals are appropriate and carry high-quality photographs, with a supported and trained workforce.","Enough dermatologists are available for the second read, given national workforce shortages."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":31,"reviewed":2,"reviewed_complete":2,"reviewed_partial":0,"record_mapped":29,"review_not_assessed":60,"not_assessed":0,"complete":false,"by_state":{"value_present":31,"not_reported":0,"unresolved":60,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"6ba28c72d814700b11be241557b3943604f9a123d976ea2cc0048b470c93b5cc"}