{"id":"AIBD-1147","title":"Romero-Fresco and Fresno find that the Miracle Channel's automatic Lexi Local captions met Canada's accuracy standard in all but one of 24 samples","summary":"Researchers who analyzed the station's reports to the CRTC found that its automatic captions reached 98% in 8/31 samples in 2019-2020, while only 1/24 fell below 98% in 2020-2021.","date_observed":"2023-12-13","date_basis":"source publication","date_precision":"day","benefit_status":"reported","source_collection":null,"source_entry":null,"account":"The Miracle Channel Association runs CJIL-DT, a television station in Lethbridge, Alberta. It captions its live news with EEG's automatic speech-to-text software, with no human captioner. From 1 March 2020 the CRTC required live English-language captions to score at least 98 under the NER model. In 2019 a group of independent stations that included the Miracle Channel had told the CRTC that voice recognition captioning could struggle to meet it. Pablo Romero-Fresco, who developed the model, and Nazaret Fresno analyzed the broadcasters' reports to the CRTC. They report that EEG's cloud version reached 98% in 8/31 samples in 2019-2020. The station then moved to Lexi Local, on a local server, and a staff member compiled glossaries with phonetic spellings. In 2020-2021 only 1/24 samples fell below 98%. The station chose its samples, and an evaluator that it engaged scored them. Its report for 2023-2024 lists scores from 98.49 to 99.38. Across the broadcasters studied, 60% of automatic samples missed the threshold. The authors note that unaided automatic captions still fell short of it.","category":"aggregate result","unexpected":false,"purpose":"EEG Enterprises built Lexi Local, an on-premises version of its Lexi captioning service, to produce live captions automatically from speech.","system":"EEG Lexi Local on-premises automatic speech-to-text captioning (after the cloud-based Lexi service)","model_version":null,"developer":"EEG Enterprises","deployer":"The Miracle Channel Association (CJIL-DT)","location":"Lethbridge, Alberta, Canada","domain":"Broadcasting / live closed captions for deaf and hard-of-hearing viewers","segment":null,"benefit_type":"Improved Accessibility","work_activities":["4.A.3.b.6"],"taxonomy_note":null,"registry_ids":{},"ratings":[{"group":"personal","depth":2,"reason":"The study reports that the station's automatic captions fell below 98% in only 1/24 samples in 2020-2021, after reaching 98% in only 8/31 samples in 2019-2020."}],"ai_role":{"status":"unassessed","statement":"The NER scores measure captions that EEG's software produced with no captioner at broadcast time, but no document compares the same programs captioned by people. The move to a local server and the staff glossaries contributed to the gains. The broadcaster wrote that corrections it uploaded to the cloud service helped improve the AI and its dictionary. The study gives no technical description of the software."},"evidence":[{"kind":"peer-reviewed study","citation":"Romero-Fresco P, Fresno N. The accuracy of automatic and human live captions in English. Linguistica Antverpiensia, New Series: Themes in Translation Studies 22 (2023): 114-133, section 3.4 (case study 4: live TV captions in Canada). Published 13 December 2023. doi:10.52034/lans-tts.v22i.774.","title":"The accuracy of automatic and human live captions in English","url":"https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/download/774/669","date":"2023-12-13","written_by":"independent","quote":"for the first time, broadcasters such as Jim Pattison Broadcast Group and The Miracle Channel Associations consistently reach accuracy rates above the required threshold in 2020-2021. Impressively, of the 55 samples analysed by these two broadcasters in 2020-2021, 52 reached 98%."},{"kind":"peer-reviewed study","citation":"Romero-Fresco P, Fresno N. The accuracy of automatic and human live captions in English. Linguistica Antverpiensia, New Series: Themes in Translation Studies 22 (2023): 114-133, section 3.4 (case study 4: live TV captions in Canada). Published 13 December 2023. doi:10.52034/lans-tts.v22i.774.","title":"The accuracy of automatic and human live captions in English","url":"https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/download/774/669","date":"2023-12-13","written_by":"independent","quote":"In 2019-2020, their cloud-based version of the automatic captioning software EEG reached the 98% accuracy rate in 8/31 samples and 10/31 scored below 97.5%. For the 2020-2021 round, they decided to use the local server of the software (called Lexi Local) and to have a dedicated member of the staff compile glossaries with their phonetic spellings to improve the accuracy of the captions. This enabled them to reach the NER threshold consistently, with only 1/24 samples below par but pretty close to the minimum accuracy rate (97.92%)."},{"kind":"peer-reviewed study","citation":"Romero-Fresco P, Fresno N. The accuracy of automatic and human live captions in English. Linguistica Antverpiensia, New Series: Themes in Translation Studies 22 (2023): 114-133, section 3.4 (case study 4: live TV captions in Canada). Published 13 December 2023. doi:10.52034/lans-tts.v22i.774.","title":"The accuracy of automatic and human live captions in English","url":"https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/download/774/669","date":"2023-12-13","written_by":"independent","quote":"They are also more accurate than automatic captions, which feature more samples below 98% accuracy and fewer samples exceeding a 99% accuracy rate (60% of the automatic samples did not reach the minimum threshold and only 1% featured an excellent accuracy).","supports_benefit":false},{"kind":"evaluation report","citation":"The Miracle Channel Association (CJIL-DT). NER98 Accuracy Standards Summary for Miracle Channel (CJIL-DT) 2020-21 and Live-Captioning NER Score Report for 2020-2021, with scores by a certified NER evaluator that the broadcaster engaged. Annual report to the CRTC under Broadcasting Regulatory Policy CRTC 2019-308. Archived copy of https://crtc.gc.ca/eng/BCASTING/ann_rep/miracle_cc2021.pdf.","title":"NER98 Accuracy Standards Summary for Miracle Channel (CJIL-DT) 2020-21","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/BCASTING/ann_rep/miracle_cc2021.pdf","date":null,"written_by":"evaluator","quote":"We are maintaining consistent NER98 accuracy through the use of an on-premise speech-to-text service called Lexi Local, from EEG Enterprises."},{"kind":"report","citation":"The Miracle Channel Association (CJIL-DT). Summary of The Miracle Channel Association (CJIL-DT) Efforts to Comply with BRP 2019-308 and NER98 Accuracy Standards, with the Live-Captioning NER Score Report for 2019-2020. Annual report to the CRTC, 30 November 2020. Archived copy of https://crtc.gc.ca/eng/BCASTING/ann_rep/miracle_cc2020.pdf.","title":"Summary of The Miracle Channel Association (CJIL-DT) Efforts to Comply with BRP 2019-308 and NER98 Accuracy Standards","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/BCASTING/ann_rep/miracle_cc2020.pdf","date":"2020-11-30","written_by":"deployer","quote":"The review process involved reviewing the video segment that was captioned and fixing all the mistakes that were presented in the exported log file and then uploading that back into the cloud service to help improve the AI and its dictionary in the hope of improving the accuracy moving forward.","supports_benefit":false},{"kind":"regulatory document","citation":"Canadian Radio-television and Telecommunications Commission. Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming. Ottawa, 30 August 2019. Archived copy of https://crtc.gc.ca/eng/archive/2019/2019-308.htm. Appendix 1, Quality Standard 3.","title":"Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/archive/2019/2019-308.htm","date":"2019-08-30","written_by":"independent","quote":"Beginning 1 March 2020, broadcasters must reach an accuracy rate of at least 98 for English-language closed captioning of live television programming","supports_benefit":false},{"kind":"regulatory document","citation":"Canadian Radio-television and Telecommunications Commission. Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming. Ottawa, 30 August 2019. Archived copy of https://crtc.gc.ca/eng/archive/2019/2019-308.htm. Section on requested flexibility for independent broadcasters, relaying the joint intervention of the English-Language Local Independent Television Stations Group (E-ILTS).","title":"Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/archive/2019/2019-308.htm","date":"2019-08-30","written_by":"independent","quote":"Further, E-ILTS acknowledged that voice recognition software, which is sometimes used by independent television stations and allows for costs savings, is improving year by year, but noted that based on preliminary testing, voice recognition-based captioning could have difficulty meeting the proposed NER 98 accuracy standard in the short term.","supports_benefit":false},{"kind":"regulatory document","citation":"Canadian Radio-television and Telecommunications Commission. Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming. Ottawa, 30 August 2019. Archived copy of https://crtc.gc.ca/eng/archive/2019/2019-308.htm. Footnote 6, the members of E-ILTS.","title":"Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/archive/2019/2019-308.htm","date":"2019-08-30","written_by":"independent","quote":"Jim Pattison Broadcast Group Limited Partnership, operating in Kamloops and Prince George, British Columbia, and Medicine Hat, Alberta; Stingray Digital Group Inc. (now Stingray Group Inc.) in Lloydminster, Alberta; Thunder Bay Electronics Limited in Thunder Bay, Ontario; The Miracle Channel Association in Lethbridge, Alberta;","supports_benefit":false}],"replication":{"produced_by":["EEG's speech-to-text service captioned the station's live news with no human captioner.","In 2020-2021 the station moved from the cloud version of the software to Lexi Local, on a local server.","A staff member compiled glossaries of words with their phonetic spellings.","Staff adjusted the glossary from the errors that the certified evaluator found."],"must_hold":["Live English-language captions must score at least 98 under the Canadian NER guidelines, as calculated by certified evaluators.","Two programs with live content must be scored each month.","Automatic captions need effective human help, such as glossaries, to reach the threshold regularly."],"echoes":[]},"paired_incident":null,"headline":"*Lexi Local* captions meet Canada's live accuracy standard in Lethbridge","metric":"only 1/24 samples fell below 98%","comparator":{"against":"The station's automatic captions in 2019-2020, from the cloud version of EEG's software","before":"reached the 98% accuracy rate in 8/31 samples","after":"only 1/24 samples below par","basis":"measured","quote":"In 2019-2020, their cloud-based version of the automatic captioning software EEG reached the 98% accuracy rate in 8/31 samples and 10/31 scored below 97.5%. For the 2020-2021 round, they decided to use the local server of the software (called Lexi Local) and to have a dedicated member of the staff compile glossaries with their phonetic spellings to improve the accuracy of the captions. This enabled them to reach the NER threshold consistently, with only 1/24 samples below par but pretty close to the minimum accuracy rate (97.92%).","url":"https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/download/774/669"},"later":{"status":"persisted","date":null,"note":"The broadcaster's report for 2023-2024 lists NER scores from 98.49 to 99.38 for its live news, captioned with Lexi Local.","quote":"NER98 Accuracy Standards Summary for Miracle Channel (CJIL-DT) 2023-2024 We are maintaining consistent NER98 accuracy through the use of an on-premise speech-to-text service called Lexi Local, from EEG Enterprises.","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/BCASTING/ann_rep/miracle_cc2024.pdf"},"value":null,"developer_country":"United States","deployer_country":"Canada","unrated_because":null,"brief_notes":null,"brief":null,"revisions":[{"date":"2026-11-01","change":"Added the record from the finding of Pablo Romero-Fresco and Nazaret Fresno.","source":"https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/download/774/669","editor":"AIBD"}],"confidence":"studied","shade":"high","claim_status":"Independently studied","claim_basis":"Sources: one independent regulatory document, one independent peer-reviewed study, the deployer's report and the evaluators' evaluation report. For the benefit itself, the cited sources include no finding by an external body.","case_jurisdictions":["Canada"],"profile":{"case_id":"AIBD-1147","catalog_version":"1.0.0","source_fingerprint":"f8c6e93e35a70ce3c025b5d53e6def72abb8c58b20347e3fb407b2d302897745","collection_refs":[{"collection_id":"aibd","version":"2026-10-31"},{"collection_id":"querist","version":"2026-10-25"},{"collection_id":"viator","version":"2026-10-25"}],"source_ids":["AIBD-1147-source-0001","AIBD-1147-source-0002","AIBD-1147-source-0003","AIBD-1147-source-0004","AIBD-1147-source-0005","AIBD-1147-source-0006","AIBD-1147-source-0007","AIBD-1147-source-0008"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"monetary observations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"EEG Enterprises","deployer":"The Miracle Channel Association (CJIL-DT)"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer","deployer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"aggregate result","ai_role":{"status":"unassessed","statement":"The NER scores measure captions that EEG's software produced with no captioner at broadcast time, but no document compares the same programs captioned by people. The move to a local server and the staff glossaries contributed to the gains. The broadcaster wrote that corrections it uploaded to the cloud service helped improve the AI and its dictionary. The study gives no technical description of the software."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"The study reports that the station's automatic captions fell below 98% in only 1/24 samples in 2020-2021, after reaching 98% in only 8/31 samples in 2019-2020."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"high","evidence_word":"studied","claim_status":"Independently studied","basis":"Sources: one independent regulatory document, one independent peer-reviewed study, the deployer's report and the evaluators' evaluation report. For the benefit itself, the cited sources include no finding by an external body.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"The study reports that the station's automatic captions fell below 98% in only 1/24 samples in 2020-2021, after reaching 98% in only 8/31 samples in 2019-2020."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"The study reports that the station's automatic captions fell below 98% in only 1/24 samples in 2020-2021, after reaching 98% in only 8/31 samples in 2019-2020."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_links[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"Independently studied. Sources: one independent regulatory document, one independent peer-reviewed study, the deployer's report and the evaluators' evaluation report. For the benefit itself, the cited sources include no finding by an external body.","evidence_word":"studied","claim_status":"Independently studied"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-1147-source-0001","AIBD-1147-source-0002","AIBD-1147-source-0003","AIBD-1147-source-0004","AIBD-1147-source-0005","AIBD-1147-source-0006","AIBD-1147-source-0007","AIBD-1147-source-0008"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"Independently studied. Sources: one independent regulatory document, one independent peer-reviewed study, the deployer's report and the evaluators' evaluation report. For the benefit itself, the cited sources include no finding by an external body.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"routine use","answer":"The system was in routine use. The broadcaster reported that it still captioned its live news with Lexi Local in 2023-2024.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-1147-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was in routine use. The broadcaster reported that it still captioned its live news with Lexi Local in 2023-2024.","inferred":false}},"economics[].amount":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].costs[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].counterfactual":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].currency_price_basis":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].gross_net":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].measure_type":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].period_unit":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].realization":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].recipient_payer":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].return_payback":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].valuation_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"evaluations[].ai_uplift":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].comparator":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].design":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].effect":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].exposure":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].id":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].measurement_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].metric":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].reproduction_materials":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].setting_sample":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].uncertainty_limitations":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"external_references[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"follow_up[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"later":{"status":"persisted","note":"The broadcaster's report for 2023-2024 lists NER scores from 98.49 to 99.38 for its live news, captioned with Lexi Local.","quote":"NER98 Accuracy Standards Summary for Miracle Channel (CJIL-DT) 2023-2024 We are maintaining consistent NER98 accuracy through the use of an on-premise speech-to-text service called Lexi Local, from EEG Enterprises.","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/BCASTING/ann_rep/miracle_cc2024.pdf"}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["later"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-1147"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Improved Accessibility"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"The Miracle Channel Association runs CJIL-DT, a television station in Lethbridge, Alberta. It captions its live news with EEG's automatic speech-to-text software, with no human captioner. From 1 March 2020 the CRTC required live English-language captions to score at least 98 under the NER model. In 2019 a group of independent stations that included the Miracle Channel had told the CRTC that voice recognition captioning could struggle to meet it. Pablo Romero-Fresco, who developed the model, and Nazaret Fresno analyzed the broadcasters' reports to the CRTC. They report that EEG's cloud version reached 98% in 8/31 samples in 2019-2020. The station then moved to Lexi Local, on a local server, and a staff member compiled glossaries with phonetic spellings. In 2020-2021 only 1/24 samples fell below 98%. The station chose its samples, and an evaluator that it engaged scored them. Its report for 2023-2024 lists scores from 98.49 to 99.38. Across the broadcasters studied, 60% of automatic samples missed the threshold. The authors note that unaided automatic captions still fell short of it.","benefit_type":"Improved Accessibility"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":2,"reason":"The study reports that the station's automatic captions fell below 98% in only 1/24 samples in 2020-2021, after reaching 98% in only 8/31 samples in 2019-2020."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"reported"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"EEG Enterprises built Lexi Local, an on-premises version of its Lexi captioning service, to produce live captions automatically from speech."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"Lethbridge, Alberta, Canada","developer_country":"United States","deployer_country":"Canada"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","developer_country","deployer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"EEG Enterprises built Lexi Local, an on-premises version of its Lexi captioning service, to produce live captions automatically from speech."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2023-12-13","date_basis":"source publication","date_precision":"day","revisions":[{"date":"2026-11-01","change":"Added the record from the finding of Pablo Romero-Fresco and Nazaret Fresno.","source":"https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/download/774/669","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["EEG's speech-to-text service captioned the station's live news with no human captioner.","In 2020-2021 the station moved from the cloud version of the software to Lexi Local, on a local server.","A staff member compiled glossaries of words with their phonetic spellings.","Staff adjusted the glossary from the errors that the certified evaluator found."],"must_hold":["Live English-language captions must score at least 98 under the Canadian NER guidelines, as calculated by certified evaluators.","Two programs with live content must be scored each month.","Automatic captions need effective human help, such as glossaries, to reach the threshold regularly."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-11-01","change":"Added the record from the finding of Pablo Romero-Fresco and Nazaret Fresno.","source":"https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/download/774/669","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Broadcasting / live closed captions for deaf and hard-of-hearing viewers"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"peer-reviewed study","citation":"Romero-Fresco P, Fresno N. The accuracy of automatic and human live captions in English. Linguistica Antverpiensia, New Series: Themes in Translation Studies 22 (2023): 114-133, section 3.4 (case study 4: live TV captions in Canada). Published 13 December 2023. doi:10.52034/lans-tts.v22i.774.","title":"The accuracy of automatic and human live captions in English","url":"https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/download/774/669","date":"2023-12-13","written_by":"independent","quote":"for the first time, broadcasters such as Jim Pattison Broadcast Group and The Miracle Channel Associations consistently reach accuracy rates above the required threshold in 2020-2021. Impressively, of the 55 samples analysed by these two broadcasters in 2020-2021, 52 reached 98%."},{"kind":"peer-reviewed study","citation":"Romero-Fresco P, Fresno N. The accuracy of automatic and human live captions in English. Linguistica Antverpiensia, New Series: Themes in Translation Studies 22 (2023): 114-133, section 3.4 (case study 4: live TV captions in Canada). Published 13 December 2023. doi:10.52034/lans-tts.v22i.774.","title":"The accuracy of automatic and human live captions in English","url":"https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/download/774/669","date":"2023-12-13","written_by":"independent","quote":"In 2019-2020, their cloud-based version of the automatic captioning software EEG reached the 98% accuracy rate in 8/31 samples and 10/31 scored below 97.5%. For the 2020-2021 round, they decided to use the local server of the software (called Lexi Local) and to have a dedicated member of the staff compile glossaries with their phonetic spellings to improve the accuracy of the captions. This enabled them to reach the NER threshold consistently, with only 1/24 samples below par but pretty close to the minimum accuracy rate (97.92%)."},{"kind":"peer-reviewed study","citation":"Romero-Fresco P, Fresno N. The accuracy of automatic and human live captions in English. Linguistica Antverpiensia, New Series: Themes in Translation Studies 22 (2023): 114-133, section 3.4 (case study 4: live TV captions in Canada). Published 13 December 2023. doi:10.52034/lans-tts.v22i.774.","title":"The accuracy of automatic and human live captions in English","url":"https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/download/774/669","date":"2023-12-13","written_by":"independent","quote":"They are also more accurate than automatic captions, which feature more samples below 98% accuracy and fewer samples exceeding a 99% accuracy rate (60% of the automatic samples did not reach the minimum threshold and only 1% featured an excellent accuracy).","supports_benefit":false},{"kind":"evaluation report","citation":"The Miracle Channel Association (CJIL-DT). NER98 Accuracy Standards Summary for Miracle Channel (CJIL-DT) 2020-21 and Live-Captioning NER Score Report for 2020-2021, with scores by a certified NER evaluator that the broadcaster engaged. Annual report to the CRTC under Broadcasting Regulatory Policy CRTC 2019-308. Archived copy of https://crtc.gc.ca/eng/BCASTING/ann_rep/miracle_cc2021.pdf.","title":"NER98 Accuracy Standards Summary for Miracle Channel (CJIL-DT) 2020-21","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/BCASTING/ann_rep/miracle_cc2021.pdf","written_by":"evaluator","quote":"We are maintaining consistent NER98 accuracy through the use of an on-premise speech-to-text service called Lexi Local, from EEG Enterprises."},{"kind":"report","citation":"The Miracle Channel Association (CJIL-DT). Summary of The Miracle Channel Association (CJIL-DT) Efforts to Comply with BRP 2019-308 and NER98 Accuracy Standards, with the Live-Captioning NER Score Report for 2019-2020. Annual report to the CRTC, 30 November 2020. Archived copy of https://crtc.gc.ca/eng/BCASTING/ann_rep/miracle_cc2020.pdf.","title":"Summary of The Miracle Channel Association (CJIL-DT) Efforts to Comply with BRP 2019-308 and NER98 Accuracy Standards","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/BCASTING/ann_rep/miracle_cc2020.pdf","date":"2020-11-30","written_by":"deployer","quote":"The review process involved reviewing the video segment that was captioned and fixing all the mistakes that were presented in the exported log file and then uploading that back into the cloud service to help improve the AI and its dictionary in the hope of improving the accuracy moving forward.","supports_benefit":false},{"kind":"regulatory document","citation":"Canadian Radio-television and Telecommunications Commission. Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming. Ottawa, 30 August 2019. Archived copy of https://crtc.gc.ca/eng/archive/2019/2019-308.htm. Appendix 1, Quality Standard 3.","title":"Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/archive/2019/2019-308.htm","date":"2019-08-30","written_by":"independent","quote":"Beginning 1 March 2020, broadcasters must reach an accuracy rate of at least 98 for English-language closed captioning of live television programming","supports_benefit":false},{"kind":"regulatory document","citation":"Canadian Radio-television and Telecommunications Commission. Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming. Ottawa, 30 August 2019. Archived copy of https://crtc.gc.ca/eng/archive/2019/2019-308.htm. Section on requested flexibility for independent broadcasters, relaying the joint intervention of the English-Language Local Independent Television Stations Group (E-ILTS).","title":"Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/archive/2019/2019-308.htm","date":"2019-08-30","written_by":"independent","quote":"Further, E-ILTS acknowledged that voice recognition software, which is sometimes used by independent television stations and allows for costs savings, is improving year by year, but noted that based on preliminary testing, voice recognition-based captioning could have difficulty meeting the proposed NER 98 accuracy standard in the short term.","supports_benefit":false},{"kind":"regulatory document","citation":"Canadian Radio-television and Telecommunications Commission. Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming. Ottawa, 30 August 2019. Archived copy of https://crtc.gc.ca/eng/archive/2019/2019-308.htm. Footnote 6, the members of E-ILTS.","title":"Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/archive/2019/2019-308.htm","date":"2019-08-30","written_by":"independent","quote":"Jim Pattison Broadcast Group Limited Partnership, operating in Kamloops and Prince George, British Columbia, and Medicine Hat, Alberta; Stingray Digital Group Inc. (now Stingray Group Inc.) in Lloydminster, Alberta; Thunder Bay Electronics Limited in Thunder Bay, Ontario; The Miracle Channel Association in Lethbridge, Alberta;","supports_benefit":false}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-1147-source-0001","AIBD-1147-source-0002","AIBD-1147-source-0003","AIBD-1147-source-0004","AIBD-1147-source-0005","AIBD-1147-source-0006","AIBD-1147-source-0007","AIBD-1147-source-0008"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"peer-reviewed study","citation":"Romero-Fresco P, Fresno N. The accuracy of automatic and human live captions in English. Linguistica Antverpiensia, New Series: Themes in Translation Studies 22 (2023): 114-133, section 3.4 (case study 4: live TV captions in Canada). Published 13 December 2023. doi:10.52034/lans-tts.v22i.774.","title":"The accuracy of automatic and human live captions in English","url":"https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/download/774/669","date":"2023-12-13","written_by":"independent","quote":"for the first time, broadcasters such as Jim Pattison Broadcast Group and The Miracle Channel Associations consistently reach accuracy rates above the required threshold in 2020-2021. Impressively, of the 55 samples analysed by these two broadcasters in 2020-2021, 52 reached 98%."},{"kind":"peer-reviewed study","citation":"Romero-Fresco P, Fresno N. The accuracy of automatic and human live captions in English. Linguistica Antverpiensia, New Series: Themes in Translation Studies 22 (2023): 114-133, section 3.4 (case study 4: live TV captions in Canada). Published 13 December 2023. doi:10.52034/lans-tts.v22i.774.","title":"The accuracy of automatic and human live captions in English","url":"https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/download/774/669","date":"2023-12-13","written_by":"independent","quote":"In 2019-2020, their cloud-based version of the automatic captioning software EEG reached the 98% accuracy rate in 8/31 samples and 10/31 scored below 97.5%. For the 2020-2021 round, they decided to use the local server of the software (called Lexi Local) and to have a dedicated member of the staff compile glossaries with their phonetic spellings to improve the accuracy of the captions. This enabled them to reach the NER threshold consistently, with only 1/24 samples below par but pretty close to the minimum accuracy rate (97.92%)."},{"kind":"peer-reviewed study","citation":"Romero-Fresco P, Fresno N. The accuracy of automatic and human live captions in English. Linguistica Antverpiensia, New Series: Themes in Translation Studies 22 (2023): 114-133, section 3.4 (case study 4: live TV captions in Canada). Published 13 December 2023. doi:10.52034/lans-tts.v22i.774.","title":"The accuracy of automatic and human live captions in English","url":"https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/download/774/669","date":"2023-12-13","written_by":"independent","quote":"They are also more accurate than automatic captions, which feature more samples below 98% accuracy and fewer samples exceeding a 99% accuracy rate (60% of the automatic samples did not reach the minimum threshold and only 1% featured an excellent accuracy).","supports_benefit":false},{"kind":"evaluation report","citation":"The Miracle Channel Association (CJIL-DT). NER98 Accuracy Standards Summary for Miracle Channel (CJIL-DT) 2020-21 and Live-Captioning NER Score Report for 2020-2021, with scores by a certified NER evaluator that the broadcaster engaged. Annual report to the CRTC under Broadcasting Regulatory Policy CRTC 2019-308. Archived copy of https://crtc.gc.ca/eng/BCASTING/ann_rep/miracle_cc2021.pdf.","title":"NER98 Accuracy Standards Summary for Miracle Channel (CJIL-DT) 2020-21","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/BCASTING/ann_rep/miracle_cc2021.pdf","written_by":"evaluator","quote":"We are maintaining consistent NER98 accuracy through the use of an on-premise speech-to-text service called Lexi Local, from EEG Enterprises."},{"kind":"report","citation":"The Miracle Channel Association (CJIL-DT). Summary of The Miracle Channel Association (CJIL-DT) Efforts to Comply with BRP 2019-308 and NER98 Accuracy Standards, with the Live-Captioning NER Score Report for 2019-2020. Annual report to the CRTC, 30 November 2020. Archived copy of https://crtc.gc.ca/eng/BCASTING/ann_rep/miracle_cc2020.pdf.","title":"Summary of The Miracle Channel Association (CJIL-DT) Efforts to Comply with BRP 2019-308 and NER98 Accuracy Standards","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/BCASTING/ann_rep/miracle_cc2020.pdf","date":"2020-11-30","written_by":"deployer","quote":"The review process involved reviewing the video segment that was captioned and fixing all the mistakes that were presented in the exported log file and then uploading that back into the cloud service to help improve the AI and its dictionary in the hope of improving the accuracy moving forward.","supports_benefit":false},{"kind":"regulatory document","citation":"Canadian Radio-television and Telecommunications Commission. Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming. Ottawa, 30 August 2019. Archived copy of https://crtc.gc.ca/eng/archive/2019/2019-308.htm. Appendix 1, Quality Standard 3.","title":"Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/archive/2019/2019-308.htm","date":"2019-08-30","written_by":"independent","quote":"Beginning 1 March 2020, broadcasters must reach an accuracy rate of at least 98 for English-language closed captioning of live television programming","supports_benefit":false},{"kind":"regulatory document","citation":"Canadian Radio-television and Telecommunications Commission. Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming. Ottawa, 30 August 2019. Archived copy of https://crtc.gc.ca/eng/archive/2019/2019-308.htm. Section on requested flexibility for independent broadcasters, relaying the joint intervention of the English-Language Local Independent Television Stations Group (E-ILTS).","title":"Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/archive/2019/2019-308.htm","date":"2019-08-30","written_by":"independent","quote":"Further, E-ILTS acknowledged that voice recognition software, which is sometimes used by independent television stations and allows for costs savings, is improving year by year, but noted that based on preliminary testing, voice recognition-based captioning could have difficulty meeting the proposed NER 98 accuracy standard in the short term.","supports_benefit":false},{"kind":"regulatory document","citation":"Canadian Radio-television and Telecommunications Commission. Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming. Ottawa, 30 August 2019. Archived copy of https://crtc.gc.ca/eng/archive/2019/2019-308.htm. Footnote 6, the members of E-ILTS.","title":"Broadcasting Regulatory Policy CRTC 2019-308: English-language closed captioning mandatory quality standards relating to the accuracy rate for live television programming","url":"https://web.archive.org/web/2026id_/https://crtc.gc.ca/eng/archive/2019/2019-308.htm","date":"2019-08-30","written_by":"independent","quote":"Jim Pattison Broadcast Group Limited Partnership, operating in Kamloops and Prince George, British Columbia, and Medicine Hat, Alberta; Stingray Digital Group Inc. (now Stingray Group Inc.) in Lloydminster, Alberta; Thunder Bay Electronics Limited in Thunder Bay, Ontario; The Miracle Channel Association in Lethbridge, Alberta;","supports_benefit":false}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"Researchers who analyzed the station's reports to the CRTC found that its automatic captions reached 98% in 8/31 samples in 2019-2020, while only 1/24 fell below 98% in 2020-2021.","account":"The Miracle Channel Association runs CJIL-DT, a television station in Lethbridge, Alberta. It captions its live news with EEG's automatic speech-to-text software, with no human captioner. From 1 March 2020 the CRTC required live English-language captions to score at least 98 under the NER model. In 2019 a group of independent stations that included the Miracle Channel had told the CRTC that voice recognition captioning could struggle to meet it. Pablo Romero-Fresco, who developed the model, and Nazaret Fresno analyzed the broadcasters' reports to the CRTC. They report that EEG's cloud version reached 98% in 8/31 samples in 2019-2020. The station then moved to Lexi Local, on a local server, and a staff member compiled glossaries with phonetic spellings. In 2020-2021 only 1/24 samples fell below 98%. The station chose its samples, and an evaluator that it engaged scored them. Its report for 2023-2024 lists scores from 98.49 to 99.38. Across the broadcasters studied, 60% of automatic samples missed the threshold. The authors note that unaided automatic captions still fell short of it."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"EEG Lexi Local on-premises automatic speech-to-text captioning (after the cloud-based Lexi service)"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"Romero-Fresco and Fresno find that the Miracle Channel's automatic Lexi Local captions met Canada's accuracy standard in all but one of 24 samples","headline":"*Lexi Local* captions meet Canada's live accuracy standard in Lethbridge"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["b4c8631be4130008f309fbaefe41e4bd3991dd214472cc21019d45a620508eaf"],"role":"SHA-256 of the complete public record and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public record, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"The Miracle Channel Association runs CJIL-DT, a television station in Lethbridge, Alberta. It captions its live news with EEG's automatic speech-to-text software, with no human captioner. From 1 March 2020 the CRTC required live English-language captions to score at least 98 under the NER model. In 2019 a group of independent stations that included the Miracle Channel had told the CRTC that voice recognition captioning could struggle to meet it. Pablo Romero-Fresco, who developed the model, and Nazaret Fresno analyzed the broadcasters' reports to the CRTC. They report that EEG's cloud version reached 98% in 8/31 samples in 2019-2020. The station then moved to Lexi Local, on a local server, and a staff member compiled glossaries with phonetic spellings. In 2020-2021 only 1/24 samples fell below 98%. The station chose its samples, and an evaluator that it engaged scored them. Its report for 2023-2024 lists scores from 98.49 to 99.38. Across the broadcasters studied, 60% of automatic samples missed the threshold. The authors note that unaided automatic captions still fell short of it.","replication":{"produced_by":["EEG's speech-to-text service captioned the station's live news with no human captioner.","In 2020-2021 the station moved from the cloud version of the software to Lexi Local, on a local server.","A staff member compiled glossaries of words with their phonetic spellings.","Staff adjusted the glossary from the errors that the certified evaluator found."],"must_hold":["Live English-language captions must score at least 98 under the Canadian NER guidelines, as calculated by certified evaluators.","Two programs with live content must be scored each month.","Automatic captions need effective human help, such as glossaries, to reach the threshold regularly."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":29,"reviewed":2,"reviewed_complete":2,"reviewed_partial":0,"record_mapped":27,"review_not_assessed":62,"not_assessed":0,"complete":false,"by_state":{"value_present":29,"not_reported":0,"unresolved":62,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"2e9053e05d4a3927e02baa96b26b2ca9237f2e51012a6e54303ad78ead73e108"}