{"id":"AIBD-1154","title":"Brown and CSUN researchers find that the CSUNny chatbot raised timely class registration at CSUN but did not lift grades, persistence or graduation","summary":"In a four-year trial with 8,708 CSUN students from 2018, the CSUNny chatbot raised timely class registration by up to 34 percentage points but had no detectable effect on grades or persistence.","date_observed":"2026-07-01","date_basis":"source publication","date_precision":"month","benefit_status":"mixed","source_collection":null,"source_entry":null,"account":"California State University, Northridge (CSUN) engaged Mainstay in 2018 to build CSUNny, a text-message chatbot for undergraduates. Staff send campaigns, and the system's non-generative AI answers questions that match a campus knowledge base. CSUN assigned about 4,361 students in its 2018 and 2019 entering cohorts to the chatbot and 4,347 to a control group at random. Catherine Mata and Lindsay Page of Brown University and Emily Russell of CSUN followed both cohorts for four years. They report that a 2018-19 registration reminder made students 34 percentage points more likely to enroll in classes by 16 August. Reminders about the Early Start summer program raised enrollment in it by 11 and 20 percentage points. They found no statistically significant effect on enrollment, persistence, credits, GPA or graduation. CSUN had adopted the chatbot to improve persistence and completion. The authors could not assess most targeted campaigns, such as FAFSA reminders. Version 2 of the working paper is dated July 2026.","category":"aggregate result","unexpected":false,"purpose":"Mainstay built CSUNny for California State University, Northridge, to send students text-message outreach and to answer their questions from a campus knowledge base.","system":"CSUNny, CSUN's AI-enabled text-message chatbot on the Mainstay platform","model_version":null,"developer":"Mainstay (formerly AdmitHub)","deployer":"California State University, Northridge (CSUN)","location":"Northridge, Los Angeles, California, United States","domain":"Education","segment":null,"benefit_type":"Increased Access to Education","work_activities":["4.A.4.a.3"],"taxonomy_note":null,"registry_ids":{},"ratings":[{"group":"personal","depth":1,"reason":"Reminders made students more likely to register for classes and Early Start by the deadlines studied, but credits, grades, persistence and graduation did not change."}],"ai_role":{"status":"unassessed","statement":"Students were randomly assigned to the chatbot or to the university's standard communications. The estimates cover the whole service, including campaigns written by staff and answers that staff supplied. The paper describes the AI as non-generative matching of questions to a knowledge base with a probability threshold. It does not describe the model."},"evidence":[{"kind":"report","citation":"Mata C, Russell E, Page LC. Sustaining AI-Enabled Student Support: A Four-Year Implementation and Impact Study. EdWorkingPaper 26-1409, version 2, July 2026, Annenberg Institute at Brown University. doi:10.26300/b0dp-sn77. Supported by the Kresge Foundation. Emily Russell is at California State University, Northridge.","title":"Sustaining AI-Enabled Student Support: A Four-Year Implementation and Impact Study","url":"https://edworkingpapers.com/sites/default/files/ai26-1409_v2.pdf","date":"2026-07-01","written_by":"deployer","quote":"Students remain receptive over time, and impacts are concentrated in improved completion of time-sensitive administrative tasks, with no detectable effects on academic performance or persistence."}],"replication":{"produced_by":["A chatbot team of CSUN staff and faculty wrote and sent the campaigns.","Offices such as the registrar and financial aid supplied lists of students for targeted reminders.","The chatbot answered questions that matched its knowledge base and flagged the rest for a staff member."],"must_hold":["The task must be time-sensitive and required for continued enrollment, such as registering for classes.","Campus offices must share student data so that targeted reminders reach the students who still need to act."],"echoes":[]},"paired_incident":null,"headline":"*CSUNny* chatbot speeds registration but leaves persistence unchanged","metric":"34 percentage points more likely to enroll in classes by 16 August","comparator":{"against":"Control-group students who received only the university's standard communications","before":null,"after":"34 percentage points more likely to enroll for classes by August 16th","basis":"measured","quote":"The first reminder was particularly effective, with treated students being 34 percentage points more likely to enroll for classes by August 16th compared to the control group.","url":"https://edworkingpapers.com/sites/default/files/ai26-1409_v2.pdf"},"later":null,"value":null,"developer_country":"United States","deployer_country":"United States","unrated_because":null,"brief_notes":null,"brief":null,"revisions":[{"date":"2026-11-01","change":"Added the record from the report of Mata, Russell and Page.","source":"https://edworkingpapers.com/sites/default/files/ai26-1409_v2.pdf","editor":"AIBD"}],"confidence":"claimed","shade":"low","claim_status":"The deployer's account","claim_basis":"Sources: the deployer's report. The cited evidence includes no independent source and no finding by an external body.","case_jurisdictions":["United States"],"profile":{"case_id":"AIBD-1154","catalog_version":"1.0.0","source_fingerprint":"828631934153d6d9c324a48a804bdf921682e48be5594aa49af8ab65fdef5455","collection_refs":[{"collection_id":"aibd","version":"2026-10-31"}],"source_ids":["AIBD-1154-source-0001"],"families":{"strategic assessments":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"evaluations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"monetary observations":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]},"institutional decisions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"Editorial assignment.","review_state":"not_assessed","coverage":"partial","record_ids":[]}},"fields":{"actors[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"developer":"Mainstay (formerly AdmitHub)","deployer":"California State University, Northridge (CSUN)"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["developer","deployer"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"actors[].ownership_control":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_contribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"ai_relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"category":"aggregate result","ai_role":{"status":"unassessed","statement":"Students were randomly assigned to the chatbot or to the university's standard communications. The estimates cover the whole service, including campaigns written by staff and answers that staff supplied. The paper describes the AI as non-generative matching of questions to a knowledge base with a probability threshold. It does not describe the model."}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ai_role","category"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].alternatives":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].assessor_date":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].depth":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":1,"reason":"Reminders made students more likely to register for classes and Early Start by the deadlines studied, but credits, grades, persistence and graduation did not change."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].display_confidence":{"state":"value_present","value":{"kind":"structured","data":{"confidence":"low","evidence_word":"claimed","claim_status":"The deployer's account","basis":"Sources: the deployer's report. The cited evidence includes no independent source and no finding by an external body.","interpretation":"Existing display rule preserved; the richer outcome-specific confidence question still needs review."}},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rationale":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":1,"reason":"Reminders made students more likely to register for classes and Early Start by the deadlines studied, but credits, grades, persistence and graduation did not change."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"assessments[].rubric_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].status_basis":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"assessments[].target":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":1,"reason":"Reminders made students more likely to register for classes and Early Start by the deadlines studied, but credits, grades, persistence and graduation did not change."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"availability":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"brief":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_kind":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"case_links[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_links":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].evidence_strength":{"state":"value_present","value":{"kind":"structured","data":{"answer":"The deployer's account. Sources: the deployer's report. The cited evidence includes no independent source and no finding by an external body.","evidence_word":"claimed","claim_status":"The deployer's account"}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":["AIBD-1154-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The deployer's account. Sources: the deployer's report. The cited evidence includes no independent source and no finding by an external body.","inferred":false}},"claims[].missingness":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"claims[].proposition":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"cross_border_flows[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"decisions[].action_outcome":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].grounds":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].institution_task":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"decisions[].success_criterion":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"institutional decisions"},"deployment_stage":{"state":"value_present","value":{"kind":"structured","data":{"classification":"routine use","answer":"The system was in routine use. CSUN has texted all incoming classes through the chatbot since fall 2020.","scope":"The period and activity described in the case."}},"provenance":{"kind":"source_review","basis":"Editorial assignment.","record_fields":["account","evidence"],"source_refs":["AIBD-1154-source-0001"],"reviewed_on":"2026-10-29"},"review_state":"reviewed","coverage":"complete","display":{"answer":"The system was in routine use. CSUN has texted all incoming classes through the chatbot since fall 2020.","inferred":false}},"economics[].amount":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].costs[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].counterfactual":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].currency_price_basis":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].gross_net":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].measure_type":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].period_unit":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].realization":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].recipient_payer":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].return_payback":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"economics[].valuation_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"monetary observations"},"evaluations[].ai_uplift":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].comparator":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].design":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].effect":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].exposure":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].id":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].measurement_method":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].metric":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].reproduction_materials":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].setting_sample":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"evaluations[].uncertainty_limitations":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"evaluations"},"external_references[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"follow_up[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"human_oversight":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"id":{"state":"value_present","value":{"kind":"record_fields","fields":{"id":"AIBD-1154"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["id"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"infrastructure_dependencies[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiaries":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].beneficiary_count":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].benefit_type":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_type":"Increased Access to Education"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].claim":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"California State University, Northridge (CSUN) engaged Mainstay in 2018 to build CSUNny, a text-message chatbot for undergraduates. Staff send campaigns, and the system's non-generative AI answers questions that match a campus knowledge base. CSUN assigned about 4,361 students in its 2018 and 2019 entering cohorts to the chatbot and 4,347 to a control group at random. Catherine Mata and Lindsay Page of Brown University and Emily Russell of CSUN followed both cohorts for four years. They report that a 2018-19 registration reminder made students 34 percentage points more likely to enroll in classes by 16 August. Reminders about the Early Start summer program raised enrollment in it by 11 and 20 percentage points. They found no statistically significant effect on enrollment, persistence, credits, GPA or graduation. CSUN had adopted the chatbot to improve persistence and completion. The authors could not assess most targeted campaigns, such as FAFSA reminders. Version 2 of the working paper is dated July 2026.","benefit_type":"Increased Access to Education"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","benefit_type"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].distribution":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].duration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].id":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].stakeholder_group":{"state":"value_present","value":{"kind":"record_fields","fields":{"ratings":[{"group":"personal","depth":1,"reason":"Reminders made students more likely to register for classes and Early Start by the deadlines studied, but credits, grades, persistence and graduation did not change."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["ratings"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].status":{"state":"value_present","value":{"kind":"record_fields","fields":{"benefit_status":"mixed"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["benefit_status"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"outcomes[].tradeoffs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"outcomes[].unexpected":{"state":"value_present","value":{"kind":"record_fields","fields":{"unexpected":false,"purpose":"Mainstay built CSUNny for California State University, Northridge, to send students text-message outreach and to answer their questions from a campus knowledge base."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["unexpected","purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"places[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"location":"Northridge, Los Angeles, California, United States","developer_country":"United States","deployer_country":"United States"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["location","developer_country","deployer_country"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"purpose":{"state":"value_present","value":{"kind":"record_fields","fields":{"purpose":"Mainstay built CSUNny for California State University, Northridge, to send students text-message outreach and to answer their questions from a campus knowledge base."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["purpose"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"record_dates":{"state":"value_present","value":{"kind":"record_fields","fields":{"date_observed":"2026-07-01","date_basis":"source publication","date_precision":"month","revisions":[{"date":"2026-11-01","change":"Added the record from the report of Mata, Russell and Page.","source":"https://edworkingpapers.com/sites/default/files/ai26-1409_v2.pdf","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["date_observed","date_basis","date_precision","revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"replication.attempts[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"replication.conditions":{"state":"value_present","value":{"kind":"record_fields","fields":{"replication":{"produced_by":["A chatbot team of CSUN staff and faculty wrote and sent the campaigns.","Offices such as the registrar and financial aid supplied lists of students for targeted reminders.","The chatbot answered questions that matched its knowledge base and flagged the rest for a staff member."],"must_hold":["The task must be time-sensitive and required for continued enrollment, such as registering for classes.","Campus offices must share student data so that targeted reminders reach the students who still need to act."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"revision_history[]":{"state":"value_present","value":{"kind":"record_fields","fields":{"revisions":[{"date":"2026-11-01","change":"Added the record from the report of Mata, Russell and Page.","source":"https://edworkingpapers.com/sites/default/files/ai26-1409_v2.pdf","editor":"AIBD"}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["revisions"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sector":{"state":"value_present","value":{"kind":"record_fields","fields":{"domain":"Education"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["domain"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].access_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].citation_locator":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"report","citation":"Mata C, Russell E, Page LC. Sustaining AI-Enabled Student Support: A Four-Year Implementation and Impact Study. EdWorkingPaper 26-1409, version 2, July 2026, Annenberg Institute at Brown University. doi:10.26300/b0dp-sn77. Supported by the Kresge Foundation. Emily Russell is at California State University, Northridge.","title":"Sustaining AI-Enabled Student Support: A Four-Year Implementation and Impact Study","url":"https://edworkingpapers.com/sites/default/files/ai26-1409_v2.pdf","date":"2026-07-01","written_by":"deployer","quote":"Students remain receptive over time, and impacts are concentrated in improved completion of time-sensitive administrative tasks, with no detectable effects on academic performance or persistence."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].id":{"state":"value_present","value":{"kind":"identifiers","identifiers":["AIBD-1154-source-0001"],"role":"Existing evidence items, in their original order"},"provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"sources[].language":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"sources[].relationship":{"state":"value_present","value":{"kind":"record_fields","fields":{"evidence":[{"kind":"report","citation":"Mata C, Russell E, Page LC. Sustaining AI-Enabled Student Support: A Four-Year Implementation and Impact Study. EdWorkingPaper 26-1409, version 2, July 2026, Annenberg Institute at Brown University. doi:10.26300/b0dp-sn77. Supported by the Kresge Foundation. Emily Russell is at California State University, Northridge.","title":"Sustaining AI-Enabled Student Support: A Four-Year Implementation and Impact Study","url":"https://edworkingpapers.com/sites/default/files/ai26-1409_v2.pdf","date":"2026-07-01","written_by":"deployer","quote":"Students remain receptive over time, and impacts are concentrated in improved completion of time-sensitive administrative tasks, with no detectable effects on academic performance or persistence."}]}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["evidence"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"sources[].upstream_sources":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"strategic_assessments[]":{"state":"unresolved","review_state":"not_assessed","coverage":"partial","family":"strategic assessments"},"summary":{"state":"value_present","value":{"kind":"record_fields","fields":{"summary":"In a four-year trial with 8,708 CSUN students from 2018, the CSUNny chatbot raised timely class registration by up to 34 percentage points but had no detectable effect on grades or persistence.","account":"California State University, Northridge (CSUN) engaged Mainstay in 2018 to build CSUNny, a text-message chatbot for undergraduates. Staff send campaigns, and the system's non-generative AI answers questions that match a campus knowledge base. CSUN assigned about 4,361 students in its 2018 and 2019 entering cohorts to the chatbot and 4,347 to a control group at random. Catherine Mata and Lindsay Page of Brown University and Emily Russell of CSUN followed both cohorts for four years. They report that a 2018-19 registration reminder made students 34 percentage points more likely to enroll in classes by 16 August. Reminders about the Early Start summer program raised enrollment in it by 11 and 20 percentage points. They found no statistically significant effect on enrollment, persistence, credits, GPA or graduation. CSUN had adopted the chatbot to improve persistence and completion. The authors could not assess most targeted campaigns, such as FAFSA reminders. Version 2 of the working paper is dated July 2026."}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["summary","account"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].autonomy_permissions":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].component_role":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].deployment_configuration":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].inputs_outputs":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_name":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].model_version":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"systems[].product_name":{"state":"value_present","value":{"kind":"record_fields","fields":{"system":"CSUNny, CSUN's AI-enabled text-message chatbot on the Mainstay platform"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["system"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"systems[].provider":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"taxonomy_crosswalks[]":{"state":"unresolved","provenance":{"kind":"curatorial","basis":"Editorial assignment.","record_fields":[],"source_refs":[]},"reason":"This is an explicit unresolved assignment: the sources on record do not settle this question.","review_state":"not_assessed","coverage":"partial"},"title":{"state":"value_present","value":{"kind":"record_fields","fields":{"title":"Brown and CSUN researchers find that the CSUNny chatbot raised timely class registration at CSUN but did not lift grades, persistence or graduation","headline":"*CSUNny* chatbot speeds registration but leaves persistence unchanged"}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["title","headline"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"},"version":{"state":"value_present","value":{"kind":"identifiers","identifiers":["a90efb833647425cadf5b13d8ccb447325d0d66e79df7cc54450a4247275146d"],"role":"SHA-256 of the complete public record and its assessments"},"provenance":{"kind":"curatorial","basis":"SHA-256 of the public record, its assessments, and its displayed answers and notes, excluding its own identifier and the derived coverage summary.","record_fields":[],"source_refs":[]},"review_state":"record_mapped","coverage":"complete"},"workflow":{"state":"value_present","value":{"kind":"record_fields","fields":{"account":"California State University, Northridge (CSUN) engaged Mainstay in 2018 to build CSUNny, a text-message chatbot for undergraduates. Staff send campaigns, and the system's non-generative AI answers questions that match a campus knowledge base. CSUN assigned about 4,361 students in its 2018 and 2019 entering cohorts to the chatbot and 4,347 to a control group at random. Catherine Mata and Lindsay Page of Brown University and Emily Russell of CSUN followed both cohorts for four years. They report that a 2018-19 registration reminder made students 34 percentage points more likely to enroll in classes by 16 August. Reminders about the Early Start summer program raised enrollment in it by 11 and 20 percentage points. They found no statistically significant effect on enrollment, persistence, credits, GPA or graduation. CSUN had adopted the chatbot to improve persistence and completion. The authors could not assess most targeted campaigns, such as FAFSA reminders. Version 2 of the working paper is dated July 2026.","replication":{"produced_by":["A chatbot team of CSUN staff and faculty wrote and sent the campaigns.","Offices such as the registrar and financial aid supplied lists of students for targeted reminders.","The chatbot answered questions that matched its knowledge base and flagged the rest for a staff member."],"must_hold":["The task must be time-sensitive and required for continued enrollment, such as registering for classes.","Campus offices must share student data so that targeted reminders reach the students who still need to act."],"echoes":[]}}},"provenance":{"kind":"legacy_record","basis":"Editorial assignment.","record_fields":["account","replication"],"source_refs":[]},"review_state":"record_mapped","coverage":"partial"}},"summary":{"total":91,"addressed":91,"known":28,"reviewed":2,"reviewed_complete":2,"reviewed_partial":0,"record_mapped":26,"review_not_assessed":63,"not_assessed":0,"complete":false,"by_state":{"value_present":28,"not_reported":0,"unresolved":63,"not_assessed":0,"not_applicable":0,"withheld":0,"source_unavailable":0}}},"presentation_version":"6f542a581a1a05b36cf84c28ba69b1f7bb9ba2672d22999404e7c1ca7a91215b"}