Support a Data-Driven Student Retention Program

Student success is institutional success.

EXECUTIVE BRIEF

Analyst Perspective

Identify technology opportunities to support student retention

For decades, the higher education industry had the enviable position of receiving a steady pipeline of students from secondary school into degree programs. The outlook today is much different. There is lower enrolment; student numbers are down both domestically and internationally. Competition for these fewer students has increased and it has become imperative for universities to ensure that students don't drop out or transfer.

There is no greater need for analytics in higher education than to identify students who are at risk of dropping out. To rely on previous measures of DFW rates often results in the alert coming too late. The institution needs data from across the student experience in order to offer students support before it's too late.

The CIO needs to be a significant advisor to the people charged with retention. The institution needs to develop a greater maturity in data governance, it needs to better leverage its existing advising technology, and it needs to involve IT in the discussion to help identify technology opportunities to meet the retention challenge.

Mark Maby
Research Director for Education, Industry Practice
Info-Tech Research Group

Executive Summary

Your Challenge

Student success and retention is becoming increasingly important as competition for students increases and students' expectations rise.

IT needs to be aware of the approach taken by the institution in order to:

• support systems integration.
• implement and perhaps build needed systems.
• ensure the institution can support data quality standards.

Common Obstacles

Most larger institutions have already purchased technology to support retention initiatives. These systems are often under-utilized due to issues related to culture, systems integration and data governance. Common obstacles are:

• A fragmented advising structure.
• Siloed data preventing proper analytics.
• Platforms purchased by advising or a single department, but not integrated across institution.

Info-Tech's Approach

Identify the student retention goals of the institution. Assess the challenges that end users face with their technology, as well as any issues with their data and systems integration. Work with stakeholders to identify the key data to support student retention.

Info-Tech Insight

Institutional culture is key to effective retention. Faculty, staff, and advisors must recognize the value of consistently and accurately recording key information. Only with accurate recording of information can leaders ask the right questions of the data to guide strategy and advising technology deliver value.

Insight summary

Overarching insight

Institutional culture is key to effective retention. Faculty, staff, and advisors must recognize the value of consistently and accurately recording key information. Only with accurate information can leaders ask the right questions of the data to guide strategy and advising technology deliver value.

Phase 1 insight

The institution must be oriented towards student success:

• Leadership has provided clear strategic goals.
• Student support services is well-organized.
• Faculty recognizes its role in student success.

Phase 2 insight

Most institutions have the technology for student retention initiatives.

With effect data culture, those technology platforms will not achieve any value.

Phase 3 insight

Most analytics programs are too narrow in scope of their data.

Leverage student journey mapping to understand the hidden factors affecting at-risk students.

Tactical insight

There needs to be executive ownership of student success.

Misunderstanding of how to use data for the purpose of student retention is a threat to success.

Student success is institutional success

The higher education market is becoming increasingly competitive

• The cost for four years of education will increase from $100,000 in 2021 to $162,818 by 2036. (Chekalov)
• With increased competition comes increased restructuring:
  • University of Tulsa is phasing out 84 low-demand degree programs. (Kline)
  • In England, universities facing financial risk will only receive significant government assistance if they agree to merge or focus on further education to the exclusion of academic research. (Grove)
  • Laurentian University in Canada applied for creditor protection announced that 58 undergraduate programs and 11 graduate programs, spanning a diversity of subjects, will be closed. (CBC News)
  • Seven percent of higher education institutions are currently in discussion about merging with another campus. Twelve percent of leaders feel their institution should. (Chronicle of Higher Education)

Prospects and subsequent enrollment are expected to continue to drop

15%
There is expected to be a 15% drop in freshman prospects in the five years leading up to 2025.
Source: KCUPA-HR, 2019

5.1%
There has already been a 5.1% drop in enrollment since 2019.
Source: National Student Clearinghouse Research Center

How is retention typically addressed?

Governance

• Leadership

Ownership of retention may be with a dedicated VP, with the Provost, with the Registrar or with Enrolment.

Retention is of critical concern to leadership, and the Board of Trustees will want to be updated. The Deans may be held accountable for their programs.

• Committees

A cross-functional retention committee will commonly address the key strategic initiatives including those related to college readiness, persistence, and timely progression to graduation.

Strategy

• Approach

Successful institutions will coordinate quantitative analysis with qualitative insight from the students' perspective. IT's key role here will be to provide strong systems integration to enable the analytics capability.

• Broad Examples of Retention Initiatives
  • Degree audit frequency
  • Auto-apply for graduation
  • Approaches tailored to specific majors
  • Training for faculty and staff on student mental health
  • Exam schedules optimized for student success

Advising Support

• Centralized / Decentralized

Advising support may be centralized for all students regardless of major or decentralized to specific departments. A decentralized approach can provide more insightful advice and be more accessible to the students. However, undeclared first year students need to be addressed separately and there may be a need for more overall advisors.

• Removing institutional barriers to students
  • Addressing course demand bottlenecks
  • Addressing DFW rates for critical courses
  • Holds on accounts that prevent students from registering
  • Generally removing unnecessary steps

Factors affecting retention and completion

Retention rates and completion rates remain challenged

63%
The retention rate at 2-year colleges from 2018-2019 was 63%.
The completion-rate in six years from four-year institutions was also 63%.
Source: National Center for Education Statistics

The wide range of factors affecting student success means there is an equally wide range of data to be integrated.

Sources: 1. New York Times, 2017. 2. BMC Public Health, 2016.

Defining the data model

The data included in the model will often align with the factors listed in the table below. Keep in mind, though, that there are likely factors affecting your specific students which are not included in the model. This can be a limitation of vendor solutions which focus on typical factors affecting retention. A student journey exercise is designed to identify those atypical factors.

Dealing with False Positives

LMS data, attendance, and other measures of student participation are some of the most valuable indicators of student success. However, the quality of the data hinges upon faculty consistently using the LMS and taking attendance.

The issue is that a student might be actively attending and participating in class, but if this is not recorded in the LMS or attendance data, then that student might be falsely identified as at risk.

The IT department may be called upon to write a script to clean the data by removing data from certain courses in order to remedy these false positives.

Balancing technology and privacy concerns

The benefits of location behavior technology and their privacy challenges

Technology now exists that enables an institution to track their students' location on campus. The primary benefit is that attendance can be taken with a consistency that wouldn't otherwise be possible. The following points are useful to consider:

• Students are more trusting of how their institution uses their personal data than they are of corporate social media (see the graph to the right).
• An institution will likely need to get permission from each student for restricted uses of the data, as it would for use of a student's photo.
• Students can opt in or opt out of this use of technology when they sign the privacy agreement that institutions are already collecting from students.
• There are legal considerations that require legal counsel's advice.

Applications of the data

Predictive analytics

Predictive analytics are used to identify a probable outcome in the future. A multiple regression analysis is commonly employed to weigh the contribution different factors are expected to have.

• Early alert systems
• Targeted student advising

An automated trigger for an at-risk student
Secondary school GPA < 92%
SAT < 540
LMS logs in weeks 2-6 < 5x
= 65% retention score
< 70% institutional retention score

Source: Evidence-Based Nursing, 2021.

In this case, the student might be identified as being at medium risk for failing or dropping out. Student support services would apply an intervention appropriate to the situation.

Descriptive analytics

Descriptive analytics are focused on the past and are used to understand how well the current state is functioning.

• Dashboards
• Institutional analysis
• Other reporting

Applications of descriptive analytics

• Courses with the highest number of DFW rates.
• Four.-year completion rates for student with first-year DFW courses
• Number of students with government aid taking less than a full credit load per semester.

Case Study

The preeminent example of retention reform.

INDUSTRY: Higher Education
SOURCE: Interview

Issue

Ineffective student advising and ineffective data-driven analytics left thousands of low-income students to drop out of the university each year.

Georgia State University was experiencing significant challenges in achieving national standards for four- to six-year graduation rates.

A closer analysis revealed that the university had more than 10,000 instances of students underperforming in a pre-requisite course. They may have received a "C" in a math course—a passing grade—but that's not good enough to indicate they will do well in the upper-level chemistry class they need to take next. In the past, these students would have been allowed to take the chemistry class, and after earning a "D" or "F" (or more typically, two or three of them) an adviser would finally reach out to help. The summary of the issue was an ineffective student advising and ineffective data-driven analytics.

At universities like Georgia State, where 58 percent of students qualify as low-income by federal standards, thousands of students were dropping out because they were running out of funds.

Case Study

The preeminent example of retention reform.

INDUSTRY: Higher Education
SOURCE: Interview

Solution

A coordinated system of analytics with advising ensured that students receive attention at the earliest opportunity.

To address this issue, GSU developed a data management and analytics strategy to inform the student and advisor of trouble early in the semester of the pre-requisite course. Now the analytics system alerts the student and the advisor that the student is at risk when the first grade is recorded, and an adviser reaches out immediately to help. Why wait until the student makes costlier mistakes instead of helping at the first sign of a problem?

GSU now delivers the guidance students need in a timely fashion and is doing it for every student. It's having a big impact, especially for low-income and first-generation students.

The average time to degree for graduating seniors has decreased by more than half a semester, saving each graduating class $18 million in tuition and fees. This means lower debt for these students at graduation. It also means more students are graduating, because more students can afford to do so.

Guided Implementation

What does a typical GI on this topic look like?

A Guided Implementation (GI) is a series

of calls with an Info-Tech analyst to help implement our best practices in your organization.

A typical GI is seven to twelve calls over the course of three to five months.

Phase 1

Readiness Assessment: Identify goals and develop detail

Support a data-driven student retention program

This phase will walk you through the following activities:

Gather documentation related to student retention.

Interview key stakeholders to understand the retention context.

Identify IT's role in the retention initiative.

This phase involves the following participants:

Usually there will be a task force or committee with the key retention stakeholders. These are the individuals with whom IT should work to better understand the context.

Step 1.1

Gather Documentation on Leadership's Directive for Student Retention

Activities

1.1.1 Source Documentation Related to the Student Retention Initiative

1.1.2 Review the Documentation and Identify Key Points and Remaining Questions

Phase 1: Readiness Assessment

This step involves the following participants:

IT retention project team

Outcomes of this step

Retention goals and other key information

Clarify the student retention context through a document review

Info-Tech Insight

Ideally the institution will have already conducted some analysis to determine which demographic groups to target and what challenges they face. This demographic profile will describe an obtainable target, achievable by focusing on specified mitigation strategies.

Typical Documentation

• Strategic mission of the institution on the website
• Retention objectives from a strategy document
• Campus planning
• Demographic profile of the current students and the contributing factors to retention challenges.

1.1.1 Source Key Documents

1 hour

1. Review the institution's website for key documentation related to student retention.
2. If there is a task force set up for the student retention initiative, contact the chair of that committee for their project charter and the demographic profile.
3. Contact student support services, institutional research, and/or the registrar for other documentation.

1.1.2 Review Key Documents

1 hour

1. Write down the questions that you are trying to answer with this activity. One of these should be to be to identify the institution's current retention rate and stated retention state. Review the Interview Guide Template for Student Retention. This will be used for interviewing stakeholders later in this phase, but it may be helpful for generating other questions.
2. Review the documents with the questions in mind.
3. Note down questions that you've answered in the Student Retention Readiness Assessment Template.
4. Incorporate any remaining questions into the Interview Guide Template for Student Retention.

Step 1.2

Interview Key Stakeholders to Clarify the Retention Goals and Develop Detail

Activities

1.2.1 Create a List of Stakeholders to Interview

1.2.2 Leverage Info-Tech's Sample Interview Questions

1.2.3 Consolidate Findings

Phase 1: Readiness Assessment

This step involves the following participants:

Student-retention task force or committee.

Outcomes of this step

Understanding of retention in the context of the institution