AI tools are developing at a breakneck pace. Every week seems to bring a new announcement about what LLM-powered systems can do. But the speed of technology is not always the right speed for adoption, especially in a highly regulated environment like higher education, where data governance and ongoing review are essential for safe, responsible use. In this post, I want to share one of Ball State’s first practical “AI-infused” workflows from IRDS: using an LLM to evaluate EAB Navigate notes and appointment summaries to produce a simple, actionable signal for advisors. The goal is intentionally narrow: identify when a student has indicated an intention to transfer out, so our outreach workflows can focus time where it is most likely to help.

The point is not to build a flashy AI agent. The point is to make a specific, deeply understood process better, without widening risk.

From data ocean to “what do we do on a Tuesday?”

Higher education runs on a daily flood of student data. IRDS has been building traditional analytics with partners like HelioCampus to support student success for years. In early 2025, IRDS worked with HelioCampus, IT, and the College of Health (CoH) to create a Target List dashboard for non-returning students. The objective was to give Student Success Directors, and others, a short list of students we predicted were likely to return, but who had not registered yet. That list was informed by the data science model built with HelioCampus, plus additional filters like holds or dismissals to filter students with “hard blocks” for different outreach.

This dashboard is useful because it’s not asking staff to boil the ocean. It’s saying, “here’s a small set of students who look like they should be here. Let’s focus.” While iterating with the staff doing this work we observed a real bottleneck. There was a hidden time sink, reading Navigate notes to find transfer intent. The first step on their process was not to immediately begin outreach; it was to go to the student’s Navigate notes to discern transfer out intent. Those students are not prioritized for outreach because they have already told someone they plan to transfer. In working with the College of Health, we discovered the practical reality of the Target List workflow. Their initial list started at 18 students, and staff spent most of their time reading Navigate notes and appointment summaries to answer a first-order question, “does this student intend to transfer out?”

In that workflow, this “needle in a haystack” reading appeared to consume 50–80% of the time spent winnowing the list down to students they actually wanted to contact. The pilot outcome made the case even clearer. After excluding students who planned to transfer out, CoH staff turned an initial list of 18 students into a list of just 2. One of those two students was reached and successfully registered, and the underlying issue turned out to be a fairly common one: Duo problems, where a solvable friction point blocks a student whose indicators otherwise look fine. This is exactly the kind of situation where targeted outreach can have outsized impact. But it’s hard to scale if the scaling cost is “read everything, manually, every time.”

Why an LLM fits for optimizing this specific job

Traditional analytics is strong at structured signals; things like grades, credit momentum, GPA trends, holds, and other indicators that can be modeled and monitored. But “transfer intent” is often expressed in plain language inside Navigate notes. LLMs are unusually good at one particular thing that matters here: reading unstructured narrative text and determining whether a specific idea is present. That doesn’t mean they are “right” by default. It means they are well suited to the job of scanning, extracting, and producing a small piece of structured output that a human can then use in context. We focused on a narrow endpoint, find a transfer-intent classification that can be surfaced directly in the Target List as a column and a filter. The project plan defines the endpoint as a simple trinary outcome (YES/NO/MAYBE) for whether the student intends to transfer out.

Governance: deliberate steps before production use

Because this involves student data and a new-ish technology, we moved deliberately and in phases. We began by testing with thoroughly anonymized comments to evaluate whether AI could accurately read Navigate-style text and match how staff classified students. In our initial testing with CoH, the prompt we developed achieved a 100% match with staff classifications. Only after validating the method did we progress to building a production-ready pipeline with IT and aligning the work with MIDAS governance and review (including formal approval to use the tool in the intended way).

How the pipeline works (designed to be boring on purpose)

This workflow is intentionally not “agentic” (not that there is anything inherently wrong with AI agents but you have to be prepared for it). It’s a controlled classification step in a deterministic workflow that produces a small, auditable, and very valuable output.

The actual architecture is:

1. Comment
   Advisors enter notes and appointment summaries in Navigate.
2. Ingestion to Redshift
   Navigate notes and appointment summaries are pulled via API into our governed environment (staged in Oracle as the data lake), then transported into Redshift for analytics and dashboarding.
3. Concatenation
   In Redshift, we roll up the text so each student has one concatenated block of Navigate narrative (notes + appointment summaries) ready for evaluation.
4. Upload to API for evaluation
   On a regular cadence, we generate a per-student JSON file and securely send it to the OpenAI API for classification.
5. Get back a decision
   The API returns a simple classification (YES / NO / MAYBE for transfer intent). We write that result back into our governed data stores and surface it in Redshift as a field and filter in the Target List dashboard used by advisors.

The key design decision is that the AI output becomes a governed data element in our existing analytics ecosystem, rather than an external black box making “live” decisions inside a transactional system.

What this changes for staff

This classification doesn’t replace the need to read notes. It changes where that reading happens. Instead of every advisor starting every review by scanning text to determine whether transfer intent is present, the Target List can surface a “Transfer Out” status derived from Navigate notes. That means staff can triage faster, reduce time spent on dead-end outreach, and focus effort where it’s most likely to matter.

In other words, we spend less time sorting, more time helping.

Moving toward enterprise-grade AI with great partners

One important lesson from this project is that “we can make an API call” is not the same as “this is enterprise-ready.” While our initial path uses the OpenAI API via a somewhat manual process, we are also exploring a second workflow optimization option with AWS in partnership with HelioCampus. We’re evaluating Redshift-native AI tooling (including options involving Bedrock and SageMaker) that may reduce steps and help us move toward a more scalable, enterprise-level pattern. That’s the direction we care about long-term. We want to bring AI into the institution in ways that are secure, governed, maintainable, and aligned with how enterprise systems should operate, rather than relying on one-off manual processes or poorly governed AI agents.

How we’ll measure success

The success criteria here is not “the AI was impressive.” The success criteria is operational:

If the filter works, it should dramatically reduce the time needed to get to the right targeted list of students. The evaluation can run as-needed (for key points in the semester cycle) or continuously. Initially, outcomes tracking will be manual so we can understand impact carefully, and after one full semester we can compare results with previous retention rates using a retrospective Target List of students.

The bigger lesson

The most valuable AI projects in higher education may not be the flashiest. They may be the ones that take unstructured information we already have, apply strong governance, and return a small, accountable signal that fits neatly into an existing workflow. That’s what we aimed for here. Use data with guardrails, capability with humility, and forge a future-focused path toward enterprise-grade implementation with strong partners.

 

Initial draft written by Michael Lane. Structuring the flow, ChatGPT 5.2. Images, Nano Banana Pro