From a manual fraud queue to a daily risk score.

To assess one school account, an analyst checked identity data in one tool and payment history in another. Then came a Jira ticket that might be out of date. Then the next school. Across roughly 10,000 schools, nobody could see which accounts needed attention today and which could wait.

The client is an edtech platform with a trust and safety team responsible for spotting fraudulent school accounts. The team knew what fraud looked like. What it lacked was time to assemble the picture, one account at a time.

Before: one account at a time

The team reviewed school accounts by hand. Its tools were a set of small Retool apps, each built around two or three queries, and Jira tickets holding what earlier reviews had found. Each app did its job. Nothing joined them up.

There was no score, no ranked list, and no history of how an account's risk had moved. Each assessment meant coordinating across tools that were never designed to work together. The process could not keep pace as the school population grew.

What we found one level down

The brief was about faster review. Mapping the workflow with the analysts showed the review itself was not the slow part. Assembling the evidence was. Every signal the team needed already existed somewhere, in data the platform already held or paid for.

So the build became a scoring layer: pull every signal together once a day, rank the schools, and explain each score. The analysts' time would go on judgment rather than on collecting data.

What we built

A daily scoring run

The system scores the full school population every day. It reads from six kinds of source:

• the platform's own data warehouses

• Stripe identity verification records

• ChargeBee subscription and billing events

• SIFT risk signals

• other fraud detection services

• the team's Jira history of earlier reviews and decisions

A driver job splits the schools into batches of 1,000 and processes them in sequence. Each batch runs as its own execution, so a failure is isolated and retried without restarting the full run. Large multi-source queries stay clear of timeouts.

When a source is unavailable, the run continues on partial data. The analyst sees which signals are missing for each school, so a gap shows up instead of quietly lowering a score.

An analyst dashboard in Retool

The team already worked in Retool, so the new dashboards went there too, with no new tool to learn. Analysts open a list of schools ranked by current risk score. Each school's detail view shows identity verification status, subscription plan, flagged course topics, recent flags, and how its risk splits across five indicator categories.

Flags read in plain language, from "SIFT score above threshold" to student billing issues. Each school's score is tracked over time. A high score that appeared overnight reads differently from one that has been building for weeks.

If a data point looks wrong, an analyst submits a correction from the dashboard. It lands in a review queue where it is actioned and tracked, without a separate ticket or an email to another team.

An admin dashboard for tuning

Fraud patterns shift, so the weights cannot be fixed in code. Fraud operations managers adjust how much each indicator category counts, directly in an admin dashboard. A simulation mode shows how a proposed change would move scores across the whole school population before it is committed. The blacklist of schools lives in the same place.

Each data source sits in its own module, with its own retrieval and scoring logic. A new source can be added to one module without restructuring the rest. Once it is live, the operations team calibrates it from the admin dashboard.

What changed

• Coverage: roughly 10,000 schools scored every day, where the team previously reviewed accounts one at a time.

• Triage: analysts start from a ranked list with the reasons attached, rather than a blank search.

• History: every school has a risk profile over time, so analysts can tell a sudden change from a slow build.

• Control: the people who run fraud operations tune the model themselves, without waiting on a developer.

The trust and safety team still makes every enforcement decision. The system decides only where they look first.