Why internal tools become production risks
reading · 00%

Why internal tools become production risks

A tool built as a side project ends up running a core process, with no access control, no audit log, and no owner.

Published
Reading6 min
CategoryProduction
Words1,229

A spreadsheet macro became the way invoices get approved. A small admin panel became the only place a refund can be issued. A cleanup script became a nightly job three teams depend on.

None of them was built as a production system. Each one now sits in the path of money, customers, or a regulator asking for a record.

Operational risk, for internal tools, is the exposure that builds when the tools a team runs daily operations on are unreliable, uncontrolled, or missing. The risk is the distance between what a tool does and how it was built. The tools that carry the most of it are rarely the ones leadership watches. They are the quiet ones that became load-bearing without anyone deciding they should.

When a tool crosses into production

A tool becomes a production risk when it runs an operation the business cannot easily undo or recover. How it was built, and who built it, matter less than what now depends on it.

A forecast prototype for one analyst is low stakes. The same prototype becomes a production risk the day finance closes the quarter from its output, with no record of how the numbers were produced.

The shift usually happens without a decision. The tool picks up users, then data, then a process with no fallback. The work moved into production, and the engineering did not follow.

So track the role the tool plays. A tool that can move money, change a customer or patient record, or produce a number a regulator may ask about needs production controls. A personal calculation that affects nobody else does not.

The four gaps

Whether a tool was prompted into existence last week or written in-house a decade ago, the same four gaps create the risk.

  • No access control. The tool cannot tell a junior operator from an administrator, so every user holds the most dangerous permission it offers.

  • No audit trail. Nobody can show who changed a record, approved a payment, or exported a dataset. That is the exact evidence an auditor asks for.

  • No owner. Nobody is accountable for its security, nobody answers when it breaks, and nobody has the authority to change it safely.

  • No safe way to change it. The person who understood it has left, and the code is undocumented. Any change may break a process nobody fully understands.

A tool missing one of these can still function. The risk grows with how many are missing and how much depends on the tool.

Old and new tools arrive here from opposite directions. The AI-built tool never had the controls. The legacy tool may have had them once, then lost them as people left.

Why AI-built tools are the fastest-growing case

Vibe coding lets someone with no engineering background prompt a working app into existence in an afternoon. A working app is exactly what feels safe to start using.

What gets baked in that afternoon is the problem. Under time pressure, a database credential or an API key goes straight into the code, because that makes it work now. Everyone who can open the tool can do everything it can do. Nothing records who did what, and the person who prompted it did not sign up to maintain a production system.

The model that wrote the code is not the cause. The same tool, rebuilt with access by role, validation at the point of entry, an audit trail, and a named owner, can run real operations. What is vibe coding covers that path.

How the risk shows up day to day

Three forms recur.

Data nobody trusts. The same information lives in several places, maintained by different people. A finance team approves invoices in a spreadsheet because the official system does not cover its edge cases. Nobody can say which version is correct.

Decisions with no record. Approvals happen over email, handoffs over Slack, and decisions out loud. When a regulator asks how a transaction was approved, there is nothing reliable to pull.

Work that lives in one person. A workflow sits in someone's head, or in a script only they maintain. At a global publisher, we found campaign workflows with no error handling and no logging. When they failed, they failed silently, and the team found out only when downstream data was already wrong.

Why leadership does not see it

Internal tools enter the business as productivity shortcuts. They skip procurement, security review, and the architecture diagram.

A tool that took an afternoon carries the assumed cost of an afternoon, so nobody thinks to review it. The real cost lands later and somewhere else: a failed audit, a data exposure, a quarter-end close that will not reconcile, or a process that stops the day one person leaves.

Leadership sees the tools it commissioned. The risk concentrates in the ones that grew: spreadsheets that became systems, scripts nobody mapped, and AI-built apps spreading faster than any review. They are a form of shadow IT, with one difference. The business built them itself, so it trusts them more than bought software.

How to close the gap

Treat the tool as the production system it already is. Find it, score its exposure, and build the missing controls into it.

  1. Take an inventory. Follow the work, not the software list. Ask each team which spreadsheets, scripts, admin panels, or AI-built apps a core process would stop without.

  2. Name an owner and review access. Someone becomes accountable, and you learn who can currently do what. This review usually surfaces the most urgent exposure, such as a shared admin login or a credential in plain text.

  3. Add controls in order of exposure. Access by role first, then an audit trail on every consequential action, then validation at the point of entry.

  4. Separate the environments. Changes get tested before they reach live operations.

Start with the three workflows where an error costs money, triggers a compliance obligation, or breaks a client commitment. A full rebuild is not needed to begin.

The review also tells you whether to harden the existing tool or rebuild it. On Retool, much of the control layer is part of the platform: permissions on each data connection, audit logs, SSO, source control, and separate environments. That shortens a rebuild, because the controls are configured rather than written from scratch.

What it looks like with the controls built in

At Saxo Bank, fragmented creative workflows moved onto one platform built on Retool, with role-based workflows defined once. Median time to market fell 78.3%, from 310 hours to 67. The controls took guesswork out of every request.

At Pico Clinics, clinics in North America and Europe had handled consent and records differently in each region. One platform brought them under a single system with an audit record.

At the global publisher, structured error handling and logging meant the team knew when a workflow failed as it happened. Before, they found out days later.

Our view

We recommend the inventory and an access review before any rebuild, because the riskiest tool is usually one leadership did not know about. The review shows which tools to harden, which to rebuild, and which can stay as they are. The operations owner decides the order.

The security and compliance checklist covers what IT will ask for. Four controls that make an internal tool safe to rely on goes deeper on each.

Worth passing along? Share this note.
LinkedInX
Newsletter

Get the next one in your inbox

We write when something is worth reading. Subscribing means you are happy to hear from us when that happens. Leave any time. Privacy policy.

Founder

John Miniadis

Keep readingProduction
Related notes

Still here? Good sign

More notes from real builds, written up for the next person who needs one.

All articles
ContactLet's chat
Let's chat

Tell us what
you're building

Describe the process that is slowing your team down: the inbox, the spreadsheet, the portal nobody owns. We'll come back with a concrete approach, a rough timeline, and, where it helps, a working prototype within weeks.

What happens next
  1. We read every messageAn architect reads what you sent before anyone replies.
  2. You hear back in ≤ 2 daysA short reply with first questions and a suggested next step.
  3. We scope and prototypeA scoping call, then a working surface, not a slide deck.
Where are you in the process?