An illustrative example: a bank's marketing team renews its CRM contract. The new release adds a model that scores and ranks prospects. Nobody in the bank built it, nobody reviewed it, and it does not appear on any list of AI systems. It now influences who gets offered credit.
A governance program cannot govern a system it does not know exists. Before committees and principles, the first job is an inventory. The US federal government made that a requirement for its own agencies: OMB Memorandum M-25-21, issued in April 2025, requires agencies to keep AI use case inventories and to designate a Chief AI Officer. It replaced an earlier memo, M-24-10, and kept both requirements.
This article sets out how to structure governance around that inventory, and how much structure to build for the number and type of systems you actually run.
What makes AI systems harder to govern
Three properties separate AI systems from the ERP and reporting systems that existing controls were designed for.
Scale of error. A person who makes a bad call affects one case. A model with a systematic error repeats it on every case it touches until someone notices.
Limited traceability. A spreadsheet model can be audited line by line. A large language model or deep learning system cannot be fully explained in the same way, even by the people who built it.
Drift. A model trained on one period can give different answers on the same kind of case later, because the world it learned from has changed. A model that passed validation at launch can fail a year later without anyone changing the code.
Layer 1: the board
In March 2024, the SEC settled its first "AI washing" cases. Two investment advisers, Delphia and Global Predictions, paid $225,000 and $175,000 for claiming AI capabilities they did not have. What a company says about its AI is a disclosure question, and disclosure belongs to the board.
The board's job is to approve risk tolerance and to receive reporting, not to review models. In practice that means four things: approve the AI principles and the risk appetite; receive regular reports on the inventory, incidents, and regulatory exposure; approve public claims about AI capability; and check that incentive pay does not reward deployment faster than the controls can follow.
Layer 2: one accountable executive
Name one executive who is accountable for AI risk. The title matters less than the authority. A Chief AI Officer, a CIO, or a Chief Risk Officer can hold the role. The federal memo lets agencies give it to an existing CIO, CDO, or CTO with AI expertise.
The test is whether that person can stop a deployment. If business units can ship a model over the accountable executive's objection, the role is advisory, and the board should know that.
Layer 3: a governance committee, sized to the inventory
A cross-functional committee approves use cases above a defined risk threshold, reviews incidents and near-misses, and maintains the risk taxonomy. The usual members are the accountable executive as chair, legal and compliance, risk, the data governance lead, the business units with the most deployments, and HR where models touch employment decisions.
Size the committee to the inventory. A company with five AI systems, all low-risk, needs a named owner and a quarterly review, not a standing committee. A company with dozens of systems in credit, pricing, hiring, or clinical decisions needs the committee, and it needs to meet often enough to clear its queue. If approvals wait months for a meeting, teams will route around it.
Layer 4: owners in the business units
Each business unit with significant AI use needs a named senior owner. That person makes sure risk assessments are done, incidents are reported, and the unit's roadmap fits its risk profile.
This is where governance breaks most easily. Business units are measured on delivery, and governance looks like friction. Put the owner's name on the inventory entry for every system in their unit, so accountability is visible rather than assumed.
Layer 5: the model itself
At the level of individual systems, governance is a set of documents and checks, and public frameworks already describe them. The NIST AI Risk Management Framework 1.0 (January 2023) organizes the work into four functions: Govern, Map, Measure, and Manage. ISO/IEC 42001:2023 specifies an AI management system that an external body can certify against. Use one of them rather than inventing a taxonomy.
- Risk tiering. Classify each system by the consequence of its decisions, its scale, how explainable it is, who it affects, and its regulatory status. Tiering decides how much of the rest applies.
- Documentation. For higher tiers: a model card stating intended use, training data, and known limits; a risk assessment; a validation report; and an approval record.
- Monitoring. Performance against a baseline, drift checks, an incident log, and thresholds that trigger review.
What regulated industries add
The EU AI Act. Its prohibitions on certain AI practices have applied since February 2025, and obligations for general-purpose AI models since August 2025. Obligations for high-risk systems were deferred to 2 December 2027, and to August 2028 for AI embedded in products covered by Annex I. Map your inventory to the Act's risk categories now, so the high-risk list is known well before the deadline.
Banking. In the US, SR 11-7, the 2011 model risk guidance that banks built their model governance on, no longer stands. On April 17, 2026, the Federal Reserve, OCC, and FDIC replaced it with SR 26-2, revised guidance that takes a risk-based approach. Banks should check how their AI models fall under its definition of a model.
Medical devices. AI that qualifies as a medical device goes through FDA review. In December 2024 the FDA finalized guidance on predetermined change control plans for AI-enabled device software, which lets a manufacturer get planned model updates authorized in advance.
Legal services. The ABA's Formal Opinion 512, issued in July 2024, applies existing duties of competence, confidentiality, and reasonable fees to generative AI use. Lawyers remain responsible for the work product.
What these rules require of your organization is a question for counsel in each jurisdiction. The governance structure's job is to make sure those questions get asked before deployment.
The gaps to check first
AI bought, not built. The CRM example above. Add an AI question to vendor onboarding and to every renewal, and ask vendors to notify you when a release adds or changes a model.
Systems that predate the framework. Models in production for years were often never assessed. Run them through tiering like any new system.
Incident response. Approval processes are often well defined. Fewer organizations have decided what happens when a model produces wrong outputs at scale: who can switch it off, who tells affected customers, and who tells the regulator.
Where to start
Build the inventory first, including vendor software. Give every entry an owner and a risk tier. Then count: the number of high-tier systems tells you whether you need a standing committee or a named owner with a quarterly review. Structure that outruns the inventory becomes paperwork, and structure that lags it leaves systems nobody is watching.
GradeCircle builds and runs six products of its own: AfroSynergy, CallerBridge, HealthSquire, Heyzl, Lexfora, and LukeAPI, two of them in beta. The assumptions in our models come from operating those systems, not from a framework. See what we run.
If you have a specific decision in front of you, scope it in five questions and see the engagement, price band, and timeline before sharing contact details.
