AI Governance for Small Business Without a Compliance Department

AI governance for small business is not a binder, a committee or a six-month program. It is a short written answer to four questions your staff are already answering informally, one prompt at a time: which tools are approved, what data may go into them, who checks an output before it reaches a customer, and who decides when those rules change. Almost every company under 200 people already has AI in production — in a proposal draft, a sales follow-up, a spreadsheet formula, a support reply. What is missing is not the technology. It is the sentence that says what is allowed.

This guide governs AI usage, which is a different job from governing data assets. Data governance consulting settles ownership, quality, lineage and retention for the records you already hold. AI governance settles behavior: what an employee may paste into a public chatbot, which vendors have been vetted, when a human must sign off, what gets logged, and who owns the policy when it needs to change. You can have immaculate data governance and still have someone pasting a signed client contract into a free consumer account on a personal login.

Everything below is written for a 10-to-200-person company — the kind we work with across Orange County — with no compliance department, no legal team down the hall and no appetite for ceremony: a one-page policy, a shadow-AI inventory you can finish in an afternoon, three data tiers, a named owner and a quarterly review that takes an hour.

What does governance actually cover in a 30-person company?

Five things, and nothing else is mandatory. An inventory of the AI tools in use, an acceptable-use rule set, data tiers that define what may leave the building, a human-review requirement for anything customer-facing or financial, and a named owner with a review date. Everything beyond those five is elaboration you can add later.

The trap at this size is copying an enterprise AI policy. Those documents assume a risk committee, a model inventory, a procurement function and someone whose whole job is reading vendor terms. Borrow their structure and you will produce a document nobody reads and nobody enforces — which is worse than no policy, because it creates a written standard you are visibly not meeting. One page that is actually followed beats twelve pages that are filed.

• A tool inventory — what is in use, by whom, on whose account
• Acceptable use — the allowed list, the banned list, and how something gets added
• Data tiers — what may be pasted where, in plain language
• Human review — which outputs a person must check before they leave the company
• An owner and a cadence — one name, one recurring date

How do you find the AI your team is already using?

Start with money and identity, not with a survey. Pull the last three months of card statements and look for small recurring per-seat software charges, then open the app-consent and connected-apps list in your Microsoft 365 or Google Workspace admin center. Between them you will find most of what is running.

Then ask the team directly, and ask it without a threat attached. People hide shadow AI when the question sounds like an audit; they volunteer it when the question is "what is saving you time right now, and what would you miss if I took it away?" You want the honest list, because the tools people rely on are the ones your policy has to actually address.

Do not forget the AI that arrived without anyone choosing it. Your CRM, your helpdesk, your meeting recorder and your design tool have all shipped AI features into existing subscriptions over the past two years, often on by default. Those count as AI in production even though no one signed a new contract, and they are the ones an inventory built from purchase orders will miss. If you are not sure what your Microsoft tenant has switched on, a Microsoft 365 consultant can read the admin surface faster than you can.

• Card and expense statements — per-seat AI subscriptions bought by individuals
• Microsoft 365 or Google Workspace admin — OAuth grants and connected apps
• Browser extensions installed on company machines
• AI features switched on inside tools you already pay for
• A no-blame conversation with each team lead

What data may go into an AI tool, and what never may?

Three tiers are enough: public, internal and confidential, plus a short never-anywhere list. Tie each tier to the kind of account rather than to a named product, so the policy survives the next tool your team adopts. The table below is the version we hand clients as a starting point; edit the examples to match your industry.

The distinction that matters most is the account, not the brand. The same model can be reached through a free personal login with no written commitment about your data, and through a business subscription your company holds under terms you have actually read. Staff rarely know which one they are using, which is why the rule should be written as "on a company account, signed in with your work email" rather than "ChatGPT is fine". If your industry carries contractual security obligations — client security addenda, insurer requirements, anything with a data-processing schedule — read those before you write your tiers, because they will already contain restrictions you must inherit.

Which rules belong on the one page?

Six rules cover most of the risk: use approved tools on company accounts, respect the data tiers, check any output that leaves the company, disclose AI use where a client or regulator would expect it, never let a model make a final decision about a person, and route new tool requests to the owner instead of expensing them.

Write each rule as an instruction rather than a principle. "Be thoughtful about confidentiality" is unenforceable; "do not paste client contracts, employee records or anything with a customer's name into a tool that is not on the approved list" is a rule a new hire can follow on day one. The human-review rule deserves the most specificity you can manage, because it is the one that prevents the failure your clients will actually see — the fabricated citation in a proposal, the invented figure in a board deck, the confidently wrong answer in a support reply.

Add one sentence that is easy to skip and expensive to omit: what an employee should do when something goes wrong. Someone will paste the wrong thing into the wrong tool. If the policy names a person to tell and promises the report will not be punished, you find out the same day and can act. If it does not, you find out from a client.

• Approved tools only, on company accounts with work email sign-in
• Data tiers govern what may be entered — no exceptions for "just this once"
• A named human checks every output that leaves the company
• Disclose AI involvement where a client, insurer or regulator would expect it
• No fully automated decisions about hiring, credit, pricing or discipline
• New tool requests go to the owner; report mistakes without fear

Who owns this when nobody's title says compliance?

One person, named in the document, with authority to approve and remove tools. In practice this is usually the operations lead, the finance lead or whoever already owns vendor relationships — not the most technical person on staff, because the job is judgment and follow-through rather than model expertise.

Give that owner three standing duties and nothing more: approve or decline new tool requests within a week, keep the inventory current, and run a quarterly review of what changed. The quarterly review is where governance survives contact with reality — vendors change their terms, teams adopt new tools, and a rule written in January stops matching how people work by June. An hour a quarter, with the inventory open, is the whole maintenance burden.

If you want a formal spine to grow into, the NIST AI Risk Management Framework organizes the same work into four functions — Govern, Map, Measure and Manage. It is voluntary, free, and written for organizations far larger than yours, so take the structure and ignore the scale. Mapping your one page onto its vocabulary later is straightforward; retrofitting a program you never started is not.

How is this different from a data governance program?

Data governance asks who owns a record, whether it is accurate, where it came from and how long you keep it. AI governance asks what a person may do with a model, what may be shown to it, and who is accountable for what it produces. The first governs assets at rest; the second governs behavior in the moment.

They meet in one place worth planning for. The moment you point an AI assistant at your own documents — a retrieval assistant over SOPs, a Copilot reading a SharePoint library — every permission mistake in that library becomes an answer the model will happily give to whoever asks. A tool that respects existing permissions is only as good as those permissions were, and most small companies have never audited them. That is the crossover where data governance consulting stops being a separate project and becomes a prerequisite for the AI one.

The sequence we recommend is usage rules first, because they are cheap and stop the bleeding this week, then permissions and data quality before any internal assistant is pointed at your own content.

What does a realistic first month look like?

Four weeks, a few hours each, done by someone who already works there. Week one is the inventory. Week two drafts the tiers and the one-page policy. Week three is a thirty-minute all-hands walkthrough. Week four settles the approved-tool list and puts the quarterly review in a calendar. Nothing here needs a consultant.

What a consultant is useful for is the next question, not this one: whether the AI you now govern is the AI you should be running at all. Our AI Readiness Sprint is a fixed-fee, two-week assessment of where AI would actually pay off in your business and what your data needs first, and you keep the roadmap either way. Broader build and rollout work sits under AI consulting. If you are still at the stage of deciding what to adopt, the tool landscape is covered separately in our guide to AI tools for small business.

One honest caveat on scope: this page is operational guidance, not legal advice. AI does not create an exemption from the privacy, employment and sector rules you already operate under, and California businesses in particular should confirm their obligations with counsel rather than inferring them from a vendor's marketing page. Governance is what makes that conversation short when it happens.

• Week 1 — inventory: statements, admin consoles, extensions, a conversation per team
• Week 2 — draft: three data tiers, six rules, one named owner
• Week 3 — brief the team in thirty minutes, with examples from their actual work
• Week 4 — publish the approved-tool list and book the quarterly review

Further reading: NIST — AI Risk Management Framework (AI RMF 1.0).