The questions a board asks about AI, and how to answer with evidence

01 / 11

The line that showed up in October's board pack

Consider a fictional 200-person insurance broker. Every quarter the board pack has the same shape: growth, claims ratio, headcount, a line on the cyber insurance renewal. This October there is a new line, added by the chair after a conversation with the firm's own insurer: AI risk. Nobody on the leadership team wrote it. Nobody owns it yet.

The temptation is to answer it with a paragraph of reassurance — we take AI seriously, we have looked into our exposure. A director who has sat through a few of these will ask a harder question back: how do you know? That is the actual problem this post solves. Not which AI policy sounds impressive, but what you can point to when someone asks you to prove it.

02 / 11

Six questions, and two different kinds of answers

A board does not usually ask are we secure. It asks something more specific, and it tends to arrive as a short list. Each of the six below has a different kind of answer. Some come from records your tenant already keeps. Others exist nowhere inside Microsoft 365 at all, no matter how good your dashboard is.

• Which AI tools are actually in use, beyond the ones IT approved on purpose
• What data can each one reach inside the tenant
• Who approved it, and was the risk actually looked at
• Does it train on our data, or just process it and move on
• Can we turn one off if we decide we do not want it
• What does our written policy say, and does anyone follow it

03 / 11

Which tools are actually connected

Start with the evidence that already exists. Any AI tool given permission to sign in with a Microsoft account shows up as an enterprise application in Entra ID, whether or not the person who clicked accept meant it as a company decision. Microsoft's guidance on managing generative AI apps also points to the Defender portal's cloud app catalog, filtered to a Generative AI category. Together, that is a real inventory, not a guess.

Entra adds a second layer: on a tenant with Entra ID P1 or P2, the usage and insights report shows successful and failed sign-ins per application, and a last sign-in date — the difference between someone consented to this once and people are still using it.

It has a real gap, too. An employee typing into an AI chatbot in a browser tab, with no Microsoft sign-in involved, leaves no trace in either list. Seeing that takes a separate, network-level sensor — Microsoft describes its shadow AI discovery capability as a network-based feature that provides visibility into unsanctioned AI applications and tools used in your organization — a different capability, not a setting inside the app inventory. Tell your board plainly which of the two you actually have switched on.

04 / 11

What each one can actually reach

An entry in the enterprise applications list is not useful by itself — the question is what it was granted. Microsoft draws a line between delegated permissions, which let an app access some data at the protected resource, while acting as that user, and permissions an administrator grants for the whole organisation, which can run with no signed-in user at all. Reading which permissions an app holds — calendar, mail, files across every user's OneDrive — is the real answer to what can it reach, and the Permissions page for each application lists exactly that.

The newer, closer answer is prompt-level visibility. Microsoft Purview's Data Security Posture Management for AI, where configured, records activity type and user, date and time, AI app category and app, app accessed in, any sensitive information types, files referenced, and sensitive files referenced — closer to what actually left the building than a permission scope gives you. Microsoft's documentation now labels that version classic, replaced by a newer Data Security Posture Management; the description above is of the classic version. Worth knowing before a board meeting, and worth being honest about: outside Copilot licensing, Microsoft's own deployment guidance describes other AI apps running on pay-as-you-go Purview billing, not a flat included capability. Confirm the current billing model before you promise your board a number.

05 / 11

Who approved it, and whether anyone looked

Every enterprise application in Entra ID carries a record of how it got there. Microsoft splits this into user consent, granted when a user signs in and accepts the prompt themselves, and admin consent, granted when a privileged administrator might grant an application access on behalf of other users (usually, on behalf of the entire organization). The Permissions page for each application separates the two tabs, so you can tell a self-service accept from a deliberate organisational decision.

What that record cannot tell you is whether the person granting it understood the request. A tenant-wide admin consent takes one click and covers every user in the organisation; Entra ID logs the fact of it, not the reasoning behind it. Our recommendation, and we will label it as ours: treat every tenant-wide consent to a new AI tool as a change that needs a named approver and a one-line reason, written down somewhere your board can see it later — because Microsoft's own record stops at who clicked.

06 / 11

Whether it trains on your data

Of the six questions, this is the one with a specific, checkable Microsoft statement — and it applies to exactly one product. Microsoft's documentation for Microsoft 365 Copilot — which that page says is now named Microsoft Copilot — states: Prompts, responses, and data accessed through Microsoft Graph aren't used to train foundation LLMs, including those used by Microsoft Copilot. That is a real, scoped commitment about Copilot's handling of your tenant's Graph data — not a statement about every AI product an employee might open in a browser.

For everything else — a chatbot someone found, an AI note-taker connected through OAuth, a coding assistant a developer signed up for — the honest answer depends on that vendor's own published terms, and nobody outside the vendor can state them for you. OpenAI, Anthropic and Google each publish their own data-use and training terms; reading the current one for the specific product and account tier in front of you is the only correct way to answer this, because those terms differ by product and plan, and they change. We will not characterise anyone's terms here — read the one for the tool your board is asking about, on the date you answer.

07 / 11

Whether you can turn one off

Technically, usually yes. Revoking an application's consent or blocking its ability to sign in removes it from the tenant going forward, from the same Entra ID screen where you found it. Microsoft's guidance on reviewing enterprise application permissions describes exactly that mechanism.

What it does not answer is whether the behaviour stops. If ten people were already using a personal or team-plan version of the same tool from a browser, revoking the Microsoft-connected version closes one entry point, not the underlying habit. Turning one off is a technical action; keeping it off is a governance question, and it is why a written approval workflow — one that records a tool as approved, conditional, or not approved, rather than leaving the decision to whoever clicked accept first — closes a gap a revoke button does not. That is our framing, not Microsoft's; Microsoft's documentation describes the mechanism, not the governance around it.

08 / 11

What your policy should say

Nothing inside Microsoft 365 writes this for you, and no product should claim otherwise. A one-page generative-AI acceptable-use policy is the cheapest artifact on this list, and the one a board actually asks to see. Our recommendation for what belongs on that page:

• Which AI tools are approved for company data, and which are not, by name
• Who can approve a new one, and what they check before saying yes
• What must never go into a prompt: client records, health information, credentials
• How an employee requests a new tool instead of just signing up for one
• How often the approved list gets reviewed, and by whom
• What happens if someone uses an unapproved tool anyway

09 / 11

The six questions, side by side

One page, columns labelled honestly, is what we would actually hand a board.

10 / 11

The frameworks the board may have already heard of

Two names come up often enough in board conversations that it helps to know what they actually are. NIST published the Artificial Intelligence Risk Management Framework, document NIST.AI.100-1, on 26 January 2023 — a voluntary framework for identifying and managing AI risk, not a certification, and not specific to any industry or company size.

ISO/IEC 42001:2023, published 18 December 2023, is titled Information technology — Artificial intelligence — Management system and specifies requirements for an organisation's own AI management system — the AI counterpart to ISO/IEC 27001 for information security. Avalon Web Services claims no certification against either standard, for this service or any other, and neither should anyone else without doing the actual work.

The UK's National Cyber Security Centre has its own entry point for exactly this audience: AI and cyber security: what you need to know, written for managers, board members and senior executives (with a non-technical background), published 13 February 2024, listing small and medium-sized organisations among its intended readers. If a director wants a primer that is not selling them anything, that is the one to hand over.

11 / 11

Where to start

Before the next board meeting, do the two-hour version of this yourself. Open Entra ID's enterprise applications list, check the consent tabs and, where licensed, the usage and insights report, and write down — in the same four columns as the table above — which of the six questions you can already answer and which are still an assumption. That single page, even half-finished, beats a paragraph of reassurance.

Disclosure: this is a category we sell into. Avalon CloudSec keeps a continuous inventory of enterprise applications and OAuth grants, including tenant-wide consents and the permissions each app was given, and classifies the AI tools it finds against a curated catalog as approved, conditional, or not approved, showing the reasoning behind each match. It cannot see AI usage inside a browser tab with no Microsoft sign-in — that needs a separate, optional network-based discovery add-on, and without it that use stays invisible to us as much as to you. It does not write your acceptable-use policy, and it cannot answer the training question for any vendor but Microsoft. To see what your own tenant's inventory already shows, email support@awservices.org.

Microsoft, Microsoft 365, Azure, Entra, Intune and Defender are trademarks of the Microsoft group of companies. Avalon CloudSec is an independent service and is not endorsed by Microsoft.