Where client information goes when you use AI

The question usually arrives in a procurement questionnaire, and usually late: what happens to our information if you use AI on it? It is a fair question and it has an answer. What makes it awkward is that most organisations have not asked it of themselves first, so the honest response at that moment is that nobody is quite sure.

Three questions wearing one coat

“Is it secure” bundles three separate things together, and a vendor can answer yes to one of them while the one you needed was another:

• Who can see it: whether anyone at the provider can read what you send, and in what circumstances
• Whether it is kept: how long inputs and outputs are retained, and what deletion actually removes
• Whether it trains anything: whether your material improves a model, and whether that is on unless somebody turns it off

Those three have different answers for the same product depending on which plan it is bought on, which is the part that catches people out. A fourth question only shows up later: how will you know when any of those answers changes? Providers change products, terms and subprocessors, and what you were told at purchase is a snapshot.

The same name is often two different products

The consumer version of a tool and the business version frequently differ on exactly those points, and the difference is contractual rather than technical. Staff who sign up individually with a work address get the first set of terms.

This is the most common way material leaves an organisation without anyone having decided that it should. Not a breach, and not shadow IT in the dramatic sense. Just a free tier that is genuinely useful, used by somebody trying to get their work done.

The same applies to AI that appears inside software you already run. The feature looks like part of a system you have used for years, and the material may still be going to another provider behind it, on terms nobody read because nobody bought anything.

What to ask, and get in writing

• Where is the material processed and stored, and in which countries?
• What is the retention period for inputs and outputs, and can it be set to zero?
• Are our inputs used to train or improve models, by default or at all?
• Which subprocessors handle the material, and where do they operate?
• What does deletion actually remove, and how long does it take to take effect?
• Can an administrator enforce these settings, or does each user choose for themselves?
• How will we be told if the terms, the subprocessors or the data handling change?

Ask for the answers in writing and keep them with the contract. A screenshot of a settings page is not the same thing as a term somebody can be held to, and settings pages change.

Your obligation does not move

Under the Privacy Act 2020 an organisation that holds personal information remains responsible for it when a supplier processes that information on its behalf, and sending it overseas carries its own requirements about the protections that apply where it lands.

There is a third test that is not written in any contract: whether the way you are using it is what the client, or the person the information is about, would reasonably expect. Neither point is a reason to avoid these tools. Both are reasons to know which tool, on which terms, before the material is inside it. Where the material is sensitive, such as health information, anything concerning children, or anything covered by an undertaking, that is a conversation worth having with someone qualified to give the answer.

Decide what may not leave

The most useful thing an organisation can do here takes an afternoon. Sort your material into what may go to a third party under an agreement, and what may not leave your control at all. Most businesses have some of each, and the second category is usually smaller than people fear once it is written down rather than assumed.

That line is a design constraint, not a veto. Work on material that cannot be sent out can still be automated, with models that run where you control them, or by keeping the sensitive part of the task inside and sending out only the part that is not. What derails these projects is meeting the constraint after the design, rather than the constraint itself.

The unanswered question is already a policy

If nobody has said what is allowed, people decide for themselves. One person pastes client material into a public chatbot. Another decides the whole category is too risky and avoids it, including the parts that would have been fine. A third uses the approved tool with the settings left where they came. That is already an AI policy. It is simply an inconsistent one, and nobody chose it.

A short written position is worth more than a long policy nobody finishes: these tools, on these terms, for this kind of material, and ask before anything else.

It also turns that procurement questionnaire into something you can answer in a paragraph, which is worth something on its own.