Not legal advice · Summaries of vendors' public pages, which change often. Verify with the vendor and your contract before relying on anything here.
AI Terms Tracker
Get weekly changes
All tools / Why it matters

Why it matters if AI tools train on your company data

Using an AI tool and letting its vendor train on your data are two different things. When a vendor trains on your prompts, files or transcripts, that content helps shape a model that other customers use. Here's why that's worth deciding on purpose, in plain English, with a source for every claim.

Not legal advice. This page explains general risks and points to public sources. It isn't legal, compliance or security advice. How these rules apply depends on your contracts, industry and location, so check with your counsel.

Training on customer data isn't sinister. It's one way vendors improve their products, and some limit it, for example to non-generative models (see Zendesk AI). Most tools in this tracker don't train on business data by default. The point isn't to avoid AI tools. It's to make the training decision on purpose instead of by default. Of the 53 tools we track, 35 say they don't train on business data by default, 9 train by default on at least some business plans, and 9 don't say clearly. See the tracker.

1Training is hard to undo

Once your data trains a model, you can't pull it back out. Opting out later only stops future use.

Deleting a file or a chat removes it from the vendor's storage. It doesn't remove what a model already learned from it. HubSpot says so plainly on its own help page: "It is not possible to delete previously used data from trained models. Opt-outs will only apply moving forward."

Europe's data protection regulators make the same point in technical terms. The European Data Protection Board notes that information from training data may "still remain 'absorbed' in the parameters of the model" and may later be extractable from it.

So the opt-out decision matters most before people start using a tool, not after.

Sources:

2Models can memorize and repeat what they were trained on

Research shows AI models can reproduce pieces of their training data word for word, including contact details and code.

In 2021, researchers from Google, OpenAI, Stanford and others extracted hundreds of verbatim passages from GPT-2, including names, phone numbers, email addresses and source code. Some of those passages appeared in only one document in the training data.

Follow-up work found that memorization grows with model size, with how often a piece of text is repeated, and with how much context the person prompting provides. In 2023, a team extracted more than 10,000 training examples from ChatGPT (gpt-3.5-turbo) using about $200 of queries. The researchers disclosed the issue to OpenAI before publishing.

NIST's Generative AI Profile lists this as a recognized privacy risk and calls it "data memorization", which "may pose exacerbated privacy risks even for data present only in a small number of training samples."

To keep this in proportion: these studies were deliberate research attacks, mostly on models trained on public web text, and vendors can add safeguards. But from the outside, you can't check whether a vendor's safeguards would protect your contract terms, code or customer list. Keeping that data out of training in the first place removes the question.

Sources:

3You lose control once data leaves your tenant

After data leaves your systems, you depend on the vendor's policies, its subprocessors, and its future terms.

In 2023, Samsung engineers uploaded internal source code to ChatGPT. Samsung then temporarily restricted staff use of generative AI tools. According to Bloomberg and The Verge, its concern was that data sent to outside AI services sits on external servers, where it is hard to retrieve and delete and could end up disclosed to other users. (There's no public report that the code later appeared anywhere. The issue was losing control of it.)

OWASP's security guidance for AI applications lists "confidential business data" among the things AI systems can expose through their output.

Terms also change. The U.S. Federal Trade Commission has warned that it may be unfair or deceptive for a company to start using data for AI training and tell people only through "a surreptitious, retroactive amendment" to its terms. That's a warning to vendors, but it's also a reminder for buyers to keep watching the terms. That's why this site tracks changes.

Sources:

4It can conflict with your contracts and regulations

Customer NDAs and DPAs, GDPR, HIPAA and professional confidentiality rules can limit how data may be used.

Your customer contracts. Check whether your NDAs and data processing agreements with customers limit how their data may be used, including by vendors you rely on, and whether letting an AI vendor train on it fits the promises you've made.

GDPR. Personal data must be collected for specified purposes and "not further processed in a manner that is incompatible with those purposes". A processor may act "only on documented instructions from the controller", and a processor that decides its own purposes for the data is treated as a controller for that processing (Art. 28(10)). The EDPB also says AI models trained on personal data "cannot, in all cases, be considered anonymous".

HIPAA. Under HHS's sample business associate provisions, a business associate agrees to "not use or disclose protected health information other than as permitted or required by the Agreement or as required by law". Model training needs to fit inside what your BAA allows.

Client confidentiality. The American Bar Association's Formal Opinion 512 says lawyers need a client's informed consent before putting information about the representation into a self-learning AI tool, and that boilerplate language in an engagement letter isn't enough. If you work in another profession with confidentiality rules, check how they treat AI tools.

Sources:

5Trade secret protection depends on keeping things secret

Under U.S. law, information is only a trade secret if you take reasonable measures to keep it secret.

Under the federal Defend Trade Secrets Act, information qualifies as a trade secret only if "the owner thereof has taken reasonable measures to keep such information secret".

Letting an AI vendor train on your pricing models, formulas, source code or deal strategy doesn't automatically forfeit protection. But turning training off where you can, and having a written policy about it, makes it easier to show you took reasonable measures. Ask your counsel how this applies to you.

Sources:

6What "de-identified" does and doesn't protect

"De-identified" removes names, not the substance. Your strategy, numbers and code stay in.

Some vendors say they train only on "de-identified" or "anonymized" data. For example, Otter's privacy policy describes "training our proprietary AI technology on de-identified audio recordings and on transcriptions (which may contain Personal Information)". De-identification usually strips out names and other identifiers. It doesn't remove what was actually said: a meeting transcript about a pricing change or an acquisition is still about that pricing change or acquisition.

The word also has no single meaning. HIPAA has a specific standard, with two approved methods: removing 18 listed types of identifiers ("Safe Harbor") or a formal expert determination. A vendor's "de-identified" may not mean either one. Under GDPR, pseudonymised data is still personal data.

Re-identification is also easier than it sounds. A 2019 study in Nature Communications estimated that 99.98% of Americans would be correctly re-identified in any dataset using 15 demographic attributes.

Sources:

✓What to do about it

None of this means avoiding AI tools. It means choosing the training setting deliberately:

  1. Look up each AI tool your teams use in this tracker, and check the plan you actually pay for.
  2. Where training is on by default, have an admin turn it off before rollout, and keep a record of when you did.
  3. Prefer contracts and DPAs that say in writing that your data won't be used for training.
  4. Where a field here says Unknown, ask the vendor to confirm in writing before you put confidential data in.
  5. Match tools to data: keep regulated or client-confidential data in tools whose terms and agreements (BAA, DPA) cover it.
  6. Re-check when vendors change their terms. That's what the weekly "What changed" email is for.

All sources

Sources checked October 2026. Laws, guidance and vendor terms change, and we'll update this page as they do.