What Happens to Your Data When You Press 'Send' on an AI Tool
Most businesses are sending customer data, financials, and internal documents to AI tools without understanding what happens during processing. The spectrum of AI privacy protection is wider than you think; recent research shows that even purpose-built security can have structural flaws.
You paste a client email into ChatGPT to draft a reply. You upload a financial report to get a summary. You feed customer feedback into an AI tool to spot patterns. You ask Claude to review a contract clause.
Every one of those actions sends data to a server you do not control. And for most businesses, that is where the understanding stops.
Where does that data go? Who can access it during processing? Is it stored? Is it used to train future models? What happens if the server is compromised?
These are not hypothetical questions. They are architectural ones; the answers vary enormously depending on which tool you use, how it is configured, and what sits between your data and the outside world.
Most businesses have never asked these questions. If your business handles anything sensitive, client information, financial data, health records, legal documents, commercial-in-confidence material, you need to start.
The Default: Your Data Leaves Your Control
When you use a standard AI tool, ChatGPT, Claude, Gemini, or any of the dozens of industry-specific AI platforms, your data travels to a cloud server, is processed by the model, and a response is returned. The data exists, at least temporarily, on infrastructure you do not own and cannot inspect.
Most major providers now offer usage policies stating they do not train on your data (particularly on business and API tiers). That is a policy commitment. It is not an architectural guarantee. The data still travels to their servers. It still exists in memory during processing. The protection is a promise, not a mechanism.
For many business uses, this is perfectly acceptable. Drafting marketing copy, brainstorming product names, summarising public information. The data is not sensitive, and the convenience outweighs the risk.
But the moment your AI workflow involves client data, employee records, financial details, health information, or legal documents, the calculus changes. A policy that says “we do not train on your data” does not answer the question that actually matters: who can access this data while it is being processed?
The Spectrum of AI Privacy
AI privacy is not binary. It exists on a spectrum, and understanding where a tool sits on that spectrum is essential before you send it anything sensitive.
Level 1: No Protection
The AI tool processes your data on shared cloud infrastructure. Your data may be logged, stored, or used for model improvement. This is the default for free-tier consumer AI tools. It is not appropriate for any business data you would not publish on your website.
Level 2: Policy Protection
The provider commits, through terms of service or a data processing agreement, not to store, log, or train on your data. Enterprise tiers of major AI providers typically offer this. The protection is contractual. The data still travels to and is processed on their infrastructure, but you have a legal commitment regarding how it is handled.
For most business uses, this is sufficient. But “sufficient” depends entirely on what you are sending. For a marketing agency drafting social media posts, policy protection is more than adequate. For a medical practice processing patient notes, it may not be.
Level 3: Partial Architectural Protection
Some systems attempt to protect data architecturally, using encryption, secure enclaves, or trusted execution environments (TEEs) to ensure data is protected during processing, not just by policy.
However, not all architectural approaches are equal. Recent research from Rutgers University demonstrated a structural vulnerability in AI systems that split computation between a secure CPU enclave and an untrusted GPU. These “partial TEE” architectures use cryptographic noise to obfuscate data before it reaches the GPU, then remove the obfuscation when it returns.
The researchers showed that attackers could characterise the noise patterns through repeated queries, filter out the protection mathematically, and recover the protected data. In testing, the attack succeeded with a 100% success rate, recovering model secrets in approximately six minutes.
This is not a minor implementation bug. It is a structural flaw in the architecture itself; the kind of vulnerability that cannot be patched because it is inherent to the design approach.
The lesson is important: architectural protection is only as strong as the architecture. “We use encryption” or “we use secure enclaves” is not sufficient. The question is how comprehensive the protection is and whether the design creates exploitable patterns.
Level 4: Full Architectural Protection
Full TEE architectures protect data throughout the entire processing pipeline. Both the CPU and GPU operate within hardware-secured enclaves: Intel TDX for the CPU, NVIDIA H100 Confidential Computing for the GPU. Data is encrypted in memory at the silicon level. It never leaves the trust boundary. There is no unprotected handoff between components, which means there is no attack surface of the kind the Rutgers researchers exploited.
In a full TEE system, not even the platform operator can access the data during processing. The protection is enforced by hardware, not policy. This is the gold standard for AI privacy; it is what is required when the data is genuinely sensitive.
The Questions Your Business Should Ask
Before sending sensitive data to any AI tool, ask these five questions:
1. Where does my data go during processing?
Not where is it stored. Where does it travel? Which servers process it? In which jurisdiction? If the answer is vague, the protection is vague.
2. Who can access my data while it is being processed?
This is the question most privacy policies do not answer. Data at rest can be encrypted. Data in transit can be encrypted. But data during processing, data in use, is where most exposure occurs. Does the system protect data during inference, or only before and after?
3. Is the protection contractual or architectural?
A data processing agreement is a promise. A trusted execution environment is a mechanism. Both have value. But if your data would cause serious harm if exposed, client health records, legal strategy, financial modelling, you need to understand the difference.
4. Has the architecture been independently validated?
The Rutgers research is a reminder that security architectures can have structural flaws invisible to non-specialists. Has the system been reviewed by independent security researchers? Are the results published? If the vendor’s only evidence is their own assurance, that is worth exactly as much as any self-assessment.
5. What are the consequences if this data is exposed?
This is the question that determines where on the spectrum your business needs to sit. If exposed data would be embarrassing, policy protection is probably sufficient. If it would trigger regulatory action, breach notification obligations, or loss of professional registration, you need architectural guarantees.
The Bottom Line
Most businesses are sending data to AI tools without understanding what happens to it during processing. For non-sensitive work, that is fine; the convenience of AI outweighs a minimal privacy risk.
But if your business handles client data, health information, legal documents, or financial records, the default is not good enough. And “we take privacy seriously” is not an architecture.
The spectrum of AI privacy, from no protection through policy commitments to partial and full architectural guarantees, is wider than most businesses realise. Where your business needs to sit on that spectrum depends on your data, your industry, and the consequences of getting it wrong.
Privacy is an architecture decision, not a settings toggle, and architecture decisions determine output quality across every aspect of an AI implementation. It is one of the first things we evaluate in every AI opportunity analysis because the right AI solution for your business depends on what data it needs to touch and what the constraints around that data are.
Perth AI Consulting delivers AI opportunity analysis for small and medium businesses. Written report and working prototype, from $1,000. Start with a conversation.