HIPAA Compliant AI, GDPR and SOC 2 for FDEs: A Practical Compliance Guide

Why Compliance Is an FDE Skill

An FDE sits with the customer, sees the real data, and ships the real system, so compliance questions land on your desk early, often before legal has weighed in. Can this model see patient records? Does a vendor's certificate cover what we are building? Details also differ by project, as covered in how FDEs help enterprises deploy AI.

The practical skill is knowing enough about HIPAA, GDPR, and SOC 2 to make good architecture decisions and to ask the right questions of compliance teams. This builds on the broader idea of AI governance for forward deployed engineers, where governance is a design constraint rather than a launch-day checkbox. It matters most in the regulated sectors covered in which industries need forward deployed engineers most, and for a sector-specific view, see forward deployed engineering in healthcare AI.

The Three Frameworks at a Glance

The three are often mentioned together but are very different things.

‍

Keeping these straight prevents the most common mistake in customer meetings: treating one as a substitute for another.

HIPAA Compliant AI: What It Actually Means

HIPAA applies when a covered entity (such as a provider or health plan) or its business associate handles protected health information. If your AI system touches PHI, you and your AI vendors are usually business associates.

The Business Associate Agreement

A BAA is a contract in which a vendor agrees to safeguard PHI under HIPAA. Without a signed BAA, a vendor should not receive PHI at all. Practitioner guides stress three points:

• A BAA covers only what it names. Consumer-tier products and some surfaces, such as consoles, playgrounds, or beta features, are often excluded.
• "HIPAA-ready" or "HIPAA-eligible" is not the same as compliant. Compliance depends on the plan, the signed BAA, and the settings you enable. As one comparison puts it, no major AI model is HIPAA compliant by default.
• New models and previews may not be covered yet. Check coverage each time you switch.

Technical and Operational Safeguards

Beyond the contract, HIPAA expects safeguards. In AI projects these usually translate into:

• Minimum necessary data. Send the model only what the task needs.
• De-identification where possible. HIPAA recognises two methods: Safe Harbor (removing specified identifiers) and Expert Determination (a qualified expert certifies low re-identification risk). Free-text fields can still leak identity through rare conditions or timelines, so test for re-identification.
• Encryption at rest and in transit, with managed keys.
• Access control: least privilege, role-based access, strong authentication.
• Audit logging of who accessed what and when, with the logs themselves secured, since prompts and outputs may contain PHI.
• Treat derived data as sensitive. Embeddings generated from PHI should be handled as PHI. Many vector databases do not offer BAAs by default, so verify before storing them.
• No unplanned reuse. Do not move patient data into your own systems or use it to train models outside the engagement's scope.

GDPR: What Changes for AI Systems

GDPR applies when you process personal data of people in the EU, wherever your company is based. The roles matter: the customer is usually the controller (decides why and how data is used) and you or your vendor may be a processor (acts on instructions).

Key ideas for AI work:

• Lawful basis. There must be a valid legal ground for processing, such as consent or legitimate interests. "We want to build an AI feature" is not a basis on its own.
• Data minimisation and purpose limitation. Collect and use only what is needed for the stated purpose.
• Data subject rights. People can ask for access, correction, and erasure. This is hard when personal data has been used in prompts, logs, fine-tuning sets, or vector stores, so design deletion paths early.
• Data processing agreement (DPA). A contract with each processor, including AI vendors, that sets out how data is handled. Most major AI providers offer DPAs to business and API customers.
• Data protection impact assessment (DPIA). Often required for high-risk processing, which can include large-scale or novel uses of AI.
• International transfers. Moving personal data outside the EU needs a valid transfer mechanism. Data residency options, such as regional storage or region-pinned processing, help but need to be verified for each provider and configuration.
• Automated decisions. Rules apply to decisions made solely by automated means with significant effects on people. Human review matters here, which connects to LLM guardrails and human-in-the-loop design.
• The EU AI Act. Separate from GDPR, it adds obligations for certain AI uses. Check whether your use case falls within it.

SOC 2: What It Is and What It Is Not

SOC 2 is an audit framework from the AICPA. An independent auditor evaluates a company's controls against the Trust Services Criteria: security, availability, processing integrity, confidentiality, and privacy. Only security is mandatory; the others are included at the company's choice.

Two report types matter:

• Type I assesses whether controls are designed appropriately at a point in time.
• Type II assesses whether controls operated effectively over a period, which is why enterprise buyers usually ask for it.

Three misunderstandings come up in customer conversations:

1. SOC 2 is not a law and not a certification. It is an auditor's report about a vendor.
2. A vendor's SOC 2 covers the vendor's platform, not your deployment. Your own configuration, access policies, and data flows still need to be sound.
3. It does not replace a BAA or a DPA. A vendor can hold a clean SOC 2 report and still be unsuitable for PHI without a signed BAA.

For FDEs, SOC 2 is mostly a vendor due-diligence tool: you request the report, read the scope and exceptions, and check whether the relevant criteria and services are covered.

Controls That Help Across All Three

Despite their differences, the frameworks reward similar engineering habits:

• Data mapping. Know what data enters the system, where it goes, and where it is stored, including logs, caches, embeddings, and backups.
• Least privilege. Give users, agents, and tools the minimum access they need. This matters especially for agents that call tools; see MCP for forward deployed engineers on connecting AI to enterprise systems safely.
• Zero data retention (ZDR) where available. Some vendors offer agreements under which prompts and outputs are not stored after processing. Confirm what exactly is covered, since some models or features carry mandatory retention.
• No training on customer data unless explicitly agreed.
• Redaction and de-identification before data reaches a model whenever the task allows.
• Audit trails that are themselves access-controlled, and human review for high-stakes outputs.
• Deployment choice. Where compliance demands, run models inside the customer's own cloud boundary or on-premises. The strictest case is air-gapped AI deployment.

A Practical Checklist for Deploying an LLM in a Regulated Setting

1. Identify the data and the jurisdiction. Does it include PHI, EU personal data, or both?
2. Define roles. Who is the covered entity, controller, business associate, or processor?
3. Get the contracts right. BAA for PHI, DPA for personal data, signed before any real data flows.
4. Confirm coverage. Check that the specific model, plan, region, and features you will use are within the agreement.
5. Review vendor assurance. Request SOC 2 Type II reports and read scope and exceptions.
6. Minimise and de-identify. Reduce what reaches the model.
7. Design data flows. Include logs, embeddings, caches, and retention periods.
8. Plan deletion and access requests. Make sure data can be found and removed.
9. Secure the stack. Encryption, least privilege, authentication, network controls.
10. Log and monitor. Keep audit trails and review them.
11. Add human oversight for consequential outputs.
12. Document decisions and involve legal early, confirming interpretations with qualified counsel.

This fits naturally into discovery in the FDE project lifecycle, where constraints are cheaper to settle than to retrofit.

A Worked Scenario: A Clinical-Notes Summariser for a Hospital Group in Europe and the U.S.

A hospital group wants an assistant that summarises clinician notes. Notes contain PHI for U.S. patients and personal data about EU patients.

The FDE starts by mapping data and roles. For U.S. data, the group is a covered entity, so the vendor needs a signed BAA that covers the exact model and surface. For EU data, the group is a controller, so a DPA is needed and a DPIA is likely. Residency options are checked so EU data is processed in the EU.

Next come design choices. Notes are de-identified before reaching the model where possible. Summaries are logged in a secured store with access limited to treating staff. Embeddings, if used, are treated as sensitive and stored only in a service that supports the required agreement. A deletion process handles erasure requests. A clinician reviews each summary before it is saved to the record. The vendor's SOC 2 Type II report is read for scope, but it is treated as one input, not the answer.

Common Mistakes

• Equating "HIPAA-eligible" with compliant. Eligibility is a starting point.
• Using consumer tools with real data. Pasting PHI or personal data into a consumer chatbot is a risk regardless of intent.
• Assuming a vendor's SOC 2 covers you. It covers their platform, not your setup.
• Forgetting derived data. Logs, embeddings, and caches count.
• Over-trusting a prompt instruction. "Do not reveal personal data" in a system prompt is not a control. Enforce it in code and permissions

TL;DR: "HIPAA compliant AI" is not a product you buy. It is a way of deploying an AI system so the right contracts, controls, and records exist around it. HIPAA governs U.S. health data and centres on a Business Associate Agreement. GDPR governs personal data of people in the EU and centres on lawful basis, data subject rights, and processor agreements. SOC 2 is an audit report on a vendor's security controls, not a law. Forward Deployed Engineers need to know which framework applies, who is responsible for what, and how each shapes architecture choices such as data flow, logging, retention, and where models run. This guide is engineering-oriented and is not legal advice

‍