HIPAA-Compliant AI — Deployment Playbook (2026)

HIPAA isn't the highest bar in healthcare AI compliance — state medical-board guidance, FDA SaMD rules, and provincial Canadian regimes all add layers. But it's the foundation, and if you get HIPAA wrong, nothing else matters.

1. Who HIPAA applies to (and how AI changes that)

HIPAA covers covered entities (healthcare providers, health plans, healthcare clearinghouses) and business associates (anyone who creates, receives, maintains, or transmits PHI on behalf of a covered entity). When AI enters the picture, both the AI vendor and any sub-processors in the AI pipeline are business associates and need their own BAA chain.

The 2023 HHS guidance on tracking technologies and the subsequent 2024-25 enforcement actions made it clear: tools that don't look like “health records” can still be subject to HIPAA if they handle data that identifies an individual's relationship to a healthcare provider. AI agents in healthcare workflows are squarely covered.

2. The three rules, mapped to AI

2.1 Privacy Rule

Governs what can be done with PHI. Most relevant to AI: authorization for non-treatment uses, minimum-necessary, individual access rights, accounting of disclosures. AI systems that surface information to non-treatment users (administration, research, marketing) must satisfy stricter authorization or de-identification requirements.

2.2 Security Rule

The largest source of control requirements for AI deployments. Administrative, physical, and technical safeguards: access management, audit controls, integrity, person-or-entity authentication, transmission security. Each one needs an explicit architectural answer in the AI system.

2.3 Breach Notification Rule

If unsecured PHI is disclosed outside the permitted set, the covered entity must notify affected individuals, HHS, and (for breaches affecting 500+) the media. The AI angle: a prompt leaking PHI to a public model API is a potential breach. We architect against the conditions that produce this.

3. BAAs for AI vendors and sub-processors

A typical healthcare AI deployment requires BAAs with:

The cloud provider (AWS, Azure, GCP) — they all offer HIPAA-eligible enterprise terms
The model vendor where applicable — Anthropic, OpenAI, Google offer enterprise BAAs through Bedrock, Azure OpenAI, Vertex AI
The vector database — Pinecone, Qdrant Cloud, and most managed vector DBs offer BAAs; self-hosted vector stores avoid the question
The observability vendor if logs may contain PHI (Datadog, New Relic, Sentry all offer BAAs)
Any third-party tool the agent calls that may see PHI

Agrus produces the BAA inventory as part of the compliance map during the Discovery Sprint. We have template BAA language for the AI-specific terms (no training on customer data, no cross-tenant retention, audit-log delivery, sub-processor disclosure) that legal teams find familiar.

4. Minimum-necessary applied to LLM prompts

The minimum-necessary standard requires that only the PHI actually needed for the task be disclosed. For LLM-based agents, this principle has practical architectural consequences:

Retrieval must scope tightly. Don't put the full patient record into context when the task only needs a vitals timeline.
Prompt templates audited. Templates that include “...and any other relevant information” fail minimum-necessary. Templates must be explicit.
De-identification where possible. When the task doesn't require identifiers, strip them on the way in. Some workflows allow this; many don't.
Per-user authorization. Different agent users see different PHI scopes based on their role and the patient relationship. The agent enforces it; the access layer double-checks.

5. Audit log requirements (Security Rule §164.312(b))

The Security Rule requires audit controls that record and examine activity in systems containing ePHI. For AI systems, each agent decision should produce a log entry containing:

Timestamp (server time + user time-zone)
Initiating user identity and role
Patient identifier (where applicable)
PHI elements accessed in the retrieval step
Prompt structure (template + parameters; not necessarily the full text if storage is sensitive)
Model identity and version
Model response summary
Downstream actions taken (none, surfaced to clinician, escalated, etc.)

Retention: six years from the date of creation or last effective date. Log destination: customer's SIEM (Splunk, Sumo Logic, Datadog, Microsoft Sentinel) under their existing retention policy. Not our logs.

6. Breach notification: the AI failure modes

The breach risks specific to AI deployments:

Prompt-injection exfiltration. A user-supplied input convinces the agent to disclose PHI to an unauthorized destination. We design against this with output filtering, destination allow-lists, and explicit PHI scrubbing on outputs.
Model training on customer data. An AI sub-processor inadvertently trains on customer prompts. BAAs explicitly prohibit this; architecture ensures it can't happen quietly.
Cross-customer log leakage. Multi-tenant observability mixing customer A's logs into customer B's reports. We architect for single-tenant logging where PHI is involved.
Excessive retention. Conversation history kept beyond purpose. Retention policies enforced programmatically.

7. The Agrus HIPAA reference architecture

Our default healthcare AI architecture:

Deployment shape: VPC inside the customer's AWS/Azure/GCP account, or dedicated cloud tenancy
Model: open-weights LLM (Llama 3.3 70B, Qwen 2.5 72B, or DeepSeek-V3) for routine queries, with a Claude/GPT private tenancy for reasoning-heavy long-tail queries
Retrieval: Postgres + pgvector or self-hosted Qdrant, with per-patient and per-user access scopes enforced in the query layer
PHI scrubbing: structured on input (template enforcement) and on output (regex + ML-based PHI detection)
Audit pipeline: OpenTelemetry-instrumented inference layer feeding into the customer's SIEM
Authentication: SSO via the customer's existing IdP, with per-role authorization
Encryption: at rest (AES-256), in transit (TLS 1.3), and key management via the customer's KMS

Full deployment treatment in our Private LLM Deployment Guide.

Next step

Get a HIPAA compliance map for your AI use case.

30-minute call with our compliance lead and a healthcare-AI engineer. Free.

Book a scoping call Talk to a persona

Frequently asked questions

Is ChatGPT or Claude HIPAA compliant?

Neither consumer endpoint (chatgpt.com, claude.ai) is appropriate for PHI. Anthropic and OpenAI both offer enterprise tenancy with HIPAA-eligible configurations — Claude through AWS Bedrock and Azure-hosted enterprise tenancy, GPT through Azure OpenAI under enterprise terms with a signed BAA. The HIPAA eligibility is contract-and-architecture-dependent, not a product feature you can buy as a single seat.

What is a BAA for an AI vendor?

A Business Associate Agreement is the HIPAA-mandated contract between a covered entity and any vendor that handles PHI on its behalf. For AI deployments, BAAs are typically required with: the model vendor (or cloud provider hosting the model), the vector database vendor, the inference infrastructure provider, observability/monitoring vendors that see PHI in logs, and any sub-processor in the data pipeline. Agrus produces a complete BAA list as part of the compliance map before any PHI flows.

Does minimum-necessary apply to LLM prompts?

Yes. The HIPAA minimum-necessary standard requires that only the PHI actually needed for the purpose be disclosed. Applied to LLMs, this means prompts should contain only the PHI required for the specific decision the agent is making — not the full patient record because retrieval was lazy. Our retrieval pipelines explicitly enforce minimum-necessary at the retrieval stage, not just at the model-prompt stage.

What audit logs does HIPAA require for an AI system?

HIPAA Security Rule §164.312(b) requires audit controls that record and examine activity in systems containing ePHI. For AI systems, this means: who initiated the query, what PHI was accessed, what the model returned, who reviewed it, what action was taken, and the timestamp on each. Retention is six years from the date of creation or last effective date. The audit log destination is the customer's SIEM, not ours.

Are open-weights models like Llama HIPAA compliant?

Open-weights models themselves aren't compliant or non-compliant — they're software. The deployment is compliant or not. An open-weights model running in your VPC with proper access controls, audit logging, encryption at rest and in transit, and a documented control set against the HIPAA Security Rule can be a fully compliant deployment. We do this routinely; it's our most common healthcare deployment pattern.

HIPAA-compliant AI, by architecture.