Healthcare AI Needs Something HIPAA Doesn't Cover

Healthcare AI Needs Something HIPAA Doesn't Cover

Sovereignty Requirements Beyond Regulatory Compliance

Your hospital signed a Business Associate Agreement. Legal reviewed it. Compliance signed off. The cloud AI vendor is HIPAA-compliant. And if a US federal agency decides it wants your patients' clinical AI interactions, none of that matters—because the CLOUD Act overrides your contract.

The Health Insurance Portability and Accountability Act (HIPAA), enacted in 1996, regulates how protected health information (PHI) is stored, transmitted, and accessed. A BAA extends HIPAA to third-party vendors, making them legally liable for security and privacy breaches. But HIPAA was written before AI clinical applications existed. It says nothing about where AI models perform inference, who accesses the model weights, whether the model was trained on similar patients, or whether the reasoning chain is auditable to clinicians.

Sovereignty—the institutional right to know where data goes, how it is processed, and by whom—fills a gap that compliance alone cannot close. HIPAA protects the record. Sovereignty protects what happens when AI reads it.

The Compliance Assumption That Breaks Down

Healthcare organizations typically approach vendor risk through a three-step process: review the BAA, audit the infrastructure, and deploy. This works for database storage or file transfer systems. It fails for AI systems because compliance status is not a permanent attribute.

Consider clinical documentation. Nuance DAX, owned by Microsoft, processes physician-patient conversations. These conversations flow to Azure servers, where the AI model generates clinical notes. The hospital's HIPAA review checked the infrastructure in Month 1. In Month 3, Microsoft updated the underlying model. The organization has no automatic notification that the model changed, no right to review the new weights, no obligation on Microsoft's part to validate the new model against the same patient populations the original was trained on.

This gap exists at scale. A radiology AI trained on imaging datasets from other hospitals cannot be inspected for demographic bias. The model was built on chest X-rays from five different health systems, but the hospital deploying it cannot verify whether the training data accurately represents the patient population it now encounters. FDA SaMD guidance requires evidence of performance across patient subgroups, but a third-party AI vendor is not obligated to share training data with every customer.

Epic MyChart AI illustrates the infrastructure problem. This patient-facing AI runs on US cloud infrastructure, backed by a BAA. But "US cloud infrastructure" means AWS or Microsoft Azure, where physical servers reside in multiple geographic locations. The hospital cannot control where specific inference operations occur. A patient query might be processed in Virginia, Washington State, or a third location depending on load balancing.

The CLOUD Act, passed in 2018, creates a second layer of compliance failure. Federal agencies can compel any American company to produce data held anywhere in the world, regardless of what a BAA says. A hospital's contract with a cloud vendor cannot override US law. If the Department of Justice or the Department of Homeland Security issues a legal demand, the vendor must comply. The hospital has no veto.

How HIPAA Audits Miss the Real Risks

HIPAA audit is retrospective and permission-based. A covered entity can audit the vendor's logs within the 6-year retention window required by the Privacy Rule. But HIPAA does not require the vendor to retain inference logs showing what the AI model "saw" or "decided." Many vendors retain logs for 30 to 90 days for operational purposes only.

Malpractice litigation, by contrast, arrives 18 months later. A patient is diagnosed with an incorrect condition. The diagnosis was supported by an AI recommendation. The plaintiff's attorney will request the AI's reasoning chain, feature importance analysis, and data that fed the inference. If those logs no longer exist, the hospital cannot defend itself. And because the vendor is not obligated to retain them under HIPAA, retention policies are often vendor-driven cost decisions, not clinical ones.

NHS DeepMind's 2017 investigation by the Information Commissioner's Office established that a data processing agreement does not equal patient consent. The ICO found that even with proper BAAs, patients had no meaningful knowledge that their data flowed to a third party for purposes they did not anticipate. A hospital that signs a BAA for vendor X does not thereby give vendors Y and Z permission to access the same records if the vendor chain extends or changes.

Clinical queries that don't look like PHI individually become identifiable in aggregate. A patient asks a chatbot three questions about medication side effects, lab timing, and medication interactions. No single query contains a name or MRN. But the combination of the three questions, tied to a timestamp and user ID, re-identifies the patient when cross-referenced against the hospital's records. The AI vendor may never see the patient's name. The vendor's system logs show enough behavioral metadata that researchers could re-identify the person.

The Three Regulatory Timelines Converging

Healthcare organizations now operate at the intersection of three separate regulatory systems, each with its own enforcement schedule and penalties.

HIPAA remains the domestic baseline, with civil penalties up to $1.5 million per violation category per year. Violations are investigated and fined by the Department of Health and Human Services Office for Civil Rights. Enforcement is relatively slow; large fines typically follow repeated breaches and demonstrated negligence.

FDA Software as a Medical Device (SaMD) guidance, finalized in 2023, requires manufacturers of AI-assisted clinical tools to provide evidence of performance across patient populations, documentation of training data, and transparency about model limitations. FDA does not typically impose financial penalties, but the agency can require vendors to halt distribution, issue recalls, or conduct post-market surveillance. For hospitals deploying FDA-regulated AI, this means inventory obligations: know which patients used the AI, validate it worked as intended for their population, and maintain records linking patients to model versions.

The European Union AI Act, effective in 2026 for high-risk AI systems, imposes fines up to €35 million or 7% of global revenue, whichever is higher. "High-risk" includes AI used in clinical diagnosis, treatment recommendations, and patient monitoring. The EU AI Act applies to vendors providing services to EU residents, which includes many US healthcare organizations serving cross-border patient populations. A hospital operating on US infrastructure for US patients is in scope if it serves EU citizens or if the AI vendor is EU-based.

These three timelines are not sequential. They overlap. An organization deploying an AI system in 2026 must simultaneously satisfy HIPAA requirements (domestic privacy), FDA SaMD requirements (clinical safety), and EU AI Act requirements (algorithmic transparency and human oversight). Waiting until enforcement begins is waiting too late.

What Sovereignty Actually Means in Practice

Sovereignty is not a binary status. It exists on a spectrum defined by institutional control over five operational dimensions: location, visibility, ownership, auditability, and portability.

Location means the hospital specifies where data is processed. Instead of "any AWS region," the requirement becomes "US East Virginia only" or "only on servers that hospital staff can physically access." France's HDS certification, established in 2011, requires health data to be hosted by a certified French provider. This predates the AI era but demonstrates that sovereignty requirements can coexist with cloud infrastructure. A hospital might accept cloud deployment if cloud access is restricted to a private, isolated subnet that no other customer uses.

Visibility means the hospital can observe what the AI is doing with patient data. This includes access logs showing which users queried which records, inference logs showing what the model processed and when, and model change notifications when the underlying algorithm is updated. Epic and other EHR vendors can provide these logs; the decision to capture and retain them is organizational.

Ownership means the hospital retains rights to the AI model itself or has explicit contractual rights to the outputs. In practice, this typically means the hospital owns its training data, can request the model be fine-tuned on proprietary datasets, and can terminate the vendor relationship without losing the ability to audit past AI recommendations. Some vendors offer model escrow—a third party holds the model code in case the vendor fails or the relationship ends.

Auditability means an independent third party can verify the model's performance, training data composition, and decision logic. This is harder than it sounds. Many AI vendors treat model weights and training data as trade secrets. But a hospital can contractually require annual third-party audits even without access to raw weights, provided the auditor can query the model with test datasets and receive feedback on reasoning.

Portability means the hospital is not locked into a single vendor and can switch to an alternative if the first vendor fails to meet requirements or changes terms. This often requires the hospital to maintain its own inference infrastructure or to ensure that multiple vendors' APIs are compatible. Portability costs money upfront but preserves institutional independence.

Real Deployments That Encountered This

Intermountain Healthcare's AI for hospital readmission prediction runs on internal infrastructure, not a public cloud. The organization trained its own model on its own patient data and deployed it on-premises. This required Intermountain to hire AI engineers and maintain the infrastructure. The payoff is that Intermountain controls the model, can verify its performance on its population, and can pause or modify it without a vendor approval cycle.

Cleveland Clinic deployed AI for radiology reporting but negotiated a contract requiring the vendor to retain inference logs for 5 years, not 90 days. The additional cost was significant, but it allows Cleveland Clinic to defend against malpractice claims and to re-audit the AI if patient outcomes suggest something went wrong. The BAA was standard; the log retention rider was not.

The VA (Veterans Affairs) uses Epic on Microsoft Azure but required that all VA patient data be processed in VA-owned data centers, not in shared AWS or Azure infrastructure. This required Azure to provide dedicated instances. The VA accepted higher costs to maintain control. The result is that the VA cannot be compelled to hand over VA patient data by the CLOUD Act in the same way a commercial hospital can, because the data does not leave VA property.

These examples are not anti-cloud. They are pro-control. Intermountain uses the cloud for non-clinical data; it just keeps clinical AI infrastructure in-house. Cleveland Clinic uses a public cloud vendor but negotiated terms to retain institutional visibility. The VA accepts Azure but enforces a geographic and administrative boundary.

How to Start

An organization does not need to abandon cloud vendors to gain sovereignty. It needs to know what sovereignty means operationally, measure the current state against that definition, and close the gap deliberately.

Audit current deployments against the five dimensions. For each AI system in production, ask: Where does inference happen? What logs are retained? Does the vendor own the model or does the hospital? Can a third party audit performance? Can the hospital switch vendors? If any answer is "no" or "we don't know," that system has a sovereignty gap.

Update procurement criteria. For new AI vendors, contracts should explicitly specify location, log retention periods, rights to model audit, and conditions for portability. These are negotiable terms, not standard boilerplate.

Plan for EU AI Act compliance now. Organizations serving EU residents or using EU-based vendors should budget for compliance documentation, risk assessments, and human oversight mechanisms by late 2025. Waiting until 2026 means building compliance under enforcement pressure.

Establish a data residency policy that applies to all vendors, not just AI. This policy defines where patient data can be processed, who can access it, and what legal frameworks apply. HIPAA does not require this; institutional governance does.

Retain inference logs by policy, not by accident. Require vendors to hold AI interaction logs for at least 36 months—the typical statute of limitations for medical malpractice claims plus operational buffer. If a vendor will not retain logs, the hospital should ask why and whether the AI is truly suitable for clinical use.

Looking Forward: Sovereignty as Operational Strategy

EU AI Act enforcement begins in 2026. That is not theoretical—it is the active deployment timeline for high-risk systems. Healthcare organizations that have not addressed sovereignty requirements by the start of 2026 will be building compliant infrastructure under regulatory pressure instead of ahead of it.

The healthcare AI market is maturing. Early adopters focused on speed and ease of integration. The next phase focuses on control. Organizations that embed sovereignty requirements into procurement, architecture decisions, and vendor management now will find it easier to adapt to incoming regulations. Organizations that treat sovereignty as a compliance checkbox at the end of the deployment cycle will face costly retrofits.

Sovereignty and compliance are complementary, not contradictory. A hospital can achieve HIPAA compliance, FDA SaMD requirements, and EU AI Act requirements while using public cloud infrastructure—but only if the organization has deliberately designed systems to maintain visibility, control, and auditability over AI operations. HIPAA ensures the record is protected. Sovereignty ensures the hospital knows what the AI does with it.