AI governance in healthcare needs an operating model

AI governance cannot remain a policy document that a committee reviews before launch. It has to become part of how a health system selects technology, prepares workflows, validates performance, responds to incidents, and decides whether an AI tool should remain in use.

The need is immediate. In a 2024 survey of 43 large nonprofit health systems, every responding organization reported some adoption activity around ambient clinical documentation. Yet only 53% reported a high degree of success in clinical documentation. Respondents cited immature AI tools as the leading barrier to adoption, followed by financial concerns and regulatory uncertainty. The study was self-reported and limited to members of the Scottsdale Institute, so its results should not be treated as a national prevalence estimate. It still shows a useful distinction. Adoption activity is not the same as dependable operating value.

For healthcare executives, the central question is no longer whether the organization has an AI committee. The question is whether the organization has repeatable controls that follow each AI use case through its full operating life.

AI adoption is moving faster than operational control

A national study published in JAMA Network Open analyzed responses from 2,174 nonfederal acute care hospitals to the 2024 American Hospital Association Information Technology Supplement. After weighting, 31.5% of hospitals reported current use of generative AI integrated with the electronic health record, while another 24.7% planned to use it within a year.

Those figures measure reported adoption, not proven benefit. They also cover generative AI integrated with the EHR rather than every AI tool used across a health system. Even with those limits, the study shows why governance has to extend beyond the technology department. AI is entering documentation, patient communication, access, revenue-cycle work, analytics, and other workflows that carry different levels of clinical, financial, privacy, and patient-experience risk.

A 2026 systematic review in npj Digital Medicine reached a related conclusion. The authors reviewed 35 AI implementation frameworks published from 2019 through 2024 and identified seven recurring governance domains. These covered organizational structure, problem definition, external tool evaluation, development, validation, deployment, and ongoing monitoring. The authors also found that many frameworks assume the expertise and resources of a large academic health system. Their proposed maturity model is a framework derived from the literature, not evidence that any specific governance design improves outcomes. Its practical value is the recognition that governance must match both the risk of the use case and the resources of the organization.

What an operating model for healthcare AI governance should do

The following six controls synthesize the recurring needs in the peer-reviewed evidence and the Joint Commission and Coalition for Health AI guidance. They are not a substitute for current legal, regulatory, clinical, privacy, security, or accreditation requirements. They are a practical structure for turning those requirements into daily work.

1. Maintain one inventory of AI use cases

A health system cannot govern tools it has not identified. The inventory should include purchased systems, embedded EHR capabilities, internally developed models, generative AI assistants, automated patient communication, analytics, and staff use of general-purpose tools.

For each use case, record its intended purpose, users, data sources, vendor or owner, affected workflow, patient impact, and current status. This makes shadow use visible and gives leaders a common view of what is being tested, deployed, changed, or retired.

2. Classify risk by use and consequence

The same model can create different risk depending on what it does. Drafting an internal summary is not equivalent to recommending clinical action. Routing a routine administrative request is not equivalent to determining whether a patient receives urgent follow-up.

Risk classification should consider proximity to patient care, the consequence of error, the sensitivity of the data, the degree of automation, the ability of a person to detect a mistake, and whether the system communicates directly with a patient. The classification should determine the depth of review, validation, monitoring, and human oversight.

3. Assign an accountable operating owner

A multidisciplinary committee can set policy and escalation rules, but every use case still needs a named owner who is responsible for performance in the actual workflow. That owner should have the authority to pause use, investigate failures, coordinate with vendors, and bring material issues to clinical, quality, privacy, security, legal, compliance, and executive leaders.

Governance weakens when responsibility is shared so broadly that nobody owns the outcome.

4. Validate the tool in the local workflow

Vendor evidence is necessary, but it does not answer whether a tool works with the organization’s patients, data, staffing model, escalation rules, terminology, and operating environment.

Local validation should test the outcome that matters in the workflow, not only technical accuracy. For a patient communication tool, that may include successful resolution, unsafe or misleading responses, escalation accuracy, language performance, patient comprehension, and effects on staff workload. For an operational prediction, it may include false positives, false negatives, subgroup performance, and whether staff can act on the output in time.

5. Govern the knowledge and data behind the answer

Many healthcare AI failures begin before the model produces an output. The underlying policy may be outdated. The scheduling rule may conflict across locations. A data feed may be incomplete. A general model may lack the local context needed to give a safe answer.

Responsible AI in healthcare therefore requires clear source ownership, version control, access rules, update processes, and traceability. People and AI systems should use current, approved knowledge rather than disconnected files, stale instructions, or tribal memory.

6. Monitor performance and create an incident path

Approval before deployment is only the beginning. Models change. Vendors release updates. Workflows shift. Input data drifts. Staff may use the tool in ways that were not anticipated.

The Joint Commission and Coalition for Health AI guidance calls for risk-based ongoing quality monitoring, defined responsibility, regular testing, current data, reporting of adverse events or recurring errors, and feedback channels with vendors. Health systems can often connect this work to existing quality, patient-safety, compliance, and incident-reporting structures rather than create a separate bureaucracy.

Monitoring should answer a direct question. Is this tool still producing safe, useful, equitable, and operationally valuable results in the setting where it is used?

Governance should protect human judgment, not freeze innovation

Poor governance creates two opposite risks. One is uncontrolled adoption. The other is a review process so broad and slow that teams route around it.

A risk-based operating model offers a better path. Lower-stakes administrative uses can move through a lighter process with defined safeguards. Higher-stakes clinical or patient-facing uses should receive deeper local validation, closer monitoring, and clearer human escalation. The goal is not identical review for every tool. The goal is review that matches the consequence of failure.

This approach also keeps the workforce in the design. AI can reduce repetitive work, surface information, draft responses, and identify patterns. People still provide judgment, empathy, clinical context, accountability, and complex problem-solving. Training and feedback should help staff understand what the tool does, what it does not do, when to question it, and how to report a concern.

The PEC360 perspective

Patient experience AI sits inside operating systems, not outside them. A scheduling answer depends on current access rules. A patient call may require accurate policy, workflow, and escalation knowledge. Interaction analysis is useful only when leaders can connect the signal to coaching, process improvement, or corrective action.

PEC360’s view is that AI, automation, enterprise knowledge, and human expertise should work as one operating system for patient experience. That makes governance more concrete. Leaders should be able to identify the source behind an answer, define where human judgment remains required, monitor what happens in real interactions, and improve the workflow when evidence shows friction or risk.

AI governance in healthcare will mature when it becomes ordinary operating work. The organizations that make that shift will be better positioned to move quickly without confusing speed with readiness.

Sources

1. Poon EG, Lemak CH, Rojas JC, Guptill J, Classen DC. Adoption of artificial intelligence in healthcare: survey of health system priorities, successes, and challenges. Journal of the American Medical Informatics Association. Published April 24, 2025. Cross-sectional survey of 43 completed responses from 67 Scottsdale Institute member health systems. Findings are self-reported and the sample is not nationally representative.
2. Everson J, Nong P, Richwine C. Uptake of generative AI integrated with electronic health records in US hospitals. JAMA Network Open. Published December 12, 2025. Survey study using responses from 2,174 nonfederal acute care hospitals in the 2024 AHA IT Supplement, with a 51.5% response rate and weighted estimates. The study measures reported EHR-integrated generative AI adoption, not clinical or financial benefit.
3. Hussein R, Zink A, Ramadan B, et al. Advancing healthcare AI governance through a comprehensive maturity model based on systematic review. npj Digital Medicine. Published February 11, 2026. Systematic review of 35 frameworks published from 2019 through 2024. The HAIRA model is derived from the reviewed literature and requires prospective validation.
4. Joint Commission and Coalition for Health AI. Guidance on the responsible use of AI in healthcare. Released September 17, 2025. Consensus guidance for healthcare organizations. It is guidance, not evidence that a specific control improves outcomes.
5. Coalition for Health AI. AI governance playbooks. Current framework covering policy, organizational structure, resources, lifecycle management, risk assessment, data management, third-party management, and education and feedback.
6. World Health Organization. Ethics and governance of artificial intelligence for health: guidance on large multi-modal models. WHO guidance for governance of large multimodal models in health. It provides principles and recommendations rather than tested estimates of operational impact.