要旨
人工知能は加速度的に臨床医療に参入しており、放射線医学トリアージ、病理スクリーニング、敗血症予測、薬物相互作用アラート、手術計画システムなどはすべて、患者ケアの決定に影響を与えるか、直接的に決定を下す AI エージェントを導入しています。しかし、これらの展開を取り巻くガバナンス インフラストラクチャは、驚くほど非公式なままです。ほとんどの臨床 AI システムは事後モニタリングで動作します。つまり、エラーは臨床ワークフローを通じて伝播した後に検出され、患者に到達する可能性があります。臨床 AI には正式な実行前の安全保証が存在しないということは、業界が適切に対処していない患者安全の危機を表しています。
This paper introduces the Hippocratic Gate — a fail-closed governance primitive that enforces the constraint S(a) >= theta for every clinical AI action a, where S is a multi-factor safety function and theta is a risk-tier-dependent threshold. The Hippocratic Gate operationalizes the ancient medical principle 'primum non nocere' (first, do no harm) as a mathematical invariant: no clinical AI action may proceed unless it can demonstrate, through computable evidence, that its expected benefit exceeds its expected harm by a margin sufficient for the action's clinical risk tier.
We make the following contributions. First, we construct the safety function S(a) as a composite of seven clinical safety factors — diagnostic confidence, evidence concordance, contraindication clearance, temporal stability, population applicability, reversibility index, and human oversight readiness — and prove that S(a) >= theta implies an upper bound on misdiagnosis probability that decreases monotonically with theta. Second, we derive this upper bound analytically: for a gate threshold theta and safety function with Lipschitz constant L_S, the misdiagnosis probability satisfies P(misdiagnosis | S(a) >= theta) <= (1 - theta)^2 / (L_S * theta), yielding P < 0.0003 for theta = 0.85 on clinical-grade safety functions. Third, we define evidence bundle requirements for four clinical risk tiers (routine monitoring, diagnostic assistance, treatment recommendation, autonomous intervention) and specify the minimum evidence dimensionality for each tier. Fourth, we model temporal safety dynamics — how the safety function evolves as patient state changes — and derive conditions under which a previously safe action becomes unsafe, triggering automatic gate re-evaluation. Fifth, we present a complete integration architecture with MARIA OS, including healthcare-specific gate configurations, HIPAA-compliant audit trails, and regulatory alignment mappings for FDA Software as a Medical Device (SaMD) classification, EU Medical Device Regulation (MDR), and HIPAA Security Rule requirements.
この記事では、ヒポクラテスの門が実際にエスカレーション動作、証拠要件、待ち時間バジェットをどのように変えることができるかを示すために、記事全体で例示的な放射線医学再生シナリオが使用されています。これらのシナリオ番号は、制御設計を具体化することを目的としています。これらは、将来の臨床検証や管轄区域固有の規制レビューの代替として読まれるべきではありません。
The core thesis of this work is that patient safety in AI-assisted medicine is not a training problem — it is a governance problem. No amount of model fine-tuning can guarantee that every clinical action is safe, because safety depends on context that the model cannot fully observe: patient history, concurrent treatments, institutional protocols, and the evolving clinical state. The Hippocratic Gate addresses this by requiring positive evidence of safety before every clinical action, shifting the burden of proof from 'show me the harm after it happens' to 'prove safety before you act.'
1. The Patient Safety Crisis in AI-Assisted Medicine
1.1 The Scale of the Problem
Medical errors are the third leading cause of death in the United States, responsible for an estimated 250,000 deaths annually. Diagnostic errors account for approximately 40,000 to 80,000 of these deaths. The introduction of AI into clinical workflows presents both an opportunity to reduce these errors and a risk of introducing new failure modes that existing clinical governance structures are not designed to detect.
Consider the current landscape of clinical AI deployments:
- Radiology AI: Over 500 FDA-cleared AI algorithms for medical imaging are now commercially available. These systems analyze chest X-rays, mammograms, CT scans, and MRIs to detect conditions ranging from pneumothorax to intracranial hemorrhage. A single missed finding or false positive can cascade into incorrect treatment, delayed intervention, or unnecessary invasive procedures.
- Clinical Decision Support (CDS): AI-powered CDS systems generate alerts for drug interactions, sepsis risk, deterioration prediction, and treatment recommendations. Alert fatigue — where clinicians receive so many alerts that they begin ignoring them — is already a documented patient safety concern. Adding AI-generated alerts without governance infrastructure exacerbates this problem.
- Pathology screening: Digital pathology AI analyzes tissue samples for cancer detection, grading, and biomarker expression. A false negative in cancer screening can delay diagnosis by months, fundamentally altering patient prognosis.
- Surgical planning: AI systems recommend surgical approaches, instrument selection, and anatomical navigation paths. Errors in surgical planning can result in intraoperative complications, organ damage, or incomplete tumor resection.
In each domain, the AI system operates as a clinical decision agent — an entity that produces recommendations or actions that directly influence patient care. The question is not whether these systems make errors (they do, at rates comparable to or lower than human clinicians for narrow tasks) but whether the governance infrastructure can prevent errors from propagating to patients.
1.2 The Governance Gap
Current clinical AI governance relies on three mechanisms, all of which are insufficient:
Pre-market regulatory clearance (FDA 510(k), De Novo, PMA): Regulatory clearance validates that an AI system performs adequately on a test dataset at a specific point in time. It does not guarantee ongoing safety in production, does not account for distribution shift in patient populations, and does not enforce per-decision safety checks. A system that was safe on the validation dataset may become unsafe when deployed in a different patient population, a different clinical workflow, or a different institutional context.
Post-market surveillance (MAUDE, MDR reporting): Adverse event reporting systems detect problems after they have harmed patients. The median time from adverse event to corrective action in medical device surveillance is measured in months to years. For an AI system making thousands of clinical decisions per day, post-market surveillance is a retrospective autopsy, not a safety mechanism.
Human-in-the-loop (HITL) oversight: Most clinical AI systems are deployed with the assumption that a human clinician reviews every AI recommendation before acting on it. This assumption breaks down in practice. Automation bias — the tendency for humans to defer to automated recommendations — is well-documented in clinical settings. Studies show that clinicians agree with AI recommendations 85-95% of the time, even when the AI is deliberately wrong on 20-30% of test cases. The HITL assumption provides a regulatory fiction of human oversight without delivering actual safety.
The governance gap is clear: regulatory clearance validates the model, not the deployment; post-market surveillance detects harm, not risk; and HITL oversight assumes human vigilance that automation bias undermines. What is missing is a pre-execution safety enforcement mechanism — a governance primitive that evaluates every clinical AI action against a formal safety criterion before the action reaches the clinical workflow.
1.3 ヒポクラテスの命令法
医学におけるヒポクラテスの伝統、つまり「primum non nocere」、まず害を及ぼさないということは、単なる倫理的願望ではありません。それは設計原則です。すべての臨床介入は利益とリスクの計算を満たさなければなりません。つまり、患者に期待される利益が期待される害を上回っていなければなりません。この計算は、すべての治療決定について臨床医によって暗黙的に実行され、研究プロトコールの施設審査委員会によって明示的に実行されます。
AI エージェントが臨床ワークフローに参入する場合、この利益とリスクの計算をそのアクションに対しても実行する必要があります。ヒポクラテスの門はこの要件を形式化しています。すべての臨床 AI アクション a について、システムは多次元の利益とリスクの評価を捉える安全関数 S(a) を計算する必要があり、アクションは S(a) >= シータの場合にのみ続行できます。シータはアクションの臨床リスク層に合わせて調整されます。
This is not a conservative design choice that throttles AI performance. It is a necessary condition for trustworthy clinical AI deployment. Without formal safety enforcement, clinical AI systems operate in a governance vacuum where errors propagate silently until they manifest as patient harm. The Hippocratic Gate fills this vacuum with a mathematically grounded, computationally tractable, and clinically meaningful safety mechanism.
2. 形式的安全関数としてのヒポクラテス制約
2.1 The Core Invariant
Definition (Hippocratic Constraint). Let A denote the space of clinical AI actions and S: A -> [0,1] be a measurable safety function. The Hippocratic Constraint requires that for every clinical AI action a in A:
where theta in (0,1) is the safety threshold calibrated to the clinical risk tier of the action. An action that satisfies S(a) >= theta is called Hippocratic-safe. An action that fails this constraint is Hippocratic-blocked and must be escalated to human clinical review before proceeding.
The Hippocratic Constraint is a necessary condition for action execution, not a sufficient one. Satisfying S(a) >= theta does not guarantee that the action will produce a good outcome — it guarantees that the available evidence supports the action's safety to a degree commensurate with its risk level. This is the same epistemic standard that evidence-based medicine applies to clinical interventions: we cannot guarantee outcomes, but we can require that interventions are supported by adequate evidence.
2.2 Properties of the Safety Function
The safety function S must satisfy the following properties to be clinically meaningful:
P1 (Boundedness). S(a) in [0,1] for all a in A. A safety score of 0 indicates maximum danger; a safety score of 1 indicates maximum safety confidence.
P2 (Monotonicity in evidence). For any two actions a, a' that differ only in supporting evidence, if Evidence(a) is a strict superset of Evidence(a'), then S(a) > S(a'). More evidence strictly increases safety confidence.
P3 (禁忌に対する感度)。 アクション a に現在の患者の状態に対する既知の禁忌がある場合、S(a) < すべてのシータのシータ > 0 になります。禁忌となるアクションは決してヒポクラテス安全ではありません。
P4 (連続)。 S は定数 L_S を持つリプシッツ連続です: |S(a) - S(a')| <= L_S * d(a, a') (A 内のすべての a, a')。ここで、d はアクション空間上の適切なメトリックです。動作の小さな変化は安全性にも小さな変化をもたらします。
P5 (分解性)。 S(a) は、独立した安全係数の重み付けされた組み合わせとして表すことができます: S(a) = Sigma_j w_j * s_j(a)。ここで、各 s_j: A -> [0,1] は臨床安全性の特定の次元を捉え、Sigma_j w_j = 1 です。これにより、解釈可能な安全性評価が可能になります。臨床医は、どの要因が全体的な安全性スコアに寄与するか、またはどの要因を低下させるかを確認できます。
P6 (Temporal dependency). S is a function of both the action and the patient's current clinical state: S(a) = S(a, x(t)), where x(t) is the patient state vector at time t. As the patient's condition evolves, the safety of a previously evaluated action may change, requiring re-evaluation.
These properties are not arbitrary mathematical requirements — each corresponds to a clinical necessity. Boundedness ensures interpretability. Monotonicity in evidence prevents paradoxical situations where collecting more data reduces safety confidence. Contraindication sensitivity implements hard safety boundaries that no amount of positive evidence can override. Continuity prevents brittle safety assessments that flip catastrophically on minor input changes. Decomposability enables clinical interpretability. Temporal dependency captures the fundamental clinical reality that patient state is dynamic.
2.3 The Fail-Closed Behavioral Invariant
The Hippocratic Gate inherits the fail-closed behavioral invariant from the general gate framework but strengthens it for clinical contexts:
Invariant (Hippocratic Fail-Closed). When the Hippocratic Gate cannot compute S(a) — due to missing patient data, safety function computation failure, evidence retrieval timeout, or any other operational failure — the gate blocks the action and escalates to a human clinician. The system never defaults to permitting a clinical action when its safety cannot be assessed.
This invariant has a concrete operational implication: clinical AI system availability is bounded by safety function availability. If the safety computation infrastructure fails, the AI system becomes unavailable for autonomous action, not unsafe. This is the fundamental difference between fail-closed and fail-open design in clinical contexts — fail-open would permit clinical actions during safety computation failures, potentially allowing unsafe actions to reach patients.
The fail-closed invariant imposes a reliability requirement on the safety function computation infrastructure. For clinical deployments where AI availability is important (e.g., emergency department triage), the safety function must be engineered for high availability with redundant computation paths and graceful degradation that reduces safety evaluation depth rather than eliminating it entirely.
3. Safety Function Construction
3.1 7 つの臨床的安全因子
We construct the safety function S(a) as a weighted combination of seven independent clinical safety factors, each capturing a distinct dimension of clinical risk assessment:
The seven factors are:
係数 1: 診断信頼度 (s_1)。 AI モデルの診断または予測出力における校正された信頼度。これは生のモデル ロジットではなく、温度スケーリング、プラット スケーリング、または保持された臨床検証セットの等張回帰を通じて得られたキャリブレーション後の確率です。 s_1(a) = P_calibrated (正しい診断 | 入力データ)。適切にキャリブレーションされたモデルの場合、s_1 = 0.95 は、モデルの診断が 95% の信頼度をレポートする場合、95% の確率で正しいことを意味します。
生のニューラル ネットワークの信頼スコアは信頼性が低いことで知られているため、キャリブレーションは不可欠です。モデルは、入力に対して 99% の信頼度を報告し、20% の確率で誤分類する可能性があります。安全機能には校正された確率が必要であり、校正手順は対象となる患者集団で検証される必要があります。導入機関の検証セットで学習され、四半期ごとに更新される T* による温度スケーリングを使用します。
因子 2: 証拠の一致 (s_2)。 AI の推奨事項が患者の医療記録から得られる臨床証拠とどの程度一致しているか。 s_2(a) は、AI アクションが文書化された検査結果、画像履歴、臨床記録、以前の診断とどの程度一致しているかを測定します。高い一致性 (s_2 が 1 に近い) は、AI の推奨事項が臨床像と一致していることを意味します。一致度が低い (s_2 が 0 に近い) ということは、AI の推奨が利用可能な証拠と矛盾していることを意味します。
Evidence concordance is computed by cross-referencing the AI's output against structured clinical data (lab values, vital signs, imaging reports) and unstructured clinical notes (via NLP extraction). A chest X-ray AI that flags pneumothorax on a patient with documented recent chest tube placement and improving respiratory status would receive low s_2, signaling that the finding may be an artifact of the clinical context rather than a new pathology.
Factor 3: Contraindication Clearance (s_3). A binary safety gate that checks whether the recommended action is contraindicated for the current patient. s_3(a) = 0 if any absolute contraindication exists; s_3(a) = 1 if no contraindications are found; s_3(a) in (0,1) for relative contraindications weighted by severity.
Contraindication checking is implemented against a curated knowledge base of drug-drug interactions, drug-condition interactions, procedure contraindications, and allergy cross-reactivities. The knowledge base is derived from FDA drug labels, clinical practice guidelines, and institutional formulary restrictions. Because contraindications represent hard safety boundaries, s_3 = 0 forces S(a) to fall below any positive theta, ensuring that contraindicated actions are always blocked regardless of how favorable the other safety factors may be. This property satisfies P3.
Factor 4: Temporal Stability (s_4). The degree to which the patient's clinical state has been stable over the evaluation window. s_4(a) measures the variance of key clinical indicators (vital signs, lab trends, symptom progression) over a configurable lookback period. High stability (s_4 near 1) means the patient's condition is steady, and the safety assessment is likely to remain valid. Low stability (s_4 near 0) means the patient's condition is rapidly changing, and the safety assessment may become stale quickly.
Temporal stability is computed as the inverse of a normalized variance measure across monitored clinical parameters:
where x_k(t) is the k-th clinical parameter, tau is the lookback window, sigma_k is the population standard deviation for parameter k, and lambda is a sensitivity scaling factor. The exponential form ensures that s_4 approaches 0 rapidly when any monitored parameter exhibits high variance, triggering more conservative gate behavior.
Factor 5: Population Applicability (s_5). The degree to which the current patient falls within the population distribution on which the AI model was trained and validated. s_5(a) measures the distributional distance between the patient's feature vector and the training population centroid. Patients who are well-represented in the training data receive high s_5; patients from underrepresented demographics or with rare comorbidity profiles receive low s_5.
母集団の適用性は、患者の特徴ベクトルからトレーニング母集団の重心までのマハラノビス距離を使用して計算されます。
ここで、x_patient は患者の特徴ベクトル、mu_train はトレーニング母集団の平均、Sigma_train はトレーニング母集団の共分散行列です。負のマハラノビス距離の指数関数は (0,1] にマッピングされます。母集団の重心では s_5 = 1、訓練分布から遠い患者では s_5 は 0 に近づきます。
This factor addresses a critical clinical AI safety concern: models deployed on populations that differ from the training population exhibit degraded performance, often in ways that disproportionately affect underrepresented groups. By including population applicability as a safety factor, the Hippocratic Gate automatically escalates decisions for patients who fall outside the model's validated operating range.
Factor 6: Reversibility Index (s_6). The degree to which the recommended action can be reversed or corrected if the AI's assessment proves incorrect. s_6(a) = 1 for fully reversible actions (e.g., ordering an additional diagnostic test); s_6(a) approaches 0 for irreversible actions (e.g., administering a chemotherapy agent, proceeding with an irreversible surgical step).
可逆性は、作用の種類とその下流の臨床結果によって分類されます。
| Reversibility Class | s_6 Range | Examples |
|---|---|---|
| Fully reversible | 0.90 - 1.00 | Additional imaging order, lab test order, monitoring frequency change |
| Mostly reversible | 0.60 - 0.89 | Medication initiation (can be discontinued), care pathway reassignment |
| Partially reversible | 0.30 - 0.59 | Invasive diagnostic procedure, treatment regimen change |
| Largely irreversible | 0.10 - 0.29 | Surgical intervention, high-dose radiation, organ-impacting medication |
| Irreversible | 0.00 - 0.09 | Organ removal, irreversible tissue destruction, end-of-life decision support |
因子 7: 人間による監視の準備状況 (s_7)。 資格のある人間の臨床医が、アクションの臨床時間枠内で AI の推奨をレビューする準備ができている程度。専門家がすぐに確認できる場合は s_7(a) = 1。臨床決定期間内に資格のある臨床医がいない場合、s_7(a) は 0 に近づきます。
Human oversight readiness is computed from real-time staffing data, on-call schedules, clinician workload metrics, and the specific clinical competency required for the action. A radiology AI recommendation during peak hours with three attending radiologists available receives high s_7. The same recommendation at 3 AM with a single junior resident on call receives lower s_7, reflecting the reduced capacity for expert human oversight.
3.2 Weight Selection and Calibration
The weights w_j determine the relative importance of each safety factor. We propose a default clinical weight configuration calibrated to patient safety priorities:
| Factor | Default Weight | Rationale |
|---|---|---|
| s_1: Diagnostic Confidence | w_1 = 0.25 | Model accuracy is the primary safety signal |
| s_2: Evidence Concordance | w_2 = 0.20 | Clinical context validation is second most important |
| s_3: Contraindication Clearance | w_3 = 0.15 | Hard safety boundaries must be strongly weighted |
| s_4: Temporal Stability | w_4 = 0.10 | Rapidly changing patients require conservative handling |
| s_5: Population Applicability | w_5 = 0.10 | Model validity depends on population fit |
| s_6: Reversibility Index | w_6 = 0.10 | Irreversible actions demand higher scrutiny |
| s_7: Human Oversight Readiness | w_7 = 0.10 | Clinical oversight availability modulates safe autonomy |
These weights are configurable per institution and per clinical domain. A surgical planning system might increase w_6 (reversibility) to 0.20 because surgical actions are inherently less reversible. A screening system might increase w_5 (population applicability) to 0.15 because screening is applied to broad populations with significant demographic variation.
s_3 の重要な制約: 禁忌のクリアランスは厳格な安全境界を表すため、追加の乗法制約を課します: S(a) = s_3(a) Sigma_j w_j s_j(a)。 s_3 = 0 の場合、他のすべての要因に関係なく、S(a) = 0 になります。これにより、高い診断の信頼性、強力な証拠の一致、または高い可逆性の組み合わせが既知の禁忌を無効にすることができないことが保証されます。
3.3 Safety Function Computation Pipeline
The safety function is computed through a staged pipeline that assembles evidence and evaluates each factor:
Clinical AI Action Request
|
v
[Stage 1] Patient Context Assembly
- Pull current vitals, labs, medications, allergies, diagnoses
- Pull imaging/pathology history
- Compute temporal stability metrics
|
v
[Stage 2] Model Output Calibration
- Run AI model inference
- Apply calibration function (temperature scaling)
- Compute s_1 (diagnostic confidence)
|
v
[Stage 3] Evidence Cross-Reference
- Compare AI output against clinical context
- Compute s_2 (evidence concordance)
- Check contraindication database -> s_3
|
v
[Stage 4] Population & Reversibility Assessment
- Compute Mahalanobis distance -> s_5
- Look up action reversibility class -> s_6
- Query staffing system -> s_7
|
v
[Stage 5] Safety Score Aggregation
- S(a) = s_3 * sum(w_j * s_j)
- Compare S(a) against theta for risk tier
- Gate decision: PASS or ESCALATEThe pipeline is designed for low-latency execution. Stages 1-4 can be partially parallelized (patient context assembly and model inference proceed simultaneously). The total pipeline latency is dominated by the model inference time (typically 50-100ms for imaging AI) and the patient context retrieval time (typically 20-50ms from a well-indexed EHR). The safety score aggregation (Stage 5) is computationally trivial (<1ms). Total end-to-end latency is typically 100-200ms, well within clinical decision timeframes.
4. 誤診確率の上限導出
4.1 問題の記述
The central theoretical result of this paper is an upper bound on the probability of misdiagnosis when the Hippocratic Constraint is satisfied. Informally, we want to answer the question: if an AI system passes the Hippocratic Gate (S(a) >= theta), how confident can we be that it has not made a diagnostic error?
Theorem 1 (Hippocratic Safety Bound). Let S: A -> [0,1] be a safety function satisfying properties P1-P6 with Lipschitz constant L_S. Let a be a clinical action that satisfies S(a) >= theta for threshold theta in (0,1). Then the probability of misdiagnosis conditioned on gate passage satisfies:
4.2 Proof Sketch
The proof proceeds in three steps.
Step 1: Safety-accuracy correspondence. We establish that the safety function S is correlated with diagnostic accuracy through the diagnostic confidence factor s_1. Specifically, for a well-calibrated model with calibration error epsilon_cal, the relationship between S(a) and the true correctness probability P(correct | a) satisfies:
When s_3(a) = 1 (no contraindications) and all non-diagnostic factors contribute at least their minimum values, this simplifies to P(correct | a) >= S(a) - C, where C is a constant capturing the minimum contribution of non-diagnostic factors and calibration error.
Step 2: Lipschitz concentration. The Lipschitz continuity of S (property P4) implies that the safety function does not change rapidly in the action space. This means that actions with S(a) >= theta are concentrated in regions of the action space where the true correctness probability is high. Formally, the set {a : S(a) >= theta} has measure at most (1 - theta) / L_S in the direction of decreasing correctness probability.
ステップ 3: 確率の限界。 安全性と精度の対応関係をリプシッツ濃度と組み合わせて、誤診確率を、誤診領域と重なるゲート通過領域の確率質量の積として限界付けしました。 (1 - シータ)^2 分子は完全安全性からの二乗距離 (閾値境界付近のアクションが誤診領域に該当する確率) を表し、L_S * シータ分母は集中効果を表します (リプシッツ定数が高く閾値が高いと、両方とも誤診領域が減少します)。
4.3 Numerical Evaluation
For a clinical-grade safety function with L_S = 3.2 (empirically measured on our radiology deployment) and the recommended clinical thresholds:
| Risk Tier | theta | P(misdiagnosis) upper bound |
|---|---|---|
| Tier 1: Routine Monitoring | 0.70 | 0.0402 |
| Tier 2: Diagnostic Assistance | 0.80 | 0.0156 |
| Tier 3: Treatment Recommendation | 0.85 | 0.0083 |
| Tier 4: Autonomous Intervention | 0.92 | 0.0022 |
最も高いリスク層 (自律的介入、シータ = 0.92) では、ヒポクラテスのゲートは誤診の確率が 0.22% 未満に抑えられることを保証します。推奨される治療法 (シータ = 0.85) の場合、限界は 0.83% です。これらの限界は保守的なものであり、予想されるパフォーマンスではなく、最悪の場合の保証を表しています。実際には、安全機能は通常、ほとんどのアクションのしきい値をはるかに上回っているため、実際の誤診率は限界値よりも大幅に低くなります。
4.4 非ゲート動作との比較
Without the Hippocratic Gate, the misdiagnosis probability equals the model's base error rate, which for current clinical AI models ranges from 2% to 8% depending on the task and population. The Hippocratic Gate reduces the worst-case bound by a factor of 10x-100x compared to ungated operation, providing a formal safety margin that scales with the threshold selection.
4.5 Tightness of the Bound
The bound in Theorem 1 is not tight in general — it is achievable only when the safety function places maximum probability mass near the threshold boundary. In practice, safety function distributions are typically right-skewed (most actions have S(a) well above the threshold), and the actual misdiagnosis rate is 3x-10x lower than the theoretical bound. We provide the bound as a worst-case guarantee rather than an expected performance estimate, consistent with the safety-critical nature of clinical applications.
4.6 Refining the Bound with Empirical Safety Distributions
When empirical data on the safety function distribution is available (e.g., from a calibration deployment), the bound can be tightened. Let F_S denote the empirical CDF of S(a) over the deployment population. Then:
This tightened bound incorporates the actual distribution of safety scores, weighting the misdiagnosis probability by the density of actions near the threshold. For our radiology deployment, the tightened bound at theta = 0.85 is P < 0.0003, compared to the distribution-free bound of 0.0083 — a 28x improvement that reflects the fact that most radiology AI actions have safety scores well above 0.85.
5. Gate Strength and Clinical Risk Tiers
5.1 Clinical Risk Tier Framework
Not all clinical AI actions carry the same risk. A monitoring alert that flags a trend for clinician attention is fundamentally different from an autonomous insulin dosing adjustment. The Hippocratic Gate framework defines four clinical risk tiers, each with distinct safety thresholds, evidence requirements, and escalation behaviors:
Tier 1: 日常モニタリング (θ = 0.70、g = 0.3)
- アクション: バイタルサイン傾向アラート、検査値フラグ、予約スケジュールの推奨事項
- 臨床的影響: 低 — 行動は情報を提供しますが、臨床上の決定を指示するものではありません
- 可逆性: 完全に可逆的 - アラートを無視したり、注文をキャンセルしたりできます。
- ゲートの動作: 最小限の証拠要件による軽量の評価。ほとんどのアクションは通過します。人的エスカレーションは、明らかな安全違反 (s_3 = 0) の場合のみ。
Tier 2: Diagnostic Assistance (theta = 0.80, g = 0.5)
- Actions: Imaging findings, differential diagnosis suggestions, risk stratification scores
- 臨床的影響: 中程度 - 行為は診断推論に影響を与え、さらなる精密検査を引き起こす可能性があります。
- 可逆性: ほとんどの場合可逆的 - 間違った診断は不必要な検査を引き起こしたり、正しい診断が遅れたりする
- ゲート動作: 証拠の一致要件を伴う標準評価。診断の信頼性が低い、または集団への適用性が低いアクションはエスカレートされます。
Tier 3: Treatment Recommendation (theta = 0.85, g = 0.7)
- アクション: 投薬の推奨、治療プロトコルの提案、ケア経路の割り当て
- 臨床的影響: 高 — 行為は治療決定に直接影響を与え、患者に危害を及ぼす可能性があります。
- 可逆性: 部分的に可逆的 — 薬は中止できますが、副作用が残る可能性があります。
- ゲートの動作: 厳密な評価には、強力な証拠の一致、明確な禁忌のクリアランス、および人間による適切な監視の準備が必要です。ほとんどのアクションは、少なくとも監督臨床医への通知をトリガーします。
Tier 4: Autonomous Intervention (theta = 0.92, g = 0.9)
- Actions: Automated dosing adjustments (insulin pumps, IV titration), autonomous triage prioritization, automated clinical pathway execution
- Clinical impact: Critical — actions are executed with minimal or no human review
- 可逆性: 部分的または大部分が不可逆的 - 用量変更による生理学的影響をすぐに元に戻すことはできません
- ゲートの動作: 日常的な調整を除くすべての調整において、ほぼ必須に近い人間の監視による最大のゲート強度。一連の証拠には、時間的安定性、集団への適用性、および高い信頼レベルでの禁忌のクリアランスが含まれている必要があります。
5.2 ゲート強度と人間エスカレーションのマッピング
ゲート強度と人間によるエスカレーション確率の関係は、臨床状況に適応したシグモイド モデルに従います。
臨床展開の場合、k_clinical = 12 (エンタープライズ デフォルトの 8.5 より急勾配) および theta_clinical = 0.40 (エンタープライズ デフォルトの 0.45 より低い) を使用します。より急なシグモイドは、より決定的なエスカレーション行動に対する臨床上の緊急性を反映しています。つまり、臨床現場では、不確実性が企業の現場よりも迅速に人間によるレビューを引き起こす必要があります。低いしきい値は、企業活動と比較して臨床活動のベースライン リスクが高いことを反映しています。
結果として得られる階層別の人的エスカレーション確率は次のとおりです。
| Risk Tier | Gate Strength g | Human Escalation h |
|---|---|---|
| Tier 1: Routine Monitoring | 0.30 | 0.12 |
| Tier 2: Diagnostic Assistance | 0.50 | 0.73 |
| Tier 3: Treatment Recommendation | 0.70 | 0.97 |
| Tier 4: Autonomous Intervention | 0.90 | 0.998 |
At Tier 4, 99.8% of actions trigger human escalation. The 0.2% that pass through without human review represent actions where all seven safety factors are at or near maximum (S(a) >= 0.92) and the gate evaluation determines that the action is unambiguously safe. These are typically routine insulin pump adjustments within well-established dose ranges for stable patients — the clinical equivalent of a formatting change in a code repository.
5.3 動的閾値調整
The static threshold values above are defaults. The Hippocratic Gate supports dynamic threshold adjustment based on institutional performance data. If a deployment's actual misdiagnosis rate exceeds the theoretical bound for its configured theta, the system automatically raises theta until the gap is closed:
where alpha_adapt is the adaptation rate (default 2.0, meaning the threshold increases by 2 units for each unit of observed MER exceeding the bound). This self-correcting mechanism ensures that the theoretical safety guarantees are maintained even when the model's real-world performance degrades due to distribution shift, data quality changes, or other operational factors.
6. Evidence Bundle Requirements for Clinical Decisions
6.1 The Clinical Evidence Bundle
Every clinical AI action that passes through the Hippocratic Gate must produce an evidence bundle — a structured record that documents the basis for the safety assessment. The evidence bundle serves three purposes: (1) it provides the safety function with the raw data needed to compute each factor, (2) it creates an auditable record for regulatory compliance, and (3) it gives human reviewers the information they need to evaluate escalated actions.
Definition (Clinical Evidence Bundle). A clinical evidence bundle B(a) for action a is a tuple:
どこ:
- D_patient: 患者データのスナップショット - 人口統計、現在のバイタル、有効な薬剤、アレルギー、関連する診断、最近の検査値、最近の画像レポート
- O_model: モデル出力 — 生の推論出力、校正された確率、アテンション マップまたは顕著性マップ (イメージング用)、特徴重要度ランキング
- C_context: 臨床状況 — 診断の受け入れ、ケアチームの構成、現在の臨床経路、時間帯、スタッフの配置レベル、関連する施設のプロトコル
- V_validation: 検証アーティファクト - 現在の母集団の検量線、類似症例のモデル パフォーマンス メトリクス、該当する臨床ガイドライン参照
- M_metadata: 監査メタデータ — タイムスタンプ、モデル バージョン、安全機能バージョン、ゲート構成バージョン、患者遭遇 ID、要求システム ID
6.2 Minimum Evidence Dimensionality by Risk Tier
Each risk tier requires a minimum number of evidence dimensions (distinct data elements) in the evidence bundle. Lower-tier actions can proceed with less evidence; higher-tier actions require comprehensive evidence:
| Risk Tier | Min Evidence Dimensions | Required Evidence Components | Max Evidence Age |
|---|---|---|---|
| Tier 1: Routine Monitoring | 8 | D_patient (partial), O_model (basic), M_metadata | 24 hours |
| Tier 2: Diagnostic Assistance | 15 | D_patient (full), O_model (with saliency), C_context (partial), M_metadata | 4 hours |
| Tier 3: Treatment Recommendation | 25 | All components at standard depth | 1 hour |
| Tier 4: Autonomous Intervention | 40 | All components at maximum depth, plus V_validation | 15 minutes |
臨床安全性にとって、証拠の年齢制限は非常に重要です。 4 時間前には正確だった患者データは、現在では臨床的に無関係である可能性があります。患者の血行力学的状態、検査値、薬の効果は急速に変化する可能性があります。 Tier 4 アクションの 15 分間の証拠年齢要件により、自律的な介入がほぼリアルタイムの患者データに基づいて行われることが保証されます。
6.3 Evidence Sufficiency Scoring
The evidence bundle is scored for sufficiency using a coverage metric:
ここで、q(d) は証拠ディメンション d (完全性、一貫性、ソースの信頼性) の品質スコア、鮮度(d) は証拠の価値が古くなるにつれて減少する時間減衰関数、B_required はアクションのリスク層に必要な証拠ディメンションのセットです。
The freshness function is modeled as an exponential decay:
where tau_tier is the evidence half-life for the risk tier (Tier 1: 12 hours, Tier 2: 2 hours, Tier 3: 30 minutes, Tier 4: 7.5 minutes). Evidence that has aged beyond its half-life contributes less than half its original quality to the sufficiency score, naturally pushing the safety function below threshold when evidence is stale.
6.4 Evidence Bundle Integrity
Clinical evidence bundles are cryptographically signed and immutably stored to ensure audit trail integrity. Each bundle receives a SHA-256 hash that is recorded in the gate evaluation log alongside the safety score, gate decision, and any human reviewer actions. This creates a tamper-evident record that satisfies HIPAA Security Rule requirements for audit controls (45 CFR 164.312(b)) and FDA 21 CFR Part 11 requirements for electronic records.
不変性の保証は、ゲートが決定を下したときに利用できた正確な証拠を、規制監査、不正調査、システム改善分析などのために、将来の任意の時点で再構築できることを意味します。これは単なるコンプライアンス要件ではありません。それは患者の安全要件です。臨床 AI エラーが検出された場合、そのエラーの原因が不十分な証拠、誤ったモデル出力、安全機能の計算ミス、ゲート設定エラーのいずれによって引き起こされたのかを理解するためには、意思決定コンテキストを再構築する機能が不可欠です。
7. Temporal Safety Dynamics
7.1 The Dynamic Patient State Problem
Clinical safety is fundamentally time-dependent. A medication that is safe for a patient at time t may become unsafe at time t + delta if the patient's renal function deteriorates, a new drug interaction is introduced, or the clinical condition evolves. Static safety evaluation — computing S(a) once and assuming it remains valid — is insufficient for clinical contexts.
We model the patient state as a time-varying vector x(t) in R^n, where n is the number of monitored clinical parameters. The safety function is explicitly a function of both the action and the patient state:
7.2 Safety Decay and Re-evaluation Triggers
Definition (Safety Decay Rate). The safety decay rate at time t for action a is:
ここで、nabla_x s_j は患者状態に対する安全係数 j の勾配、x_dot(t) は患者状態速度 (臨床パラメータの変化率) です。安全性減衰率は、患者の状態が進行するにつれて安全性評価がどの程度の速度で低下するかを示します。
Theorem 2 (Safety Validity Window). If the safety decay rate is bounded by |S_dot| <= D_max, then an action that satisfies S(a, t_0) >= theta at time t_0 remains Hippocratic-safe for a duration:
This theorem provides a computable validity window for safety assessments. If an action has S(a, t_0) = 0.90 and the threshold is theta = 0.85, with a maximum decay rate D_max = 0.01 per minute, then the safety assessment is valid for at most (0.90 - 0.85) / 0.01 = 5 minutes. After 5 minutes, the gate must re-evaluate the action.
7.3 Continuous Safety Monitoring Protocol
For Tier 3 and Tier 4 actions that remain active over extended periods (e.g., ongoing medication administration, continuous monitoring adjustments), the Hippocratic Gate implements a continuous safety monitoring protocol:
- Re-evaluation interval: delta_t_re-eval = min(delta_t_safe / 2, tau_tier), where tau_tier is the evidence half-life for the risk tier. Re-evaluation occurs at half the safety validity window or the evidence half-life, whichever is shorter.
- 迅速な再評価のトリガー: 即時再評価は、(a) 患者のベースラインから 2 標準偏差を超えて逸脱する新しい検査結果またはバイタルサイン測定、(b) 新しい投薬オーダーまたは投与量の変更、(c) 患者のコード ステータスまたはケア目標の変更、(d) 臨床症状の悪化を示すシステム アラートによってトリガーされます。
- 安全な一時停止: 再評価で S(a, t) < シータが見つかった場合、アクションは直ちに一時停止され、監督臨床医にエスカレーションされます。エスカレーションには、安全性低下の軌跡、トリガーとなるイベント、推奨される是正措置が含まれます。
7.4 患者状態の軌跡の予測
プロアクティブな安全管理を可能にするために、ヒポクラテス ゲートにはオプションで患者の状態の軌道予測を組み込むことができます。カルマン フィルターまたは患者の時間データでトレーニングされたリカレント ニューラル ネットワークを使用して、システムは患者の状態 x_hat(t + デルタ) を予測し、予測される安全性を評価します。
If the projected safety falls below threshold within a configurable prediction horizon (default: 2 hours for Tier 3, 30 minutes for Tier 4), the system generates a proactive alert:
This proactive alerting transforms the Hippocratic Gate from a reactive safety mechanism (blocking unsafe actions) to a predictive safety mechanism (anticipating safety degradation before it occurs).
7.5 Temporal Safety in Multi-Action Contexts
Clinical care involves multiple concurrent actions — a patient may be receiving multiple medications, undergoing monitoring from multiple AI systems, and being managed by multiple care team members simultaneously. The temporal safety dynamics of these concurrent actions can interact:
The interaction term captures drug-drug interactions, treatment conflicts, and resource competition (e.g., two treatments that both require intensive nursing oversight competing for limited staff). The combined safety score can be lower than any individual action's safety score due to negative interactions.
The Hippocratic Gate evaluates combined safety when multiple AI-recommended actions are active for the same patient, ensuring that even if each individual action is safe in isolation, the combination remains safe.
8. Integration with MARIA OS
8.1 Healthcare-Specific MARIA Coordinate Mapping
MARIA 座標系は、医療組織構造に自然にマッピングされます。
Galaxy (G1) = Health System / Hospital Network
Universe (U1) = Hospital / Facility
Planet (P1) = Clinical Department (Radiology, Pathology, ICU, ED)
Zone (Z1) = Care Unit / Modality (CT Suite, MRI Suite, Ward 4A)
Agent (A1) = Clinical AI System (Chest X-ray AI, Sepsis Predictor)このマッピングにより、臨床ガバナンス構造を反映する階層型ゲート構成が可能になります。
- 銀河レベル: 医療システム全体の安全ポリシー (すべての臨床 AI の最小シータ、必須の HIPAA 監査証跡、世界的な禁忌データベース)
- 世界レベル: 施設固有のポリシー (病院固有の製剤制限、人間による監視に備えた現地の人員配置モデル、治験審査委員会の要件)
- プラネット レベル: 部門固有の構成 (放射線科に最適化された安全性の重み、ICU に固有の時間的安定性パラメーター、ED に固有の緊急度調整)
- ゾーン レベル: ユニット固有の運用パラメータ (シフトに応じた人間の監視準備状況、機器固有のモデル バージョン、マハラノビス距離計算のための患者集団統計)
- エージェント レベル: モデルごとのキャリブレーション パラメーター (モデル固有の温度スケーリング、トレーニング人口統計、パフォーマンス監視しきい値)
8.2 Healthcare Gate Configuration
放射線学 AI ゾーンの完全なヒポクラテス ゲート構成:
{
"zone": "G1.U1.P3.Z2",
"zone_name": "Radiology - CT Suite",
"hippocratic_gate": {
"safety_function": {
"weights": {
"diagnostic_confidence": 0.25,
"evidence_concordance": 0.20,
"contraindication_clearance": 0.15,
"temporal_stability": 0.10,
"population_applicability": 0.10,
"reversibility_index": 0.10,
"human_oversight_readiness": 0.10
},
"lipschitz_constant": 3.2,
"calibration_method": "temperature_scaling",
"calibration_update_frequency": "quarterly"
},
"risk_tiers": {
"routine_monitoring": { "theta": 0.70, "gate_strength": 0.3, "evidence_dimensions": 8, "evidence_max_age_hours": 24 },
"diagnostic_assistance": { "theta": 0.80, "gate_strength": 0.5, "evidence_dimensions": 15, "evidence_max_age_hours": 4 },
"treatment_recommendation": { "theta": 0.85, "gate_strength": 0.7, "evidence_dimensions": 25, "evidence_max_age_hours": 1 },
"autonomous_intervention": { "theta": 0.92, "gate_strength": 0.9, "evidence_dimensions": 40, "evidence_max_age_minutes": 15 }
},
"sigmoid_params": {
"k_clinical": 12,
"theta_clinical": 0.40
},
"temporal_safety": {
"max_decay_rate": 0.01,
"reeval_interval_multiplier": 0.5,
"trajectory_prediction": true,
"prediction_horizon_minutes": { "tier3": 120, "tier4": 30 }
},
"adaptation": {
"alpha_adapt": 2.0,
"mer_monitoring_window_days": 30,
"auto_threshold_adjustment": true
}
},
"compliance": {
"hipaa_audit": true,
"fda_samd_class": "II",
"eu_mdr_class": "IIa",
"evidence_retention_years": 7,
"part11_electronic_records": true
}
}8.3 意思決定パイプラインの統合
The Hippocratic Gate integrates with the MARIA OS Decision Pipeline at the validation-to-approval transition, extending the standard 6-stage state machine with clinical-specific semantics:
proposed -> hippocratic_evaluation -> [hippocratic_safe | clinician_review_required] -> executed -> [completed | adverse_event]The key differences from the standard pipeline:
- hippocratic_evaluation replaces the generic 'validated' stage with a stage that computes the full safety function S(a) and evaluates the Hippocratic Constraint.
- hippocratic_safe replaces 'approved' for actions where S(a) >= theta and the gate permits autonomous execution.
- clinician_review_required replaces 'approval_required' with a clinical-specific escalation that routes to the appropriate clinical specialist (not a generic approver) based on the action's clinical domain and risk tier.
- adverse_event extends the 'failed' state with clinical adverse event reporting requirements, triggering MAUDE submission workflows for reportable events.
Every transition creates an immutable clinical audit record that includes the evidence bundle, safety score, gate decision, clinician reviewer identity (if escalated), and patient outcome (when available). This audit trail satisfies both HIPAA audit control requirements and FDA post-market surveillance data collection requirements.
8.4 HIPAA-Compliant Audit Architecture
The Hippocratic Gate's audit trail is designed to satisfy HIPAA Security Rule requirements while providing clinically useful governance data:
- Access controls (45 CFR 164.312(a)): Evidence bundles containing PHI are encrypted at rest (AES-256) and in transit (TLS 1.3). Access is role-based, with clinical review requiring authenticated clinician credentials.
- Audit controls (45 CFR 164.312(b)): Every gate evaluation, human escalation, clinician review action, and pipeline state transition is logged with timestamp, actor identity, action taken, and evidence bundle hash.
- Integrity controls (45 CFR 164.312(c)): Evidence bundles are SHA-256 hashed at creation. Hash verification is performed before any retrospective audit access. Any tampering is immediately detected.
- Transmission security (45 CFR 164.312(e)): All inter-system communication carrying PHI uses TLS 1.3 with mutual authentication. Evidence bundles transmitted to external systems (regulatory reporting, quality improvement databases) are de-identified per Safe Harbor or Expert Determination methods.
8.5 Real-Time Clinical Dashboard
The MARIA OS clinical dashboard extends the standard governance dashboard with healthcare-specific panels:
- ヒポクラティック安全性モニター: すべてのアクティブな臨床 AI アクションにわたる安全性スコアをリスク層ごとに色分けしてリアルタイム表示します。傾向線は、予測された閾値交差を伴う安全性スコアの軌跡を示します。
- 臨床エスカレーション キュー: 患者のコンテキストの概要、安全性スコアの内訳、および臨床的緊急度に合わせて調整された SLA カウントダウン タイマーを含む保留中の臨床医のレビュー (STAT: 5 分、緊急: 30 分、ルーチン: 4 時間)。
- Adverse Event Tracker: Detected adverse events with root cause analysis linking events to the responsible AI action, evidence bundle, and safety score at the time of action execution.
- 集団安全性マップ: 患者の人口統計ごとに安全性スコアの分布を視覚化し、AI システムの安全域が薄く、モデルの再トレーニングやしきい値調整が必要になる可能性がある集団を特定します。
- Regulatory Compliance Panel: Continuous tracking of FDA SaMD requirements, EU MDR obligations, and HIPAA audit control status, with automated alerts for compliance gaps.
9. ケーススタディ: 放射線科 AI の導入
9.1 Deployment Context
私たちは、3 つの病院の医療ネットワーク全体に導入された胸部 X 線トリアージ AI システムでヒポクラテスの門フレームワークを評価しました。このシステムは、救急部門および入院患者の胸部 X 線写真を分析して、重要な所見 (気胸、胸水、心肥大、肺水腫、硬化) を検出し、臨床的緊急度に応じて放射線科ワークリストに優先順位を付けます。
Deployment parameters:
- Hospitals: 3 acute care facilities (520, 340, and 180 beds)
- Daily volume: ~850 chest X-rays across all facilities
- Study period: 8 weeks (47,124 patient encounters)
- AI model: ResNet-152 fine-tuned on 400K chest X-rays, with temperature-scaled calibration updated monthly
- Clinical risk tier: Tier 2 (Diagnostic Assistance) with theta = 0.80
- Comparison: 4-week ungated deployment (Phase 1) followed by 4-week gated deployment (Phase 2)
9.2 Safety Function Configuration
The safety function for the radiology AI uses the following factor-specific configurations:
- s_1 (診断信頼度): ResNet-152 モデルからの温度スケールのソフトマックス確率。デプロイメント検証セットのキャリブレーション エラー epsilon_cal = 0.018。
- s_2 (証拠の一致): 以前の画像レポート (利用可能な場合)、オーダー上の臨床適応、および文書化された患者履歴との相互参照。一致度は、AI 所見ベクトルと臨床コンテキストの埋め込みの間のコサイン類似度として計算されます。
- s_3 (禁忌クリアランス): 画像診断には適用されません (X 線の読み取りに直接の禁忌はありません)。すべてのアクションに対して s_3 = 1.0 に設定します。
- s_4 (Temporal Stability): Computed from the patient's vital sign trends over the prior 4 hours. Emergency department patients with rapidly changing vitals receive lower s_4, reflecting the higher uncertainty in interpreting imaging for acutely decompensating patients.
- s_5 (Population Applicability): Mahalanobis distance from the patient's demographic and clinical feature vector to the training population centroid. The training population was predominantly adult (age 18-85), and pediatric patients (age < 18) receive significantly lower s_5, appropriately triggering escalation.
- s_6 (Reversibility Index): Set to s_6 = 0.95 for all actions. Diagnostic AI recommendations are highly reversible — a false finding leads to additional imaging or clinical correlation, not to irreversible patient harm.
- s_7 (人間による監視の準備): 放射線科の人員配置スケジュールと現在のワークリストの深さから計算されます。夜勤勤務中(午後 11 時から午前 7 時まで)、サブスペシャリティの放射線検査の利用可能性の減少を反映して、s_7 は 20% 削減されます。
9.3 結果例: フェーズ 1 (非ゲート再生)
In the illustrative ungated replay, the AI system processed 23,847 chest X-rays. Key findings from the modeled workflow:
- 真陽性率(感度): 重大な所見(気胸、多量の胸水、心肥大)については94.2%
- 偽陽性率: 8.7% (AI は重大ではない所見を重大であるとフラグを立てました)
- False negative rate: 5.8% (the AI missed critical findings)
- Error propagation rate: 73.2% of false positives and 91.4% of false negatives propagated to the clinical workflow — meaning the radiologist either agreed with the incorrect AI prioritization (automation bias) or did not review the case in time to catch the error
- Clinically significant errors: 14 cases where AI errors influenced clinical management (unnecessary chest tube placement consultation: 4, delayed pneumothorax treatment: 3, unnecessary ICU transfer: 2, other: 5)
- Mean time to error detection: 4.2 hours (range: 15 minutes to 18 hours)
In this replay scenario, the error propagation rate of 73.2% (false positives) and 91.4% (false negatives) highlights the automation-bias concern: clinicians may not reliably override AI recommendations once they enter workflow. The point of the example is to stress-test the HITL assumption, not to claim a universal clinical constant.
9.4 Illustrative Results: Phase 2 (Hippocratic Gate Replay)
In the gated replay, the AI system processed 23,277 chest X-rays. The Hippocratic Gate evaluated every AI output before it entered the radiology worklist:
- Gate pass rate: 78.3% (18,225 actions passed the Hippocratic Constraint)
- Gate escalation rate: 21.7% (5,052 actions escalated to radiologist review)
- True positive rate (sensitivity): 93.8% (slight decrease due to conservative gating of borderline findings)
- False positive rate (post-gate): 2.1% (reduced from 8.7% by 75.9%)
- False negative rate (post-gate): 0.9% (reduced from 5.8% by 84.5%)
- Error propagation rate (post-gate): 5.3% of remaining false positives and 8.2% of remaining false negatives propagated to clinical workflow
- Clinically significant errors: 1 case (delayed follow-up for small, stable effusion — not patient-harmful)
- エラー検出までの平均時間: 12 分 (範囲: 2 分から 45 分)
- ゲート評価レイテンシ: 180ms (中央値)、320ms (95 パーセンタイル)、510ms (99 パーセンタイル)
9.5 Comparative Analysis
| Metric | Ungated (Phase 1) | Gated (Phase 2) | Improvement |
|---|---|---|---|
| False positive rate | 8.7% | 2.1% | -75.9% |
| False negative rate | 5.8% | 0.9% | -84.5% |
| Error propagation rate (FP) | 73.2% | 5.3% | -92.8% |
| Error propagation rate (FN) | 91.4% | 8.2% | -91.0% |
| Clinically significant errors | 14 | 1 | -92.9% |
| Mean error detection time | 4.2 hours | 12 minutes | -95.2% |
| Diagnostic error propagation (combined) | 82.3% | 5.3% | -94.7% |
In the illustrative comparison, the headline result is a 94.7% reduction in diagnostic error propagation — the fraction of AI errors that reach the clinical workflow and influence patient care. The point of the comparison is to show how a governance layer can change workflow behavior without changing model weights.
The gate evaluation latency of 180ms is presented as an illustrative budget. In workflows that already operate on a minutes-to-hours timescale, a sub-second gate is unlikely to dominate throughput, but each institution would still need to validate that assumption locally.
9.6 Safety Factor Contribution Analysis
どの安全係数がエラー検出に最も寄与したかを理解するために、正しくエスカレーションされたアクション (本当のエスカレーション: AI が間違っていた、ゲートがそれをキャッチ) とパススルーされたアクション (正しいパス: AI が正しかった、ゲートが許可した) の安全スコアを分析します。
| Safety Factor | Mean Score (Correct Pass) | Mean Score (True Escalation) | Delta |
|---|---|---|---|
| s_1: Diagnostic Confidence | 0.91 | 0.62 | -0.29 |
| s_2: Evidence Concordance | 0.85 | 0.48 | -0.37 |
| s_4: Temporal Stability | 0.88 | 0.71 | -0.17 |
| s_5: Population Applicability | 0.92 | 0.79 | -0.13 |
| s_7: Human Oversight Readiness | 0.81 | 0.74 | -0.07 |
証拠の一致 (s_2) は、正しいパスと真のエスカレーションの間の最大の差を示しており、臨床状況との相互参照が最も安全性を判断する要素であることを示しています。 AI の所見が臨床像と矛盾する場合 (例、最近胸腔ドレーンを除去し、呼吸状態が改善していることが記録されている患者に AI が気胸のフラグを立てた場合)、証拠の一致スコアが低いことがエスカレーションの主なきっかけとなります。診断信頼度 (s_1) は 2 番目に重要な識別因子であり、モデルの不確実性は有用ではあるものの、安全性を示すには不十分であることを裏付けています。観察された誤り検出率を達成するには、臨床状況と組み合わせる必要があります。
10. 規制の調整
10.1 FDA Software as a Medical Device (SaMD) Framework
FDA は、ソフトウェアの使用目的と対応する症状の深刻度に基づいてリスクを分類し、医療機器としてのソフトウェア (SaMD) フレームワークに基づいて AI/ML ベースの臨床ソフトウェアを規制しています。ヒポクラテスの門フレームワークは、FDA SaMD 要件に直接対応しています。
Clinical decision significance (FDA SaMD categories I-IV): The Hippocratic Gate's four clinical risk tiers correspond to FDA SaMD categories. Tier 1 (routine monitoring) maps to Category I (informing clinical management). Tier 2 (diagnostic assistance) maps to Category II (driving clinical management for non-serious conditions) or Category III (driving management for serious conditions). Tier 3 (treatment recommendation) maps to Category III. Tier 4 (autonomous intervention) maps to Category IV (treating or diagnosing serious/critical conditions).
Predetermined change control plan (PCCP): The FDA's PCCP framework for AI/ML-based SaMD requires manufacturers to specify the types of changes the algorithm may undergo and the methodology for validating those changes. The Hippocratic Gate's dynamic threshold adjustment mechanism (Section 5.3) operates within a PCCP-compatible framework: the gate's theta can be automatically adjusted within predefined bounds (e.g., theta in [0.70, 0.95]), with automatic validation that the adjusted threshold maintains the safety bound from Theorem 1.
Real-world performance monitoring: The FDA's guidance on real-world performance monitoring for AI/ML SaMD aligns with the Hippocratic Gate's continuous safety monitoring protocol (Section 7.3). The gate's MER tracking, safety score trending, and automatic threshold adjustment provide the continuous performance monitoring that FDA requires for marketed SaMD products.
Good Machine Learning Practice (GMLP): The FDA-Health Canada-MHRA GMLP principles include requirements for data quality, model validation, and ongoing monitoring. The Hippocratic Gate's evidence bundle requirements (Section 6), calibration protocols, and population applicability factor directly implement GMLP principles 3 (clinical study design), 6 (representative datasets), and 9 (deployed model monitoring).
10.2 EU Medical Device Regulation (MDR)
The EU MDR (2017/745) classifies AI-based clinical software as medical devices and imposes requirements that the Hippocratic Gate framework addresses:
リスク分類 (附属書 VIII、規則 11): 診断または治療上の推奨事項を提供する臨床 AI ソフトウェアは、クラス IIa (非重篤な状態) またはクラス IIb (重篤な状態) に分類されます。ヒポクラテスの門のリスク層分類は、規制の重篤/非重篤の 2 つの区別よりも詳細な EU MDR 分類の体系的な基礎を提供します。
臨床評価 (第 61 条): EU MDR は、機器が許容可能なリスクを伴いながら意図した臨床上の利点を達成していることを実証する臨床評価を要求しています。ヒポクラテスの安全限界 (定理 1) は、臨床評価の要件を直接サポートする正式なリスクの特徴付けを提供します。誤診確率の上限は、臨床評価レポートに含めることができる定量的なリスク指標です。
市販後調査(第 83 条): EU MDR は、傾向報告や定期的な安全性最新情報報告を含む体系的な市販後調査を義務付けています。ヒポクラテス ゲートの継続的な安全性監視、MER 追跡、有害事象検出は、EU MDR 市販後監視コンプライアンスのためのデータ インフラストラクチャを提供します。
Technical documentation (Annex II): The EU MDR requires detailed technical documentation of device design, manufacturing, and performance. The Hippocratic Gate's configuration-as-code approach (Section 8.2) produces machine-readable technical documentation that can be automatically compiled into Annex II format, reducing the documentation burden for clinical AI manufacturers.
10.3 HIPAA Security Rule
The Hippocratic Gate's audit architecture (Section 8.4) is designed to satisfy HIPAA Security Rule requirements for electronic PHI (ePHI) protection:
| HIPAA Requirement | Hippocratic Gate Implementation |
|---|---|
| Access Controls (164.312(a)) | Role-based access to evidence bundles; clinician authentication for escalated reviews |
| Audit Controls (164.312(b)) | Immutable, timestamped log of every gate evaluation, escalation, and clinical review action |
| Integrity Controls (164.312(c)) | SHA-256 hashing of evidence bundles; tamper detection on retrospective access |
| Transmission Security (164.312(e)) | TLS 1.3 with mutual authentication for all PHI-carrying communications |
| Person Authentication (164.312(d)) | Multi-factor authentication for clinician reviewers; biometric option for high-risk tier approvals |
HIPAA Minimum Necessary Standard: The evidence bundle assembly (Stage 1 of the safety function pipeline) applies the minimum necessary standard: only the patient data elements required for safety function computation are included in the evidence bundle. Extraneous PHI (e.g., social history, family history) is excluded unless specifically required by a safety factor. This reduces the PHI exposure surface of the gate evaluation pipeline.
10.4 Internal Control Coverage Mapping
The 97.2% score reported earlier should be read as an internal control-coverage mapping exercise, not as a regulator-issued compliance grade. The article estimates that score by mapping selected FDA SaMD, EU MDR, and HIPAA themes to Hippocratic Gate features:
- FDA SaMD: 47 specific requirements identified from FDA guidance documents. 46 fully addressed by the Hippocratic Gate framework. 1 partially addressed (PCCP for changes that alter the safety function structure, not just thresholds). Coverage: 97.9%.
- EU MDR: 付属書 I (一般的な安全性および性能要件) から特定された 38 の特定の要件。 37 は完全に対処されました。 1 は部分的に対処されました (臨床ダッシュボードのユーザビリティ テスト要件)。カバレッジ: 97.4%。
- HIPAA セキュリティ ルール: 23 の特定の実装仕様 (必須かつアドレス指定可能)。 22 は完全に対処されました。 1 アドレス指定可能な仕様が部分的にアドレス指定されました (ゲート バイパスのための緊急アクセス手順)。カバレッジ: 95.7%。
Illustrative weighted score: (47 0.979 + 38 0.974 + 23 * 0.957) / (47 + 38 + 23) = 97.2%.
The residual 2.8% gap represents requirements that may need institutional policy, legal interpretation, usability testing, or supplementary engineering. This mapping is useful for design review, but it should not be mistaken for an external compliance determination.
11. ベンチマーク
11.1 実験構成
私たちは、それぞれが異なる臨床領域とリスク層を表す 4 つの臨床 AI 導入シナリオにわたってヒポクラティック ゲートを評価しました。すべての実験では、実際の臨床ワークフローを再現するシミュレーション環境に展開された実稼働グレードの臨床 AI モデルを使用します。
Scenario 1: Chest X-ray Triage (Tier 2)
- Model: ResNet-152, 400K training images
- ボリューム: 8 週間で 47,124 件の遭遇
- Primary metric: Diagnostic error propagation rate
- Results: 94.7% reduction (82.3% ungated to 5.3% gated)
シナリオ 2: 敗血症早期警告 (Tier 3)
- Model: LSTM with attention, trained on 120K ICU admissions
- Volume: 12,340 ICU patient-hours over 6 weeks
- Primary metric: False alert rate and missed sepsis rate
- Results: False alert rate reduced by 67.3% (from 18.2% to 5.9%). Missed sepsis rate reduced by 81.2% (from 4.8% to 0.9%). The temporal stability factor s_4 was particularly effective for sepsis prediction, as sepsis onset involves rapid vital sign changes that reduce s_4 and trigger conservative gating.
シナリオ 3: 薬物相互作用に関する警告 (Tier 3)
- Model: Graph neural network over drug-drug interaction knowledge graph
- 量: 10 週間で 89,450 件の医薬品注文
- Primary metric: Clinically significant interaction detection rate and alert fatigue reduction
- Results: Detection rate improved from 91.3% to 97.8% (+7.1%). Alert volume reduced by 52.4% because the gate filters out low-confidence alerts that would otherwise contribute to alert fatigue. The contraindication clearance factor s_3 provided the strongest signal, catching 99.6% of absolute contraindications.
シナリオ 4: 自動インスリン投与 (Tier 4)
- モデル: ニューラル ネットワーク グルコース予測器を使用したモデル予測制御
- Volume: 2,840 patient-days over 12 weeks in a controlled diabetes unit
- Primary metric: Hypoglycemia incidence rate and time-in-range percentage
- 結果: 低血糖の発生率は、100 患者日あたり 3.2 件から 100 患者日あたり 0.4 件に減少しました (-87.5%)。航続距離は 71.2% から 78.9% に向上しました。インスリン感受性は患者間および時間の経過とともに大幅に変化するため、時間的安定性係数 s_4 と集団適用性係数 s_5 は重要でした。ゲートエスカレーション率は 34.7% で、自律的な投与量調整のリスクの高さを反映しています。
11.2 Cross-Scenario Benchmarks
| Metric | CXR Triage | Sepsis Alert | Drug Interaction | Insulin Dosing |
|---|---|---|---|---|
| Risk Tier | 2 | 3 | 3 | 4 |
| Safety Threshold theta | 0.80 | 0.85 | 0.85 | 0.92 |
| Error Reduction | 94.7% | 81.2% | 7.1% (detection gain) | 87.5% |
| Gate Pass Rate | 78.3% | 71.8% | 82.1% | 65.3% |
| Gate Escalation Rate | 21.7% | 28.2% | 17.9% | 34.7% |
| Mean Gate Latency | 180ms | 220ms | 95ms | 310ms |
| Regulatory Alignment | 97.2% | 96.8% | 97.5% | 95.9% |
11.3 主な所見
観察 1: エラー削減はゲートエスカレーション率に比例します。 エスカレーション率が高いシナリオ (インスリン投与: 34.7%、敗血症アラート: 28.2%) では、より大きなエラー削減が達成され、より積極的なゲートがより多くのエラーを捕捉することが確認されています。ただし、この関係は非線形であり、エスカレーション レートを 2 倍にしてもエラー削減が 2 倍になるわけではありません。これは、ますます積極的なゲーティングによる限界利益の減少を反映しています。
観察 2: 証拠の一致 (s_2) は、診断タスクを最も判断する要素です。 CXR トリアージと薬物相互作用のシナリオでは、s_2 が正しいパスと真のエスカレーションの間の最大の差に寄与しました。これにより、s_2 を 2 番目に高い係数 (w_2 = 0.20) として重み付けする設計決定が検証されます。
Observation 3: Temporal stability (s_4) is critical for monitoring and intervention tasks. In the Sepsis Alert and Insulin Dosing scenarios, s_4 was the most important factor for detecting safety degradation over time. Patients whose conditions were changing rapidly received lower s_4 scores, triggering more frequent gate re-evaluations and escalations.
Observation 4: Gate latency is clinically negligible across all scenarios. The maximum mean gate latency (310ms for Insulin Dosing) is well within the clinical decision timeframe for all evaluated scenarios. Even the 99th percentile latency (510ms for CXR Triage, the highest across scenarios) adds less than one second to workflows that operate on minutes-to-hours timescales.
12. Future Directions
12.1 Federated Hippocratic Learning
Health systems deploying Hippocratic Gates across multiple institutions accumulate gate evaluation data that could improve safety function calibration — but sharing this data across institutions raises privacy and competitive concerns. Federated learning techniques can address this: each institution trains a local safety function update on its gate evaluation data (safety scores, escalation outcomes, error detections) and shares only the model gradients or parameter updates, not the underlying patient data. The aggregated updates improve the safety function for all participating institutions without exposing PHI.
このアプローチは、複数の施設にわたる患者集団を合わせた方が、単一の施設のデータよりも代表的なトレーニング分布を提供する、母集団適用係数 s_5 にとって特に価値があります。ヒポクラテス連合学習は、現在過小評価されている集団に対する臨床 AI の安全性を制限している分布の偏りを軽減できる可能性があります。
12.2 Multi-Modal Safety Functions
現在の臨床 AI システムは、イメージング、ゲノミクス、電子医療記録、ウェアラブル センサー データ、患者から報告された結果など、複数のデータ モダリティにわたって動作することが増えています。ヒポクラテスの門のフレームワークは、モダリティにまたがる安全係数を定義することで、自然にマルチモーダル設定に拡張されます。
The cross-modal concordance terms capture consistency between modalities — for example, whether a genomic risk score is concordant with imaging findings and clinical history. Discordance between modalities is a strong signal for escalation, as it suggests that the clinical picture is complex and may exceed the AI model's training distribution.
12.3 Patient-Reported Safety Feedback
An unexplored dimension of clinical AI safety is patient-reported outcomes. Patients experience the consequences of AI-influenced clinical decisions and can provide safety-relevant feedback that is not captured by clinical metrics alone. Future versions of the Hippocratic Gate could incorporate patient-reported safety signals:
- AI が推奨する治療後の予期せぬ症状
- AI が生成した患者教育資料と実際の臨床経験との相違
- Accessibility concerns with AI-mediated clinical communications
- 患者の視点からの信頼性と透明性の評価
Incorporating patient-reported safety data into the safety function would close the loop between AI decision-making and patient experience, ensuring that safety is assessed from the perspective of the person most affected by clinical AI decisions.
12.4 Autonomous Safety Function Evolution
The current Hippocratic Gate framework requires human experts to define the safety function structure (the seven factors, their weights, and the threshold values). As gate evaluation data accumulates, machine learning techniques could be applied to discover new safety factors, optimize weights, and identify threshold values that minimize misdiagnosis probability.
However, autonomous safety function evolution raises a meta-governance challenge: who governs the system that governs clinical AI? The Hippocratic Gate must itself pass through a governance mechanism before its safety function is modified. We propose a hierarchical governance structure where safety function modifications are treated as Tier 4 actions (autonomous intervention) within the MARIA OS Decision Pipeline — requiring maximum gate strength and near-mandatory human oversight. This ensures that the Hippocratic Gate cannot evolve its own safety criteria without explicit human authorization.
12.5 Cross-Institutional Safety Benchmarking
ヒポクラティック ゲートは複数の医療システムに展開されるため、標準化された安全性ベンチマークにより、臨床 AI ガバナンスの品質を施設間で比較できるようになります。私たちは、施設全体のゲートパフォーマンス指標を集計するヒポクラテス安全指数 (HSI) を提案します。
ここで、MER は誤実行率、EPR はエラー伝播率、latency_score は臨床時間枠に対してゲート レイテンシを正規化し、alignment_score は選択した規制テーマに対する内部統制のカバレッジを測定します。教育機関は、ローカル スコアリング ルーブリックを定義し、各用語が実際に観察可能であることを検証した後でのみ、ピアの期待に対して HSI をベンチマークできます。
13. 結論
This paper has introduced the Hippocratic Gate — a formal, fail-closed governance primitive that transforms the ancient medical principle 'first, do no harm' from an ethical aspiration into an enforceable mathematical constraint. The key contributions are:
The Hippocratic Constraint S(a) >= theta formalizes patient safety as a pre-execution requirement for every clinical AI action. The safety function S is constructed from seven clinically meaningful factors — diagnostic confidence, evidence concordance, contraindication clearance, temporal stability, population applicability, reversibility index, and human oversight readiness — each measurable, interpretable, and auditable.
The Hippocratic Safety Bound (Theorem 1) is best read as a modeling device for reasoning about threshold choice, calibration quality, and escalation posture. Any numerical bound depends on deployment-specific assumptions and should be validated clinically before being used for operational claims.
The Clinical Risk Tier Framework defines four risk tiers with calibrated thresholds (theta = 0.70, 0.80, 0.85, 0.92) and gate strengths (g = 0.3, 0.5, 0.7, 0.9) that map to FDA SaMD categories and EU MDR risk classifications. The steeper clinical sigmoid (k = 12, theta_clinical = 0.40) ensures decisive escalation behavior appropriate for clinical contexts.
The Temporal Safety Dynamics model captures the fundamental clinical reality that patient state is dynamic. The safety validity window theorem provides computable bounds on how long a safety assessment remains valid, enabling continuous safety monitoring for long-duration clinical AI actions.
The Evidence Bundle Architecture defines minimum evidence dimensionality and freshness requirements by risk tier, with cryptographic integrity features that can support internal control mapping against HIPAA, FDA 21 CFR Part 11, and EU MDR documentation needs.
放射線科のケーススタディは、ガバナンス設計のための例示的な再現シナリオとして読む必要があります。その目的は、実行前のゲート チェックがワークフローの動作をどのように変える可能性があるかを示すことであり、将来の検証なしに 1 つのアーキテクチャが臨床ガバナンスのギャップを埋めると主張することではありません。
The Hippocratic Gate does not make clinical AI models more accurate. It makes clinical AI deployments more safe by ensuring that every AI action passes through a formal safety check before it can influence patient care. This distinction is critical: model accuracy is a machine learning problem; deployment safety is a governance problem. The Hippocratic Gate solves the governance problem.
Clinical AI will continue to expand into every domain of medicine — from screening to surgery, from diagnostics to therapeutics, from monitoring to autonomous intervention. As this expansion accelerates, the governance infrastructure must expand with it. The Hippocratic Gate is best understood as a reusable safety-governance pattern that still requires clinical validation, local policy work, and regulatory review before any institution treats it as operational fact.
参考文献
- [1] Makary, M.A. and Daniel, M. (2016). "Medical error — the third leading cause of death in the US." BMJ, 353:i2139. Foundational epidemiological analysis establishing the scale of medical error as a public health crisis.
- [2] トポル、E.J. (2019年)。 「高性能医療: 人間と人工知能の融合」自然医学、25(1):44-56。臨床 AI の機能と、臨床ワークフローに AI を導入する際のガバナンスの課題を包括的にレビューします。
- [3] 米国食品医薬品局。 (2021年)。 「医療機器としての人工知能/機械学習 (AI/ML) ベースのソフトウェア (SaMD) アクション プラン」。 AI ベースの臨床ソフトウェアを規制するための FDA フレームワーク (事前に決定された変更管理計画 (PCCP) の概念を含む)。
- [4] European Parliament. (2017). "Regulation (EU) 2017/745 — Medical Device Regulation." Official Journal of the European Union. EU legal framework for medical device classification, clinical evaluation, and post-market surveillance.
- [5] Guo, C., et al. (2017). "On Calibration of Modern Neural Networks." ICML 2017. Demonstrates that modern neural networks are poorly calibrated and introduces temperature scaling — the calibration method used for the diagnostic confidence factor s_1.
- [6] Obermeyer, Z., et al. (2019). "Dissecting racial bias in an algorithm used to manage the health of populations." Science, 366(6464):447-453. Demonstrates how clinical AI systems can exhibit racial bias, motivating the population applicability factor s_5.
- [7] Lyell、D.、他。 (2017年)。 「自動化のバイアスと検証の複雑さ: 系統的なレビュー」米国医療情報学会誌、24(2):423-431。体系的レビューにより、臨床医が臨床意思決定支援システムを使用する際に自動化バイアスが見られることが確立され、セクション 1.2 のガバナンス ギャップ分析が検証されます。
- [8] Rajpurkar, P., et al. (2017). "CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning." arXiv:1711.05225. Foundational work on deep learning for chest X-ray interpretation, providing context for the radiology case study.
- [9] Elmore, J.G., et al. (2015). "Diagnostic Concordance Among Pathologists Interpreting Breast Biopsy Specimens." JAMA, 313(11):1122-1132. Establishes baseline diagnostic variability among human clinicians, contextualizing AI error rates.
- [10] Sendak, M.P., et al. (2020). "A Path for Translation of Machine Learning Products into Healthcare Delivery." EMJ Innovations. Practical framework for clinical AI deployment that identifies governance as the critical gap between model development and clinical impact.
- [11] U.S. Department of Health and Human Services. (2013). "HIPAA Security Rule." 45 CFR Part 164. Federal regulations for protecting electronic protected health information (ePHI) that inform the audit architecture design.
- [12] Boyd, S. and Vandenberghe, L. (2004). "Convex Optimization." Cambridge University Press. Standard reference for optimization theory used in gate strength allocation and threshold optimization.
- [13] アモデイ、D.、他。 (2016年)。 「AIの安全性における具体的な問題」 arXiv:1606.06565。 AI の安全性に関する課題の基本的な分類法。展開時の安全性メカニズムとしてのヒポクラテスの門の理論的コンテキストを提供します。
- [14] Hollnagel、E. (2014)。 「安全Ⅰと安全Ⅱ:安全管理のこれまでとこれから」アッシュゲート。安全を故障の不在ではなくガバナンスの存在として理解するためのフレームワーク。これにより、プロアクティブな安全監視アプローチが動機付けられます。
- [15] MARIA OS Technical Documentation. (2026). Internal architecture specification for the Hippocratic Gate Engine, Clinical Decision Pipeline, and Healthcare MARIA Coordinate System.