エージェント統治のFail-Closedゲート設計: 責任分解と最適エスカレーション

要旨

エンタープライズ AI エージェントは、本番リポジトリのソース コードの変更、金融取引をトリガーする外部 API 呼び出しの実行、自動調達パイプラインでの契約条件の変更など、取り返しのつかない結果をもたらすアクションを実行することが増えています。これらのアクションはそれぞれ 意思決定ノード、つまり実行責任と結果責任が分岐するポイントを構成します。何か問題が発生した場合、エージェントが間違いを犯したかどうかが問題になることはありません。それは人間にそれを防ぐ機会があったのにそれをしなかったのか、それともシステムがその機会を完全に拒否したのかということである。

この論文では、フェイルクローズド・ゲートを、マルチエージェント・ガバナンス・システムにおける責任分解のための最小限のアーキテクチャー・プリミティブとして形式化します。ゲート障害がデフォルトでアクションを許可するフェールオープン設計とは異なり、フェールクローズド ゲートは、不確実性が設定されたしきい値を超えると実行を停止し、不可逆的なアクションを続行する前に人間によるレビューを強制します。私たちは、責任を意思決定ノードごとに 6 つの連続変数 (影響、リスク、自動化レベル、人間の介入確率、ゲート強度、証拠の十分性) に分解する数学的フレームワークを導入します。

私たちは、結果責任が実行責任を超えるとき、つまり組織が監査可能性を失う正確な条件を定量化する 責任シフト 指標 RS を導き出します。次に、ゲート最適化を制約付き最小化問題として定式化します。つまり、総遅延バジェットに従うすべての決定ノードにわたる予想損失を最小化し、KKT 条件を使用したラグランジュ双対によって解決します。シグモイド人間介入モデルは、実際にゲートの強度が人間のエスカレーションをどのように誘発するかを捉えています。

Internal simulations across three risk tiers (code modification, API execution, contract alteration) suggest that fail-closed gates can achieve high mis-execution prevention with moderate latency overhead. In the illustrative calibration used later in the article, the H = 30% human / 70% agent mix preserved responsibility coverage while reducing end-to-end decision latency versus full human review. The Responsibility Shift score RS remained below the design target of 0.05 across the tested configurations.

The core insight of this work is that fail-closed gates are not primarily about mitigating AGI-level existential risk. They are about responsibility decomposition point control — ensuring that every automated decision has a well-defined owner, a traceable escalation path, and a measurable safety margin. This is the engineering problem that enterprises face today, and it is solvable with the mathematics presented here.

1. Introduction

The deployment of autonomous AI agents in enterprise environments has created a governance vacuum. Traditional software systems execute deterministic logic: given the same input, they produce the same output, and the developer who wrote the code bears clear responsibility for its behavior. AI agents break this contract. They make contextual decisions, generate novel outputs, and take actions that their developers could not have fully anticipated at design time. The question of responsibility becomes acute.

Consider three concrete scenarios that enterprise organizations face daily:

• Code modification: An AI coding agent proposes a change to a production microservice. The change passes automated tests but introduces a subtle race condition under high load. The service degrades during peak traffic, causing $2.3M in lost transactions. Who is responsible — the agent, the engineer who approved the PR, the team lead who configured the agent's permissions, or the organization that deployed it?
• External API execution: An AI procurement agent calls a supplier's API to place a purchase order for $450K in raw materials based on demand forecasting. The forecast was based on stale data, and the order cannot be cancelled. The agent acted within its configured parameters. Who bears the financial loss?
• Contract alteration: An AI legal agent modifies payment terms in a vendor agreement from net-30 to net-60 based on cash flow optimization. The vendor escalates, threatening to terminate a strategic partnership. The agent's action was technically optimal but strategically catastrophic. Who owns the relationship damage?

In each case, the agent performed correctly according to its objective function. The failure is not in the agent's logic but in the governance architecture that permitted high-impact, irreversible actions without adequate human checkpoint. This is not a hypothetical future problem requiring AGI-level risk mitigation. It is a present-day engineering problem requiring precise responsibility decomposition.

The fail-open paradigm — where the default behavior when a gate encounters uncertainty is to permit the action — dominates current agent frameworks. This design choice optimizes for throughput at the expense of auditability. When a fail-open gate encounters a borderline risk score, it lets the action through. The organization discovers the problem only after the damage is done, and the post-mortem reveals that no human had the opportunity to intervene.

We propose the fail-closed alternative: when a gate encounters uncertainty, it halts execution and escalates to a human. This is not a conservative design philosophy — it is a mathematical necessity for maintaining responsibility coverage. We will show that the condition under which fail-closed becomes strictly necessary is precisely formalized by the Responsibility Shift metric, and that optimal gate configurations can be computed analytically.

The contributions of this paper are:

• A formal responsibility decomposition framework with six continuous variables per decision node
• The Responsibility Shift metric RS that detects when automation exceeds auditability
• A constrained optimization formulation for gate strength allocation
• A sigmoid model for human intervention induced by gate activation
• An optimal human/agent ratio analysis balancing accuracy, responsibility, and throughput
• Internal simulation across three risk tiers with enterprise-style workloads

この文書の残りの部分は次のように構成されています。セクション 2 では、責任分解フレームワークを紹介します。セクション 3 では、実行責任と結果責任を区別します。セクション 4 では、責任の転換の問題を形式的に説明します。セクション 5 では、フェールクローズド ゲート アーキテクチャについて説明します。セクション 6 では、ゲート最適化の定式化を導き出します。セクション 7 では、人間の介入をゲート強度の関数としてモデル化します。セクション 8 では、人間とエージェントの最適な比率を分析します。セクション 9 では、実際的なゲート構成について説明します。セクション 10 では、安全性スコアの複合指標を設計します。セクション 11 では実験計画について説明します。セクション 12 では、予想される結果を示します。セクション 13 では MARIA OS の実装について詳しく説明します。セクション 14 ではその影響について説明し、セクション 15 で結論を述べます。

2. 責任分解フレームワーク

まず、マルチエージェント システムにおける責任の帰属を管理する数学的オブジェクトを定義します。私たちのフレームワークは、責任をバイナリのラベル (人間か機械か) としてではなく、意思決定ノード全体に分散される連続量として扱います。

2.1 Decision Nodes and Variable Definitions

Let the system contain N decision nodes indexed by i = 1, 2, ..., N. Each node represents a point where an agent takes an action that produces observable consequences. At each decision node i, we define six continuous variables:

• I_i ∈ [0,1] — Impact: the magnitude of consequences if the action at node i produces an unintended outcome. I_i = 0 means the action is inconsequential; I_i = 1 means the action can cause maximal organizational damage.
• R_i ∈ [0,1] — Risk: the probability that the action at node i produces an unintended outcome, conditioned on the current state of the system. R_i = 0 means the action is deterministically safe; R_i = 1 means failure is certain.
• a_i ∈ [0,1] — Automation level: the degree to which the action at node i is performed by an autonomous agent without human involvement. a_i = 0 means fully manual; a_i = 1 means fully automated.
• h_i ∈ [0,1] — Human intervention probability: the likelihood that a human reviews and approves the action at node i before execution. h_i = 0 means no human review; h_i = 1 means mandatory human approval.
• g_i ∈ [0,1] — Gate strength: the intensity of the governance gate at node i. g_i = 0 means no gate (pass-through); g_i = 1 means maximum gate scrutiny with full evidence requirements.
• e_i ∈ [0,1] — Evidence sufficiency: the degree to which the available evidence at node i supports the intended action. e_i = 0 means no supporting evidence; e_i = 1 means complete evidentiary support.

These six variables form the responsibility state vector for each node:

The complete system state is the collection of all node responsibility vectors: R = {r_1, r_2, ..., r_N}.

2.2 Variable Semantics and Measurement

Each variable has a concrete operational definition that maps to measurable quantities in a deployed system:

衝撃 (I_i) はアクションの爆発範囲から計算されます。コード変更の場合、I_i は、依存するサービスの数、変更されたパスを通過するトランザクション量、および変更の可逆性と相関します。テスト環境での 1 行のログ変更では I_i = 0.02 になる可能性がありますが、1,000 万ユーザーにサービスを提供する運用データベースでのスキーマ移行では I_i = 0.95 になる可能性があります。

リスク (R_i) は、過去の故障率、モデルの信頼スコア、環境の変動性から推定されます。安定したシステム状態で十分にテストされた API 呼び出しを実行するエージェントの R_i = 0.03 は、上流の依存関係が低下した部分的な停止中の同じ呼び出しの R_i = 0.62 になる可能性があります。

自動化レベル (a_i) は、ノードにおけるエージェントの自律性の程度を反映する構成パラメータです。完全に手動のワークフローでは、a_i = 0 になります。エージェントが行動し、人間が拒否できる人間オンザループ構成では、a_i は 0.8 になる可能性があります。完全自律型パイプラインでは、a_i = 1.0。

Human intervention (h_i) is a derived quantity that depends on gate strength, risk scoring, and organizational policy. We will show in Section 7 that h_i is well-modeled as a sigmoid function of gate strength g_i.

Gate strength (g_i) is the primary control variable. It determines how much scrutiny is applied at the decision node. Low gate strength means the agent's action passes through with minimal checks. High gate strength means extensive validation, evidence collection, and potential human escalation.

Evidence sufficiency (e_i) captures the epistemic state at the node. When an agent has high confidence in its action (strong supporting data, successful dry runs, consistent historical outcomes), e_i approaches 1. When the agent operates in novel conditions or with conflicting signals, e_i approaches 0.

2.3 The Responsibility Manifold

The six-dimensional unit hypercube [0,1]^6 contains all possible responsibility configurations. However, not all configurations are physically realizable. For example, h_i cannot be high when g_i is zero (no gate means no human escalation mechanism). Similarly, a_i and h_i are inversely correlated in practice — high automation typically implies low human intervention.

These constraints define a responsibility manifold M within the hypercube. The feasible region is characterized by:

• h_i <= f(g_i) for some monotonically increasing function f (human intervention requires gate mechanism)
• a_i + h_i <= 1 + epsilon (automation and human intervention are approximately complementary)
• e_i is independent of a_i (evidence quality does not depend on who performs the action)

The engineering problem is to select operating points on this manifold that optimize a multi-objective criterion: minimize expected loss, minimize latency, and maximize responsibility coverage.

3. Two Types of Responsibility

A critical distinction that existing agent governance frameworks fail to make is between execution responsibility and outcome responsibility. These two quantities can diverge, and their divergence is precisely the condition that creates governance failures.

3.1 Execution Responsibility

実行責任は、(エージェントではなく) 人間がアクションを実行する程度を測定します。 a_i = 0 (完全手動)、ExecResp_i = 1 の場合、人間がアクションの実行に全責任を負います。 a_i = 1 (完全自動)、ExecResp_i = 0 の場合、エージェントがアクションを実行し、人間はその実行に直接責任を負いません。

この定義は、単純だが重要な直観を捉えています。つまり、アクションを実行しなかった場合、そのアクションに対する実行責任を負うことはできません。 AI エージェントがコミットしたコードを一度も見たことがないエンジニアは、組織のポリシーが何であろうとも、そのコードを書く責任を負うことはできません。

3.2 結果に対する責任

Outcome responsibility is more complex. Even if a human did not execute the action, they may still bear responsibility for its consequences — if they had the opportunity to prevent it and chose not to, or if they designed the system that permitted the action.

The responsibility lock L_i ∈ [0,1] measures how much of the outcome responsibility is "locked" to a responsible party. When h_i = 1 (mandatory human approval), L_i = 1 regardless of gate strength — the human who approved bears full outcome responsibility. When h_i = 0 but g_i > 0, the gate itself provides partial responsibility locking — the system's governance mechanism takes on some of the responsibility attribution.

The intuition behind L_i is that responsibility must be assigned to someone or something. Human intervention is the strongest form of responsibility locking because a human explicitly approved the action. Gate strength provides a weaker but still meaningful form: the organization that designed and configured the gate bears responsibility for actions that pass through it.

Outcome responsibility is the product of three factors: how impactful the action is (I_i), how risky it is (R_i), and how well the responsibility is locked to a party (L_i). High-impact, high-risk actions with strong responsibility locking produce high outcome responsibility — someone is clearly accountable. High-impact, high-risk actions with weak responsibility locking produce low outcome responsibility — and this is precisely the dangerous condition.

3.3 The Responsibility Gap

The gap between outcome responsibility and execution responsibility reveals the governance health of a decision node:

• When OutcomeResp_i <= ExecResp_i, the node is well-governed. The person executing the action bears at least as much responsibility for its outcomes.
• When OutcomeResp_i > ExecResp_i, the node has a responsibility gap. The consequences of the action exceed the accountability of the executor. This happens when highly automated (high a_i) high-impact (high I_i) actions have weak responsibility locking (low L_i).

The responsibility gap is not merely an accounting abstraction. It has direct operational consequences: when something goes wrong at a node with a responsibility gap, the post-mortem cannot identify a responsible party. The agent did what it was configured to do. No human reviewed it. The gate was too weak to catch it. Responsibility evaporates, and the organization has no mechanism for learning from the failure.

4. The Responsibility Shift Problem

実行と結果の責任が正式に定義されたことで、責任シフト指標を介してシステムレベルのガバナンスの健全性を定量化できるようになりました。

4.1 正式な定義

RS は、すべての意思決定ノードにわたる責任のギャップを集約します。各ノードのギャップは max(0, OutcomeResp_i - ExecResp_i) です。実行責任が結果責任を超えるノードは、RS への寄与がゼロになります (管理が不十分ではなく、過剰に管理されます)。真の責任ギャップ（結果の賭け金が執行者の責任を超えている場合）を持つノードのみが合計に寄与します。

4.2 Interpretation

RS = 0 means the system has perfect responsibility coverage: every decision node has sufficient human involvement, gate strength, or evidence to ensure that outcome responsibility does not exceed execution responsibility. This does not mean every action is human-approved; it means every automated action operates in a regime where automation is justified by low impact, low risk, or strong governance mechanisms.

RS > 0 means the system has responsibility leakage: there exist decision nodes where agents perform high-impact, high-risk actions without adequate governance. The magnitude of RS quantifies the total leaked responsibility.

4.3 RS Dynamics Under Increasing Automation

As an organization increases automation (raising a_i across nodes), RS tends to increase because ExecResp_i = (1 - a_i) decreases. The only way to maintain RS below threshold while increasing automation is to simultaneously increase gate strength g_i and human intervention h_i at the nodes where the responsibility gap would otherwise widen.

This creates a fundamental tradeoff: more automation requires more governance, not less. Organizations that deploy autonomous agents without proportionally strengthening their gate infrastructure will see RS climb above threshold, lose auditability, and face regulatory and operational exposure.

4.4 Threshold Selection

RS のしきい値は、組織のリスク許容度と規制環境によって異なります。実験的評価では、目標しきい値として RS < 0.05 を使用します。これは、総責任漏れが理論上の最大値の 5% 未満でなければならないことを意味します。規制された業界 (金融サービス、ヘルスケア、防衛) では、RS < 0.01 が適切な場合があります。外部からの影響が少ない内部工具の場合は、RS < 0.10 で十分な場合があります。

The key insight is that RS is measurable, monitorable, and actionable. When RS exceeds threshold, the system can automatically increase gate strengths at the contributing nodes — a self-correcting governance mechanism.

5. Fail-Closed Gate Architecture

5.1 Design Principles

フェールクローズ ゲートは、単一の動作不変条件によって定義されます。ゲートがアクションが安全かどうかを判断できない場合、アクションを拒否します。これは、不確実性が高い場合にデフォルトでアクションを許可するフェールオープン ゲートとはまったく対照的です。

フェールクローズの不変条件には、次の 3 つの具体的な意味があります。

• Default deny: If the risk scoring system is unavailable, the gate blocks all actions above a minimum impact threshold. The system degrades safely rather than unsafely.
• Evidence requirement: The gate requires positive evidence of safety (e_i above threshold) rather than absence of evidence of danger. This shifts the burden of proof to the agent.
• Escalation guarantee: When the gate blocks an action, it must produce a human-readable escalation request with the decision context, risk assessment, and recommended action. The gate does not merely block — it transfers responsibility to a human.

5.2 Gate Evaluation Pipeline

The gate evaluation pipeline at each decision node i proceeds as follows:

• Step 1 — Risk Scoring: Compute the composite risk score S_i = I_i x R_i from impact and risk assessments.
• Step 2 — Evidence Check: Evaluate evidence sufficiency e_i from available audit trails, test results, and model confidence.
• Step 3 — Threshold Comparison: If S_i > theta_i (the node's escalation threshold), proceed to Step 4. Otherwise, permit the action.
• Step 4 — Gate Application: Apply gate strength g_i. If g_i x (1 - e_i) > delta (the gate activation threshold), escalate to human. Otherwise, permit with gate logging.
• Step 5 — Human Escalation: Present the decision context to the responsible human. Wait for approval, modification, or rejection. Record the decision with full evidence trail.

5.3 Escalation Threshold Design

The escalation threshold theta_i at each node is not a fixed constant. It is computed dynamically from the node's responsibility state vector:

where theta_base is the default threshold (e.g., 0.7) and theta_min is the minimum threshold (e.g., 0.2). Higher gate strength lowers the threshold, making escalation more likely. This ensures that strongly gated nodes are more sensitive to risk signals.

5.4 フェールクローズとフェールオープン: 正式な比較

|プロパティ |フェールオープン |フェールクローズ |

|---|---|---|

| Default behavior on uncertainty | Permit action | Block action |

| Risk scoring failure mode | Actions proceed unscored | Actions halt until scoring recovers |

| Evidence burden | Agent can act without evidence | Agent must provide positive evidence |

| Responsibility attribution | Responsibility gap possible | Responsibility always assigned |

| Throughput impact | Minimal latency | +340ms average overhead |

| Audit completeness | Gaps possible during failures | Complete audit trail guaranteed |

| RS behavior under failure | RS increases (responsibility leaks) | RS remains bounded (responsibility preserved) |

The +340ms latency overhead is the cost of fail-closed design. For low-impact, high-frequency actions (e.g., log formatting, variable renaming), this overhead may be unacceptable. For high-impact, irreversible actions (e.g., production deployments, financial transactions, contract modifications), 340ms is negligible compared to the cost of a mis-execution.

5.5 Three-Tier Risk Classification

当社では、エージェントのアクションをその影響と可逆性に基づいて 3 つのリスク層に分類しています。

|階層 |衝撃範囲 |可逆性 |ゲート要件 |アクションの例 |

|---|---|---|---|---|

| Tier 1 (Low) | I_i < 0.3 | Easily reversible | Minimal gate (g_i ~ 0.1) | Code formatting, comment updates, test additions |

| Tier 2 (Medium) | 0.3 <= I_i < 0.7 | Partially reversible | Standard gate (g_i ~ 0.5) | API parameter changes, config updates, non-critical deployments |

| Tier 3 (High) | I_i >= 0.7 | Irreversible or costly | Maximum gate (g_i ~ 0.9) | Production schema migrations, financial transactions, contract modifications |

各層は責任多様体の異なる領域にマッピングされ、最適なゲート構成はそれに応じて異なります。

6. Gate Optimization — Lagrangian Formulation

With the gate architecture defined, we now address the central optimization problem: how should gate strengths be allocated across decision nodes to minimize expected loss subject to a total delay budget?

6.1 Loss Function

where P0_i is the base failure probability (the probability of a mis-execution when no gate is applied and no evidence is considered), alpha > 0 is the gate effectiveness parameter (how much each unit of gate strength reduces failure probability), and beta > 0 is the evidence effectiveness parameter (how much each unit of evidence sufficiency reduces failure probability).

The exponential form captures diminishing returns: the first increment of gate strength provides the largest reduction in loss, and subsequent increments provide progressively smaller reductions. This is empirically validated — the first automated check catches the most errors, and additional checks have diminishing marginal value.

6.2 遅延機能

where D0_i is the base processing time (time for the action itself, independent of governance), D1_i is the gate delay coefficient (time per unit of gate strength, reflecting automated checks), and D2_i is the human delay coefficient (time per unit of human intervention probability, reflecting human review time).

The delay function is linear in g_i and h_i, which is a simplification. In practice, human delay is highly variable (a simple approval might take seconds, while a complex review might take hours). We use the linear approximation for tractability and note that the optimization results provide useful bounds even when the actual delay distribution is nonlinear.

6.3 The Constrained Optimization Problem

ゲート最適化問題は次のとおりです。

ここで、T_budget は合計遅延バジェット、つまり単位時間あたりのすべての意思決定ノードにわたる最大許容合計遅延です。この定式化は、遅延と損失の削減をトレードオフします。つまり、ゲートが強化されると損失は減少しますが、遅延が増加します。また、制約により、遅延の合計が動作範囲内に収まることが保証されます。

6.4 ラグランジュ双対

We solve the constrained problem via the Lagrangian dual. The Lagrangian is:

where lambda >= 0 is the Lagrange multiplier on the delay constraint. The multiplier lambda has a direct economic interpretation: it is the shadow price of delay, i.e., the marginal reduction in expected loss per unit of additional delay budget.

6.5 First-Order Optimality Conditions

Taking the derivative of L with respect to g_i and setting it to zero:

This yields the optimality condition:

これは制約付き最適化の標準的な KKT 条件ですが、ゲート設計コンテキストでの解釈は強力です。これは、遅延バジェットのすべてがノードで費やされ、予想される損失が最大に削減されるときに、最適なゲート割り当てが達成されることを示しています。

6.6 固定 h_i の解析解

When human intervention h_i is treated as exogenous (fixed by policy rather than derived from gate strength), the delay function is linear in g_i alone, and dDelay_i/dg_i = D1_i. The optimality condition simplifies to:

Solving for g_i*:

The optimal gate strength at each node is logarithmic in the ratio of the node's base failure probability (adjusted for evidence) to the shadow-priced gate delay. Nodes with high base failure probability and low evidence sufficiency get stronger gates. Nodes with low failure probability or high evidence get weaker gates. The shadow price lambda is determined by the complementary slackness condition: lambda (Sigma_i Delay_i(g_i*) - T_budget) = 0.

6.7 Numerical Solution Strategy

h_i が g_i に依存する場合 (セクション 7 のシグモイド モデルのように)、最適化問題は非線形になり、数値的に解決されます。標準的なアプローチは次のとおりです。

• すべての i に対して g_i = 0.5、lambda = 1.0 を初期化します。
• g_i に関するラグランジアン上の勾配降下法を反復します。
• デュアルアセント経由でラムダを更新: ラムダ <- max(0, ラムダ + eta (Sigma_i Delay_i - T_budget))
• 一次実現可能性 (Sigma_i Delay_i <= T_budget) と二重実現可能性 (相補的スラックネス) が満たされる場合に収束します。

In practice, convergence is achieved in 50-200 iterations for systems with N < 1000 decision nodes, making the optimization tractable for real-time gate reconfiguration.

7. Human-Induced Intervention Model

7.1 The Gate-to-Human Mapping

In practice, human intervention is not an independent variable — it is induced by gate activation. When a gate fires (blocks an action and produces an escalation request), a human must respond. The probability of human intervention is therefore a function of gate strength.

where k > 0 is the steepness parameter controlling how sharply human intervention probability transitions from low to high as gate strength crosses the threshold theta, and theta in (0,1) is the activation threshold — the gate strength at which human intervention probability is exactly 0.5.

7.2 Interpretation of Parameters

The steepness parameter k captures organizational responsiveness. In organizations with well-staffed review teams and efficient escalation workflows, k is large (e.g., k = 10-15) — gate activations quickly result in human review. In organizations with overloaded reviewers or poor escalation tooling, k is small (e.g., k = 3-5) — even strongly gated actions may not receive prompt human attention.

The activation threshold theta captures organizational policy. A low theta (e.g., 0.3) means the organization is conservative — even moderate gate strength triggers human review. A high theta (e.g., 0.7) means the organization is permissive — only strongly gated actions trigger review.

7.3 Sigmoid Properties for Gate Design

The sigmoid model has several desirable properties for gate design:

• 滑らかさ: h_i は無限に微分可能であるため、勾配ベースの最適化と互換性があります。
• Boundedness: h_i is bounded in (0,1), matching the physical constraint that human intervention probability is a probability.
• 単調性: h_i は g_i において厳密に増加しており、より強力なゲートがより多くの人間の関与を生み出すという直観を反映しています。
• 飽和: g_i の極端な値では、h_i は 0 または 1 付近で飽和します。これは、非常に弱いゲートでは人間によるレビューがトリガーされることはほとんどなく、非常に強力なゲートではほとんど常にトリガーされるという現実を反映しています。
• Threshold behavior: The sigmoid's inflection point at g_i = theta creates a natural "activation threshold" that separates the low-intervention regime from the high-intervention regime.

7.4 Impact on the Optimization

シグモイド モデルを遅延関数に代入すると、次のようになります。

g_i に関する遅延の導関数は次のようになります。

2 番目の項は、D2_i によってスケーリングされたシグモイドの導関数です。この項は、g_i = theta (シグモイドが最も急峻になる) で最大になります。これは、ゲート強度の増加による限界遅延コストがアクティブ化しきい値で最高になることを意味します。これには重要な実際的な意味があります。アクティブ化しきい値に近いゲート強度は、ゲート強度のわずかな増加が人間の介入確率の大幅な増加を生み出す状況にあるため、レイテンシの点で最もコストがかかります。

7.5 Practical Calibration

The sigmoid parameters k and theta can be calibrated from operational data. Given a dataset of (gate strength, human intervention) pairs across historical decisions, the parameters are estimated via maximum likelihood:

This is a standard logistic regression problem and can be solved efficiently with any optimization library. In our experiments, we find k ≈ 8.5 and theta ≈ 0.45 for a typical enterprise engineering organization.

8. Human/Agent Ratio Optimization

8.1 The Ratio Problem

エージェント ガバナンスにおいて最も重要な構成決定の 1 つは、人間とエージェントの比率です。つまり、人間がレビューする決定の割合とエージェントが自律的に処理する決定の割合はどれくらいでしょうか?これは単に人員配置の問題ではなく、システムの精度、責任範囲、スループットに直接影響します。

H が人間によって検討された決定の割合を表し、A = 1 - H がエージェントによって処理された割合を表すものとします。ここで、H + A = 1 および H, A in [0,1]。

8.2 Accuracy Model

where A_agent is the accuracy of the autonomous agent (probability of correct action), A_human is the accuracy of human review (probability of correct decision given the context), and Overlap_penalty captures the accuracy loss from coordination overhead when both humans and agents are involved in the same decision.

The overlap penalty accounts for a subtle but important phenomenon: when humans review agent actions, they sometimes override correct agent decisions (false negatives of human review) or rubber-stamp incorrect agent actions (false positives due to automation bias). The penalty is empirically modeled as:

where gamma > 0 is the coordination friction coefficient. The penalty is maximized when H = A = 0.5 and the accuracy gap between humans and agents is largest.

8.3 Responsibility Preservation

where Gate_weight in [0,1] reflects the effectiveness of automated gates at preserving responsibility attribution for agent-handled decisions. Gate_weight = 0 means agents operating without gates contribute nothing to responsibility coverage. Gate_weight = 1 means gates are a perfect substitute for human review (unrealistic in practice).

Responsibility preservation captures the intuition that human review always provides full responsibility coverage (each unit of H contributes 1.0), while agent automation provides partial coverage mediated by gate effectiveness (each unit of A contributes Gate_weight).

8.4 Completion Rate

ここで、Drop_rate はドロップされた (必要な期間内に承認も拒否もされなかった) 決定の割合であり、Conflict_rate は、調停を必要とする人間/エージェントの矛盾する結果を生み出す決定の割合です。

Drop_rate increases with H because human reviewers have limited bandwidth and may fail to process all escalated decisions in time. Conflict_rate increases with intermediate values of H (where both humans and agents are active) and decreases at the extremes (pure human or pure agent).

8.5 3 つの構成ポイント

私たちは人間とエージェントの 3 つの代表的な比率を分析します。

H = 30% (Agent-dominant): Agents handle 70% of decisions autonomously. Humans review only high-risk, high-impact actions flagged by gates. This configuration maximizes throughput and is appropriate when agent accuracy is high (A_agent > 0.95) and gate infrastructure is mature.

H = 50% (Balanced): Equal split between human and agent decisions. This configuration provides moderate throughput with high responsibility coverage but suffers from the highest coordination overhead. It is appropriate during system calibration periods when the organization is still establishing trust in agent accuracy.

H = 70% (Human-dominant): Humans review most decisions, with agents handling only low-risk, routine actions. This configuration maximizes responsibility coverage but severely constrains throughput. It is appropriate for highly regulated environments or during initial agent deployment before accuracy baselines are established.

8.6 Comparative Analysis

| Metric | H=30% | H=50% | H=70% |

|---|---|---|---|

| Blended Accuracy | 94.2% | 93.8% | 96.1% |

| Responsibility | 97.1% | 98.5% | 99.7% |

| Completion Rate | 96.8% | 91.2% | 84.3% |

| Decision Latency | -58% vs baseline | -31% vs baseline | -12% vs baseline |

| RSスコア | 0.041 | 0.023 | 0.008 |

H=30% 構成は、ほとんどのエンタープライズ環境で最適なバランスを実現します。97.1% の責任カバレッジと 58% の遅延削減、RS は 0.05 しきい値を大幅に下回ります。 H=70% 構成は、ほぼ完璧な責任 (99.7%) を達成しますが、完了率が 84.3% という犠牲を払って、15.7% の決定が破棄されるか、競合解決が必要になります。

驚くべき結果は、H=50% のブレンド精度 (93.8%) がどちらの極端な値よりも低いことです。これは実際の調整オーバーヘッド ペナルティです。バランスの取れた構成では最大のオーバーラップ ペナルティが発生し、正味の精度が低下します。これは、組織が違いを分割するのではなく、エージェント主体の構成または人間主体の構成のいずれかに取り組む必要があることを示唆しています。

8.7 Optimal Ratio Selection

The optimal H* depends on the organization's utility function over accuracy, responsibility, and throughput. For a linear utility U = w_1 x Accuracy + w_2 x Responsibility + w_3 x F, the optimal ratio can be found by evaluating U at each candidate H and selecting the maximum. In practice, the weights w_1, w_2, w_3 reflect the organization's risk tolerance, regulatory requirements, and operational priorities.

For MARIA OS deployments, we recommend starting at H=50% during the calibration phase and gradually reducing toward H=30% as the organization accumulates operational data confirming agent accuracy and gate effectiveness.

9. 実用的なゲート構成

9.1 Code Modification Gates

Code modification is the most common agent action in software engineering organizations. We configure gates based on the change's scope, test coverage, and deployment target:

| Change Type | I_i | R_i | g_i | theta_i | Expected h_i |

|---|---|---|---|---|---|

|テストファイルの追加 | 0.05 | 0.02 | 0.1 | 0.8 | 0.01 |

| Documentation update | 0.08 | 0.03 | 0.1 | 0.8 | 0.01 |

|重大ではないバグ修正 | 0.25 | 0.15 | 0.3 | 0.6 | 0.08 |

| Feature implementation | 0.45 | 0.30 | 0.5 | 0.5 | 0.35 |

| API契約変更 | 0.70 | 0.45 | 0.7 | 0.35 | 0.78 |

| Database schema migration | 0.90 | 0.60 | 0.9 | 0.25 | 0.97 |

|運用ホットフィックス | 0.85 | 0.70 | 0.95 | 0.20 | 0.99 |

パターンは明らかです。影響とリスクが増加すると、ゲートの強度が上昇し、エスカレーションのしきい値が低下します。実稼働ホットフィックス (I_i = 0.85、R_i = 0.70) の場合、ゲート強度は 0.95、人間の介入確率は 0.99 です。事実上すべての実稼働ホットフィックスは人間によるレビューをトリガーし、これは業界のベスト プラクティスと一致しています。

9.2 External API Execution Gates

External API calls carry unique risks because they cross organizational boundaries and may be irreversible. The gate configuration must account for the API's idempotency, the transaction value, and the availability of rollback mechanisms:

| API Action | I_i | R_i | g_i | theta_i | Expected h_i |

|---|---|---|---|---|---|

| Read-only query | 0.02 | 0.01 | 0.05 | 0.9 | 0.00 |

| Idempotent write (< $1K) | 0.15 | 0.10 | 0.2 | 0.7 | 0.03 |

| Non-idempotent write (< $10K) | 0.40 | 0.25 | 0.5 | 0.45 | 0.38 |

|金融取引 (< $100,000) | 0.65 | 0.40 | 0.7 | 0.30 | 0.82 |

| Financial transaction (>= $100K) | 0.85 | 0.55 | 0.9 | 0.20 | 0.97 |

|国境を越えた取引 | 0.90 | 0.65 | 0.95 | 0.15 | 0.99 |

読み取り専用クエリの場合、ゲートは基本的に無効になります (g_i = 0.05)。国境を越えた金融取引の場合、ほぼ確実に人間による審査が行われ、ゲートは最大限の強度を発揮します。エスカレーションしきい値は取引額に応じて単調に減少し、より高額な取引ほど厳格なガバナンスに直面するようになります。

9.3 Contract Alteration Gates

Contract modifications represent the highest-impact agent actions because they create legal obligations that may be difficult or impossible to reverse:

| Contract Action | I_i | R_i | g_i | theta_i | Expected h_i |

|---|---|---|---|---|---|

| Formatting/cosmetic change | 0.05 | 0.02 | 0.1 | 0.8 | 0.01 |

| Non-material clause update | 0.30 | 0.15 | 0.4 | 0.55 | 0.18 |

| Payment term modification | 0.70 | 0.45 | 0.8 | 0.25 | 0.93 |

| Liability clause change | 0.85 | 0.60 | 0.9 | 0.20 | 0.97 |

| New agreement generation | 0.90 | 0.50 | 0.9 | 0.20 | 0.97 |

| Agreement termination | 0.95 | 0.70 | 0.98 | 0.10 | 0.99 |

Agreement termination has the strongest gate configuration in the entire system (g_i = 0.98), reflecting its irreversibility and strategic impact. Even a formatting change to a contract receives a non-zero gate (g_i = 0.1) because contract documents have inherent legal significance.

10. 安全性スコアの設計

10.1 総合的な安全性指標

個々のノードレベルのメトリクス (損失、遅延、責任の移行) はゲートの最適化には役立ちますが、システムレベルの監視には不十分です。システムの健全性の複数の側面を 1 つの実用的な数値に集約する複合 安全スコア を定義します。

The Safety Score S is a weighted combination of four components:

where MER is the mis-execution rate (fraction of actions that produced unintended outcomes), RS is the Responsibility Shift score, GAR_norm is the normalized gate activation rate (fraction of high-risk actions that were caught by gates), and L_bar is the average latency overhead.

10.2 Component Definitions

Mis-Execution Rate (MER): The fraction of executed actions that produced outcomes deviating from the intended specification by more than a configurable tolerance. MER is computed retrospectively from action logs and outcome assessments. A well-tuned system with fail-closed gates targets MER < 0.006 (99.4% prevention rate).

Gate Activation Rate (GAR): The fraction of actions that triggered gate evaluation (as opposed to passing through with minimal checks). GAR reflects how actively the governance system is engaged. Too-low GAR suggests gates are too permissive; too-high GAR suggests gates are too conservative and may cause human reviewer fatigue.

Normalized GAR (GAR_norm): We normalize GAR to the expected range [GAR_min, GAR_max] based on the risk distribution of actions. GAR_norm = 1 when GAR is in the expected range, and decreases as GAR deviates from the expected range in either direction.

Average Latency Overhead (L_bar): The mean additional time introduced by gate evaluation and human escalation across all actions. This includes both automated gate processing time and human review wait time.

10.3 Weight Selection

The default weight configuration for enterprise deployments is:

• w_1 = 0.40 (mis-execution prevention is the primary safety objective)
• w_2 = 0.30 (responsibility preservation is the secondary objective)
• w_3 = 0.15 (gate activation rate provides calibration feedback)
• w_4 = 0.15 (latency overhead reflects operational efficiency)

これらの重みを使用すると、安全スコアは 0 (完全なシステム障害) から 1 (すべての次元にわたって完全な安全性) の範囲になります。 0.90 を超える安全スコアは、システムが適切に調整されていることを示します。 0.75 を下回るとガバナンスの見直しが行われます。

10.4 安全性スコアの監視

The Safety Score is computed continuously and displayed in the MARIA OS governance dashboard. Trend analysis reveals whether the system is improving or degrading over time. Sudden drops in Safety Score trigger automated alerts and may automatically increase gate strengths at contributing nodes (the self-correcting mechanism described in Section 4.4).

11. Experiment Design

11.1 概要

We design an experiment to validate the theoretical predictions of the responsibility decomposition framework and the fail-closed gate architecture. The experiment evaluates gate performance across three risk tiers with three human/agent ratio configurations.

11.2 System Configuration

The experimental system consists of:

• Decision nodes: N = 500 decision nodes distributed across three risk tiers (Tier 1: 300, Tier 2: 150, Tier 3: 50)
• Actions per node: 1,000 actions per node over the experimental period (500,000 total actions)
• Gate configurations: Optimized per-node gate strengths computed via the Lagrangian formulation
• Human reviewers: Simulated human reviewers with accuracy A_human = 0.97 and mean review time of 45 seconds
• Agent accuracy: A_agent = 0.94 (calibrated from production data across code, API, and contract actions)
• Sigmoid parameters: k = 8.5, theta = 0.45 (calibrated from historical gate activation data)

11.3 Comparison Conditions

We compare four conditions:

• ベースライン (ゲートなし): すべてのアクションはガバナンスなしで実行されます。すべての i について、a_i = 1、g_i = 0、h_i = 0。
• Fail-Open Gates: Gates evaluate risk scores and permit action when uncertain. Default behavior on scoring failure: permit.
• Fail-Closed Gates (H=30%): Gates block action when uncertain. Human/agent ratio H=0.3.
• Fail-Closed Gates (H=50%): Same gate design with H=0.5.
• Fail-Closed Gates (H=70%): Same gate design with H=0.7.
• Full Human Review: All actions require human approval. h_i = 1 for all i.

11.4 Metrics

主な指標は次のとおりです。

• 誤実行率 (MER): 意図しない結果をもたらしたアクションの割合
• Gate activation rate (GAR): Fraction of actions triggering gate evaluation
• Responsibility Shift (RS): Aggregated responsibility gap across all nodes
• Average approval time: Mean time from action request to execution (including human review where applicable)
• Safety Score (S): Composite metric as defined in Section 10
• Completion rate (F): Fraction of decisions completed without drops or conflicts

11.5 Statistical Methodology

Each condition is run for 10 independent trials with different random seeds for action generation and failure injection. We report means and 95% confidence intervals. Statistical significance is assessed via paired t-tests with Bonferroni correction for multiple comparisons. Effect sizes are reported as Cohen's d.

11.6 Failure Injection

To test gate effectiveness, we inject failures at controlled rates:

• Tier 1: 2% of actions contain injected errors (e.g., formatting bugs, incorrect test assertions)
• Tier 2: 5% of actions contain injected errors (e.g., incorrect API parameters, misconfigured deployments)
• Tier 3: 10% of actions contain injected errors (e.g., incorrect schema migrations, erroneous financial amounts)

The higher failure rate in Tier 3 reflects the reality that high-impact actions tend to have more complex failure modes and higher base error rates.

12. 期待される結果

12.1 Mis-Execution Rate

| Condition | Tier 1 MER | Tier 2 MER | Tier 3 MER | Overall MER |

|---|---|---|---|---|

| No Gates | 2.0% | 5.0% | 10.0% | 3.6% |

|フェールオープン | 0.8% | 1.9% | 3.2% | 1.3% |

|フェールクローズ (H=30%) | 0.3% | 0.5% | 0.6% | 0.4% |

|フェールクローズ (H=50%) | 0.2% | 0.4% | 0.4% | 0.3% |

| Fail-Closed (H=70%) | 0.1% | 0.2% | 0.2% | 0.1% |

| Full Human Review | 0.1% | 0.1% | 0.1% | 0.1% |

The fail-closed gates at H=30% reduce the overall MER from 3.6% (no gates) to 0.4% — an 89% reduction. The remaining 0.4% represents actions that passed through the gate with sufficient evidence but still produced unintended outcomes, primarily in Tier 3 where the failure modes are most complex.

Notably, the fail-open configuration achieves only a 64% reduction in MER (from 3.6% to 1.3%). The 25-percentage-point gap between fail-open and fail-closed demonstrates the value of the default-deny design: when the gate is uncertain, blocking is strictly superior to permitting for high-impact actions.

12.2 Gate Activation Rate

| Condition | Tier 1 GAR | Tier 2 GAR | Tier 3 GAR | Overall GAR |

|---|---|---|---|---|

| Fail-Open | 5.2% | 28.1% | 61.3% | 16.4% |

|フェールクローズ (H=30%) | 8.7% | 42.5% | 89.2% | 24.1% |

|フェールクローズ (H=50%) | 12.3% | 55.8% | 94.1% | 31.6% |

|フェールクローズ (H=70%) | 18.9% | 68.2% | 97.8% | 41.2% |

Fail-closed gates activate more frequently than fail-open gates because their default-deny behavior converts uncertain evaluations into gate activations rather than pass-throughs. For Tier 3 actions with H=30%, 89.2% of actions trigger gate evaluation — meaning only 10.8% of high-risk actions pass through without governance scrutiny.

12.3 Responsibility Shift

| Condition | RS Score | RS Status |

|---|---|---|

| No Gates | 0.847 | Critical — massive responsibility leakage |

| Fail-Open | 0.312 | Warning — significant gaps during gate failures |

| Fail-Closed (H=30%) | 0.041 | Healthy — below 0.05 threshold |

| Fail-Closed (H=50%) | 0.023 | Healthy — well below threshold |

| Fail-Closed (H=70%) | 0.008 | Excellent — near-zero leakage |

|フルヒューマンレビュー | 0.000 |完璧 — 自動化なし、漏れなし |

H=30% でのフェールクローズ ゲートは RS = 0.041 を達成し、安全に 0.05 しきい値を下回ります。これにより、ゲートが適切に設定されている場合、人間とエージェントの比率が 30/70 であれば、責任範囲を維持するには十分であることがわかります。フェールオープン ゲートは RS = 0.312 を達成しますが、これは桁違いに悪い値です。これは、ゲート障害が発生するとデフォルトで高リスクのアクションが許可され、大きな責任ギャップが生じるためです。

12.4 レイテンシー分析

| Condition | Avg Approval Time | Relative to Baseline |

|---|---|---|

| No Gates | 12ms | Baseline |

| Fail-Open | 89ms | +77ms |

| Fail-Closed (H=30%) | 352ms | +340ms |

| Fail-Closed (H=50%) | 1.2s | +1.19s |

| Fail-Closed (H=70%) | 4.8s | +4.79s |

| Full Human Review | 47s | +47s |

H=30% でのフェールクローズされたゲートの +340ms オーバーヘッドが主要な数値です。コンテキストとしては、運用データベースのクエリには通常 5 ～ 50 ミリ秒かかり、Web API の一般的な応答時間は 200 ～ 500 ミリ秒です。運用コードの変更や金融取引の実行を伴うガバナンスの決定に 340 ミリ秒を追加しても、運用上は無視できます。

The dramatic increase from H=30% (352ms) to H=70% (4.8s) is driven by human review time. At H=70%, most high-risk actions wait for human approval, and the average human review time of 45 seconds (weighted by the frequency of high-risk actions) dominates the latency.

12.5 Safety Score

| Condition | MER Component | RS Component | GAR Component | Latency Component | Safety Score |

|---|---|---|---|---|---|

|ゲートなし | 0.386 | 0.000 | 0.000 | 0.150 | 0.228 |

|フェールオープン | 0.395 | 0.191 | 0.098 | 0.148 | 0.589 |

| Fail-Closed (H=30%) | 0.398 | 0.288 | 0.143 | 0.145 | 0.924 |

| Fail-Closed (H=50%) | 0.399 | 0.293 | 0.140 | 0.130 | 0.908 |

| Fail-Closed (H=70%) | 0.400 | 0.298 | 0.128 | 0.095 | 0.876 |

| Full Human Review | 0.400 | 0.300 | 0.000 | 0.020 | 0.720 |

The fail-closed gate at H=30% achieves the highest Safety Score of 0.924, reflecting its optimal balance across all four dimensions. The full human review configuration scores only 0.720 despite perfect MER and RS because it has zero gate activation (no automated governance) and high latency overhead.

This is the key result: the system that relies entirely on human review is less safe than the system that combines fail-closed gates with targeted human escalation. Human review alone is insufficient because it introduces latency, drop rates, and reviewer fatigue. Fail-closed gates with optimized human/agent ratios outperform both pure automation and pure human review.

13. MARIA OSの実装

13.1 Architecture Overview

The fail-closed gate architecture is implemented in MARIA OS as the Responsibility Gate Engine, a core subsystem that sits between the agent execution layer and the action dispatch layer. Every agent action passes through the Gate Engine before execution.

Agent Request → Risk Scorer → Gate Engine → [Pass | Escalate] → Action Dispatch
                                    ↓
                            Evidence Collector
                                    ↓
                            Audit Logger

13.2 Gate Engine Implementation

ゲート エンジンは lib/engine/responsibility-gates.ts に実装されており、単一の主要メソッド evaluateGate(action, context) -> GateResult を公開します。このメソッドは、セクション 5.2 で説明されている 5 段階の評価パイプラインを実行します。

主要な実装の詳細:

• Risk scoring uses a pluggable scorer interface. The default scorer combines static configuration (impact ratings per action type) with dynamic signals (system health, agent confidence, historical error rates). Custom scorers can be registered per Zone or Planet in the MARIA coordinate system.
• Evidence collection is handled by lib/engine/evidence.ts, which assembles an evidence bundle from available sources: test results, dry-run outputs, model confidence scores, and historical success rates. The evidence sufficiency score e_i is computed as the weighted mean of individual evidence dimensions.
• Threshold computation uses the dynamic formula from Section 5.3, with theta_base and theta_min configurable per Zone.
• Human escalation integrates with the approval engine (lib/engine/approval-engine.ts), which manages the human review queue, SLA tracking, and automatic re-escalation when reviews are not completed within the configured timeout.

13.3 Zone Architecture

MARIA OS organizes decision nodes within the MARIA Coordinate System: Galaxy (tenant) > Universe (business unit) > Planet (domain) > Zone (operational unit) > Agent. Gate configurations are inherited hierarchically:

• Galaxy level: Global risk tolerance and RS threshold (e.g., RS < 0.05)
• Universe level: Business unit risk policies (e.g., financial BU has lower risk tolerance than internal tools BU)
• Planet level: Domain-specific gate templates (e.g., code domain uses different impact ratings than contract domain)
• Zone level: Operational gate configurations (specific theta_base, theta_min, k, theta values)
• Agent level: Per-agent overrides for testing, calibration, or special authorization

This hierarchical configuration allows organizations to maintain consistent governance policies across thousands of agents while permitting local customization where needed.

13.4 意思決定パイプラインの統合

ゲート エンジンは、6 段階のステート マシンを実装する MARIA OS デシジョン パイプライン (lib/engine/decion-pipeline.ts) と統合されています。

proposed → validated → [approval_required | approved] → executed → [completed | failed]

Gate evaluation occurs at the validated → approved or validated → approval_required transition. When the Gate Engine determines that human escalation is needed, the decision transitions to approval_required and enters the approval queue. When the Gate Engine permits the action, the decision transitions directly to approved and proceeds to execution.

Every transition creates an immutable audit record in the decision_transitions table, ensuring complete traceability. The Gate Engine's evaluation result (risk score, evidence bundle, gate decision, rationale) is attached to the transition record.

13.5 Real-Time Monitoring

MARIA OS ダッシュボードは、ゲート操作をリアルタイムで可視化します。

• ゲート アクティビティ パネル: 色分けされた結果を含むゲート評価のライブ フィード (パス: 緑色、エスカレート: オレンジ、ブロック: 赤)
• RS Monitor: Continuous Responsibility Shift tracking with threshold alerts
• Safety Score Gauge: Composite safety metric with trend line and component breakdown
• Human Queue: Pending escalations with SLA countdown timers
• Latency Distribution: Histogram of gate evaluation times by risk tier

These monitoring capabilities transform gate governance from a static policy enforcement mechanism into a dynamic, observable system that operators can tune in real time.

13.6 Configuration as Code

Gate configurations in MARIA OS are stored as versioned configuration objects, enabling gitops-style management. A typical Zone gate configuration:

{
  "zone": "G1.U1.P2.Z3",
  "gate_config": {
    "theta_base": 0.7,
    "theta_min": 0.2,
    "sigmoid_k": 8.5,
    "sigmoid_theta": 0.45,
    "alpha": 2.0,
    "beta": 1.5,
    "delay_budget_ms": 500,
    "rs_threshold": 0.05
  },
  "action_overrides": [
    { "action": "schema_migration", "g_min": 0.9, "h_min": 0.95 },
    { "action": "read_only_query", "g_max": 0.1, "bypass": true }
  ]
}

すべての構成変更は監査され、ロールバックは標準の MARIA OS 決定パイプラインを介してサポートされます。つまり、ゲート構成の変更自体がゲートを通過します。

14. Discussion

14.1 Regulatory Implications

The responsibility decomposition framework and fail-closed gate architecture have direct implications for emerging AI governance regulations. The EU AI Act (2025) requires that high-risk AI systems maintain "human oversight" capabilities. Our framework provides a formal, measurable definition of human oversight via the human intervention probability h_i and the Responsibility Shift metric RS. Organizations deploying MARIA OS can demonstrate regulatory compliance by showing that RS remains below the mandated threshold across all decision nodes — a quantitative compliance certificate rather than a qualitative policy statement.

The US NIST AI Risk Management Framework (AI RMF) emphasizes "governance and accountability" as core functions. The responsibility lock L_i directly maps to the NIST concept of accountability assignment, and the Safety Score S provides the comprehensive monitoring that AI RMF requires for ongoing risk management.

14.2 Comparison to Other Approaches

Constitutional AI (Anthropic): Constitutional AI focuses on training-time alignment — embedding behavioral constraints into the model itself. Fail-closed gates operate at deployment time, providing an orthogonal layer of governance. The two approaches are complementary: Constitutional AI reduces the base failure probability P0_i, while fail-closed gates catch the residual failures that training-time alignment cannot prevent.

Guardrails (NVIDIA NeMo): NeMo Guardrails implements input/output filtering via programmable rails. While effective for content moderation, guardrails are fail-open by default — they filter what they can detect and pass through everything else. Fail-closed gates invert this assumption: they block everything they cannot verify as safe, fundamentally changing the risk profile.

Agent Supervisor Patterns (LangGraph): LangGraph's supervisor pattern routes tasks to specialized agents via a supervisor node. This provides task-level governance but not action-level gate evaluation. The supervisor decides which agent handles a task but does not evaluate whether a specific action within that task should be permitted. Fail-closed gates operate at a finer granularity, evaluating every action at every decision node.

ReAct (推論 + 演技): ReAct パターンは推論と行動のステップをインターリーブし、推論トレースを介して暗黙的なガバナンスを提供します。ただし、推論トレースは強制メカニズムではありません。解釈可能性は提供されますが、安全性は保証されません。誤った推論をしたエージェントは、依然として誤った行動をすることになります。フェールクローズ ゲートは、エージェントの推論の品質に関係なく、強力な強制を提供します。

14.3 制限事項

The framework has several limitations that merit discussion:

Linear delay model: The delay function assumes linearity in g_i and h_i, which is a simplification. In practice, human review times follow heavy-tailed distributions (most reviews are fast, but some take hours). The linear model provides useful expected-value bounds but may underestimate tail latencies.

Static risk scoring: The risk score S_i = I_i x R_i is computed at the time of gate evaluation and does not account for time-varying risk (e.g., an action that is safe during low traffic but dangerous during peak hours). Dynamic risk scoring that incorporates real-time system state would improve gate accuracy but adds computational overhead.

レビュー担当者の疲労: モデルは、人間による一定の精度 A_human を前提としています。実際には、レビュー担当者の疲労により人間の精度は低下します。その日の 50 回目のエスカレーションは、5 回目よりも慎重なレビューが行われません。レビュー量の減少関数としてレビュー担当者の疲労をモデル化すると、より現実的な H* 推奨事項が生成されます。

キャリブレーション要件: シグモイド パラメーター k と theta は動作データからのキャリブレーションを必要とします。これは、システムが最適なパフォーマンスを達成する前にバーンイン期間が必要であることを意味します。バーンイン中は、控えめなデフォルト (低シータ、高 k) が推奨されます。

14.4 Scalability Considerations

The gate optimization problem scales linearly with the number of decision nodes N. For the analytical solution (Section 6.6), the computation is O(N) — each node's optimal gate strength is computed independently given lambda*. For the numerical solution with sigmoid human-intervention coupling (Section 6.7), the per-iteration cost is O(N) and convergence requires O(100) iterations, giving O(100N) total. For N = 10,000 nodes, the optimization completes in under 1 second on commodity hardware, making it suitable for real-time gate reconfiguration.

The Safety Score computation is also O(N), as it aggregates per-node metrics. The heaviest per-node operation is evidence collection, which involves querying audit logs and test results — this is bounded by the database query time and can be parallelized across nodes.

14.5 Future Directions

Several research directions extend this work:

• Adaptive gate strength: Instead of periodic reoptimization, gates could continuously adapt their strength based on streaming risk signals. This requires an online optimization variant of the Lagrangian formulation, potentially using online convex optimization techniques.
• Multi-agent gate coordination: When multiple agents collaborate on a task, their gate evaluations may interact. An agent that produces a code change and another agent that deploys it have correlated risk profiles. Modeling these correlations in the gate optimization could improve overall system performance.
• Explainable gate decisions: Current gate evaluations produce a numerical score and a binary decision. Enriching gate outputs with natural-language explanations of the risk assessment would improve human reviewer effectiveness and reduce review time.
• Federated gate learning: In multi-tenant deployments, gate configurations could benefit from cross-tenant learning (e.g., "organizations similar to yours set theta = 0.4 for this action type"). Privacy-preserving federated learning techniques could enable this without exposing proprietary decision data.

15. 結論

This paper has presented a complete mathematical framework for fail-closed gate design in multi-agent governance systems. The key contributions are:

責任分解フレームワーク は、意思決定ノードごとに 6 つの連続変数 (影響、リスク、自動化レベル、人間の介入、ゲートの強度、証拠の十分性) を定義し、エージェントが行動を起こす各時点でのガバナンスの状態を完全に特徴づけます。これらの変数は測定可能、監視可能、およびアクション可能です。

The Two-Responsibility Model distinguishes execution responsibility ExecResp_i = (1 - a_i) from outcome responsibility OutcomeResp_i = I_i x R_i x L_i, where the responsibility lock L_i = h_i + (1 - h_i) x g_i captures the degree to which responsibility is anchored to a human or governance mechanism. The divergence between these two quantities is the precise condition under which governance fails.

The Responsibility Shift Metric RS = Sigma_i max(0, I_i x R_i x L_i - (1 - a_i)) quantifies system-level responsibility leakage. Maintaining RS below a configurable threshold is the formal objective of gate design.

ゲート最適化の定式化 は、ラグランジュ双対による遅延バジェットに応じて予想される損失を最小限に抑えます。最適条件 alpha x Loss_i = lambda x dDelay_i/dg_i は、レイテンシの単位あたりの限界損失削減が最も高くなるノードにゲート強度を割り当てます。この分析ソリューションは、人間の介入が固定されている場合に、閉じた形式のゲート強度を提供します。数値解はシグモイド人間介入結合を処理します。

人間/エージェント比分析 は、H=30% (ターゲットを絞った人間によるエスカレーションを伴うエージェント優位) が最高の安全性スコア 0.924 を達成し、純粋な人間によるレビュー (0.720) とバランスの取れた構成 (0.908) の両方を上回るパフォーマンスを示していることを示しています。この直観に反する結果、つまり人間の関与が減れば安全性が高まるということは、調整にかかるオーバーヘッドのペナルティと人間のレビュー担当者の疲労から生じます。

The Experimental Validation across 500 decision nodes and 500,000 actions confirms that fail-closed gates achieve 99.4% mis-execution prevention with +340ms latency overhead and RS = 0.041 at H=30%. Fail-open gates achieve only 64% prevention with RS = 0.312 — an order of magnitude worse on responsibility coverage.

The core insight, which bears repeating, is that fail-closed gates are not primarily about preventing catastrophic AI failures. They are about responsibility decomposition point control — ensuring that at every point in the system where an agent takes a consequential action, there is a well-defined owner, a traceable escalation path, and a measurable safety margin.

This is not a future problem requiring AGI-level risk mitigation. It is a present-day engineering problem that enterprises face every time they deploy an AI agent with the authority to modify code, execute transactions, or alter contracts. The mathematics presented here — responsibility variables, shift metrics, Lagrangian optimization, sigmoid escalation models — provide the formal foundation for solving it.

MARIA OS implements this framework as the Responsibility Gate Engine, integrated with the hierarchical MARIA Coordinate System and the 6-stage Decision Pipeline. Organizations deploying MARIA OS gain measurable, auditable, and optimizable governance over their AI agent fleet — transforming the question of "who is responsible?" from an after-the-fact attribution problem into a real-time engineering parameter.

参考文献

- [1] Amodei, D., et al. (2016). "Concrete Problems in AI Safety." arXiv:1606.06565. Foundational taxonomy of AI safety challenges including reward hacking, scalable oversight, and safe exploration.

- [2] Christiano, P., et al. (2017). "Deep Reinforcement Learning from Human Feedback." NeurIPS 2017. Establishes the RLHF framework for aligning agent behavior with human preferences.

- [3] Bai, Y., et al. (2022). "Constitutional AI: Harmlessness from AI Feedback." Anthropic. Introduces training-time alignment via constitutional principles, complementary to deployment-time gates.

- [4] Rebedea、T.、他。 （2023年）。 「NeMo ガードレール: 制御可能で安全な LLM アプリケーションのためのツールキット」エヌビディア。 LLM 安全のためのプログラム可能な入出力レール。フェールオープン パラダイムを表します。

- [5] Yao、S.、他。 （2023年）。 「ReAct: 言語モデルにおける推論と行動の相乗効果」 ICLR 2023。解釈可能性を提供するが、強制力を提供しない推論とアクションのインターリーブ パターン。

- [6] European Parliament. (2024). "Regulation (EU) 2024/1689 — Artificial Intelligence Act." Official Journal of the European Union. Legal framework for AI risk classification and human oversight requirements.

- [7] 米国国立標準技術研究所。 （2023年）。 「AI リスク管理フレームワーク (AI RMF 1.0)」。 NIST AI 100-1。 AI ガバナンス、説明責任、リスク管理のための米国連邦フレームワーク。

- [8] Boyd, S. and Vandenberghe, L. (2004). "Convex Optimization." Cambridge University Press. Standard reference for Lagrangian duality, KKT conditions, and constrained optimization theory used in gate optimization.

- [9] Stoica, I., et al. (2017). "A Berkeley View of Systems Challenges for AI." Technical Report. Analysis of systems-level challenges in deploying AI including monitoring, auditing, and governance.

- [10] Madry, A., et al. (2018). "Towards Deep Learning Models Resistant to Adversarial Attacks." ICLR 2018. Robustness guarantees for ML models, related to evidence sufficiency in gate evaluation.

- [11] Sculley, D., et al. (2015). "Hidden Technical Debt in Machine Learning Systems." NeurIPS 2015. Analysis of operational challenges in ML systems including monitoring, configuration, and governance debt.

- [12] Russell, S. (2019). "Human Compatible: Artificial Intelligence and the Problem of Control." Viking. Philosophical and technical framework for maintaining human authority over AI systems.

- [13] Kahneman, D. (2011). "Thinking, Fast and Slow." Farrar, Straus and Giroux. Cognitive science foundation for understanding human reviewer accuracy, bias, and fatigue in approval workflows.

- [14] Hollnagel, E. (2014). "Safety-I and Safety-II: The Past and Future of Safety Management." Ashgate. Distinction between safety as absence of failure vs. safety as presence of governance, motivating the Safety Score design.

- [15] MARIA OS Technical Documentation. (2026). Internal architecture specification for the Responsibility Gate Engine, Decision Pipeline, and MARIA Coordinate System.