For AI-assisted simulation planning in physics, where current workflows rely on the AI grad student pattern plus assumption registries, how should we allocate AI effort between (a) generating more diverse candidate simulation campaigns (parameter sweeps, boundary conditions, toy-model variants) and (b) acting as an uncertainty accountant that prunes or reprioritizes runs based on expected epistemic value, and under what concrete conditions (compute budget, model cost, availability of analytic limits) does shifting effort toward (b) actually reduce wasted runs without increasing the risk of collectively missing rare but high-payoff regimes?

Answer

Split AI effort by regime:

1. Default allocation

• Low/medium cost models, loose compute: bias to (a) diversity.
  • Use AI to propose wide parameter sweeps, BC variants, toy models.
  • Minimal (b): only remove obviously redundant or numerically pathological runs.
• High-cost models or tight budgets: shift toward (b) accountant.
  • AI scores runs by expected epistemic value (EEV) given current posteriors, analytic limits, and assumption registry.
  • Keep a fixed exploration quota (e.g., ≥10–20% runs in low-EEV but underexplored regions) to avoid missing rare regimes.

2. When to favor (a) generation

• Conditions:
  • Per-run cost (time or money) low.
  • Weak analytic limits or benchmarks; little prior knowledge.
  • Model cheap enough to rerun if you later notice gaps.
• Policy:
  • Let AI enumerate broad design spaces and scenario families.
  • Use assumption registry only to tag regimes, not to prune them.
  • Simple guardrail: insist on some coverage of extremes for each key dimension (min/max, a few log-spaced points).

3. When to favor (b) uncertainty accounting

• Conditions:
  • Per-run cost high (large HPC jobs, long wall-clock, expensive data).
  • Strong analytic limits / invariants / toy models.
  • Assumption registry populated with past failures and safe regimes.
• Policy:
  • AI computes EEV per candidate using:
    • Expected shrinkage in parameter/posterior volume.
    • Expected chance of breaking a key assumption or invariant.
    • Novel use of assumptions relative to registry.
  • Schedule: top-EEV runs + reserved exploration band.

4. Safeguards against missing rare high-payoff regimes

• Hard exploration floors:
  • Reserve a fixed budget fraction for runs chosen by diversity metrics, not EEV.
  • E.g., enforce coverage of tails in each dimension and a few random points per campaign.
• Registry-aware diversity:
  • Favor candidates that exercise under-tested assumptions or regimes with few prior runs, even if local EEV is modest.
• Periodic re-scoring:
  • After each batch, re-estimate EEV and update the exploration/exploitation mix.

5. Concrete trigger conditions for shifting toward (b)

• Compute/budget:
  • If expected cost of naive full sweep > ~3× your acceptable budget, move to accountant-first.
• Model cost:
  • If a single run is ≥ hours on many cores or ties up scarce hardware, use (b) heavily.
• Analytic structure:
  • If you can list several reliable limits / invariants and have relevant toy models, (b) gains power.
  • If you cannot, keep (b) light and rely more on (a) plus coarse coverage rules.

6. Practical split (rough guide)

• Cheap models, weak structure: ~80% (a), 20% light (b).
• Mixed cost, some structure: ~50% (a), 50% (b) with explicit exploration floor.
• Very expensive, strong structure: ~20–30% (a) (design + diversity), 70–80% (b) (EEV + registry).

7. Interaction with assumption registries

• Use registry to:
  • Down-rank runs that live in regimes already known to be uninformative or numerically unreliable.
  • Up-rank runs that test assumptions with prior failures in similar systems.
• Avoid using registry as a hard gate on unexplored corners; treat it as a weighting signal combined with explicit diversity quotas.