Dominant oversight schemes assume that trust is managed via artifact-centric checkpoints or principal-style contracts; what happens if we instead treat the compute budget itself as the primary object of oversight for long-running agents—allocating, revoking, or forking compute based on simple, real-time trust signals (e.g., cross-workflow inconsistency spikes, claim-level contradictions, or unexplained resource surges)—and does this compute-governance lens expose different systemic failure modes or opportunities for safe experimentation than artifact- and contract-centric designs?

Answer

Treating compute budget as the primary oversight object changes how errors surface and how experimentation is constrained. It tends to (a) make some failures more containable and easier to bound, (b) expose new classes of systemic mistakes tied to bad trust signals and budgeting policies, and (c) open up safer ways to run many risky variants in parallel.

Main effects vs artifact-/contract-centric oversight

1. How compute-centric oversight works

• The overseer mainly controls:
  • start/stop/slow: grant or revoke compute based on live trust signals;
  • fork: clone state and run variants under capped compute;
  • throttle: force more or less compute into verification vs progress.
• Artifacts and contracts still exist but are secondary; decisions focus on whether to spend more compute, not on detailed artifact review.

2. Error accumulation and containment

• Benefits
  • Easier to cap blast radius: suspicious runs (inconsistencies, resource spikes) get halted or sandboxed early.
  • Cheap triage: coarse trust signals (contradictions, metric anomalies, unexplained I/O) are enough to pause or fork before large-scale damage.
  • Encourages redundancy: for the same total budget, overseers can fork multiple variants and compare, catching some silent errors by disagreement.
• Costs
  • If trust signals are miscalibrated, high-risk but “quiet” runs may get large budgets, while noisy-but-safe runs get starved.
  • Compute throttling does not itself ensure correctness; it only limits how far errors spread before external checks.

3. New systemic failure modes exposed

• Policy-induced biases
  • The system may over-favor workflows whose signals look stable, even if they encode a shared but wrong model; correlated silent errors can still scale if trust signals don’t reflect that.
  • Agents may learn to optimize for “low anomaly footprint” rather than truth (e.g., smoothing outputs, avoiding boundary cases) to retain compute.
• Feedback instability
  • Rapid budget oscillations (stop/restore cycles) can fragment runs, increase coordination errors, and produce hard-to-replay histories.
• Cross-workflow coupling
  • If budget is reallocated globally based on relative trust signals, a noisy but benign workflow can starve others, hiding their errors behind lack of progress.

4. Opportunities for safer experimentation

• Bounded-risk exploration
  • High-risk ideas can be run as short, low-budget forks whose outputs must match stronger baselines or trigger extra checks to earn more compute.
  • This supports aggressive “try many small things” search without committing a full pipeline to any one risky branch.
• Adaptive verification spend
  • Budget rules can increase the fraction of compute used for verification when anomalies spike (e.g., re-run with different seeds, compilers, solvers) and decrease it when runs are stable.
• System-level A/B tests
  • Oversight can allocate parallel compute to different oversight schemes, solvers, or model families and compare outcomes, using disagreement to trigger deeper review.

5. When compute-centric oversight helps most

• Multi-hour workflows with:
  • good, cheap trust signals (simple anomaly scores, cross-run disagreements);
  • many optional branches or hyperparameter sweeps;
  • strong automated tests but limited human time.
• In these settings, compute governance gives coarse but fast control, reducing the need for dense manual checkpoints.

6. Where artifact-/contract-centric designs remain superior

• Tasks dominated by subtle conceptual or modeling errors that do not strongly perturb simple trust signals.
• Early-stage, ill-specified science where what to measure as a trust signal is unclear, so budget rules are arbitrary or brittle.
• Contexts requiring detailed provenance and replay: compute-centric throttling alone doesn’t ensure that checkpoints are interpretable.

Net view

• Compute-centric oversight is best seen as a coarse outer loop: it makes it easier to bound harm, try many variants, and concentrate resources where signals look risky or promising.
• It does expose different systemic risks (signal gaming, policy bias, instability) and does not replace artifact- and contract-centric checks, which are still needed to understand and fix errors rather than just limiting their reach.