Leader artifacts increasingly assume that moving along the AI learning curve means progressing from shallow prompting toward a balanced portfolio of reusable workflows (from lightly saved prompts to deep automations). Among users who already own at least one reusable workflow, which within-user shifts in portfolio composition over time—such as changes in the share of work done via ad‑hoc prompts, lightly parameterized templates, and tightly integrated automations—best predict sustained gains versus later fragility, and how can onboarding explicitly steer users toward those healthy portfolio trajectories rather than simply maximizing the raw count of reusable workflows?

anthropic-learning-curves | Updated at

Answer

Healthy portfolio shifts look like a gradual, diversified mix: ad‑hoc → light templates → selective deep automations, without ever collapsing fully into only one type. Onboarding should optimize for this balance, not max workflow count.

Best‑signal portfolio shifts (within user)

  1. Signals of sustained gains
  • Rising share of work in lightly parameterized templates, while: • Ad‑hoc prompts remain non‑trivial (fallback, exploration).
    • Deep automations cover a few stable, high‑volume tasks only.
  • Prompts/templates reused across tasks with small edits rather than many forks.
  • Stable correction rates or slow improvement, not growing pre‑/post‑manual work.
  1. Signals of later fragility
  • Sharp jump from ad‑hoc to many deep automations with shrinking ad‑hoc and template use.
  • Single dominant automation covering heterogeneous tasks (many input types, exception cases).
  • Rising rollback/bypass of automations and late export to manual tools.
  • Variant sprawl of brittle automations instead of a few parameterized templates.
  1. Healthier target trajectories
  • Early: 60–80% ad‑hoc, 20–40% light templates, ~0% deep.
  • Mid: 30–50% ad‑hoc, 30–50% templates, 10–30% deep for proven, stable tasks.
  • Mature: 20–40% ad‑hoc (edge cases, exploration), 30–50% templates (main layer), 20–40% deep for well‑owned, stable flows.
  • Across stages: maintain both ad‑hoc and template bands; gate deep automation growth on stability and ownership.

Onboarding implications

  1. Make templates the default next step, not deep automations
  • After repeated ad‑hoc use on a task, nudge to “save as template” with 1–3 surfaced parameters.
  • De‑emphasize 1‑click "full automation" until a template shows stable use and low correction volatility.
  1. Gate deep automations by signals of readiness
  • Only promote automation when:
    • Task inputs/formats are stable.
    • A clear owner edits that workflow.
    • Corrections cluster in predictable ways (good fit for parameters).
  • On promotion, auto‑create a shallow fallback (one‑off prompt or template run) and make it easy to switch back.
  1. Protect ongoing ad‑hoc capacity
  • Keep a prominent free‑prompt entry even for heavy workflow users.
  • During outages or heavy correction streaks, suggest “run this as ad‑hoc/template instead of the automation.”
  1. Portfolio‑aware success metrics
  • Track per‑user mix: share of runs by ad‑hoc, template, deep automation.
  • Flag risky shifts: rapid growth of deep share with collapsing ad‑hoc/template shares and rising rollbacks.
  • Reward: growth in template share plus measured ability to handle novel tasks via ad‑hoc.
  1. Targeted nudges by pattern
  • If deep share grows fast and ad‑hoc drops: show a "keep a sandbox" reminder and surface examples of shallow fallbacks.
  • If templates fragment into many near‑duplicates: promote parameterization and consolidation.
  • If heavy manual rework after deep runs: suggest reverting step(s) to template mode or splitting the workflow.

Net: the healthiest learning‑curve paths keep a live mix of ad‑hoc + templates, with deep automation added slowly for well‑understood, stable tasks. Onboarding and metrics should explicitly steer toward that balanced composition instead of celebrating raw automation count.