Why “AGI” arguments stall without a concrete target
Most “AGI” arguments stall for the same reason product arguments stall: nobody agrees on what the system must reliably do. One person means “passes a set of exams,” another means “replaces a knowledge worker,” and a third means “can be trusted to run a hospital workflow without supervision.” When the target is fuzzy, every new capability looks like proof and every failure looks like a moved goalpost.
A concrete target forces uncomfortable details: what tasks, in what environments, with what error rate, and who carries responsibility when it’s wrong. Without that, benchmarks become a proxy for usefulness, and governance debates turn into ideology. The practical move is to replace “AGI” with a named job-to-be-done and the minimum reliability needed to deploy it.
Which real-world jobs are we trying to automate or augment?
Picture a team deciding whether to “use AI” for customer support. The real question isn’t whether the model is broadly intelligent; it’s whether it can handle the specific work mix: classify issues, ask clarifying questions, follow policy, draft accurate replies, and escalate edge cases. Those are jobs with measurable outcomes: time-to-resolution, refund leakage, customer churn, and compliance risk. “Automate” and “augment” also imply different targets. Full automation demands low variance, strong guardrails, and clean handoffs across shifts and channels. Augmentation can tolerate more uncertainty if a human stays in the loop and the interface makes review fast.
This is why “replace accountants” is too coarse to be useful. Month-end close, expense auditing, tax prep, and fraud detection each require different access, permissions, and error tolerance, and each breaks in different ways when data is missing or incentives are misaligned. Mapping the actual workflows is slower than running benchmarks, but it’s the only way to decide what capability matters.
Capability isn’t one thing: decompose it into testable parts
In practice, “capable” is a bundle of narrower skills that fail independently. A support agent needs reading comprehension, but also constraint following (“never promise refunds”), tool use (pull an order record), and interactive diagnosis (ask the one question that disambiguates two common issues). An accounting copilot might write a clean memo yet still mis-handle spreadsheet logic, forget to reconcile totals, or invent a citation when the audit trail is thin.
A useful capability map separates at least: domain knowledge, reasoning under uncertainty, instruction fidelity, long-horizon planning, tool reliability, and calibration (knowing when it doesn’t know). Each part is testable with scenarios drawn from the workflow: missing fields, conflicting policies, adversarial requests, or time pressure. The constraint is cost: building and maintaining scenario suites, gold labels, and realistic test environments takes real labor, and it changes as the business and regulations change.
The hidden constraint: trust, reliability, and responsibility
A familiar failure mode is the demo that looks competent, then collapses the first time it meets ambiguity: a customer asks for an exception, a policy conflicts with a past promise, or a tool call returns partial data. That gap isn’t about “intelligence” so much as whether the system is dependable under the messy conditions where work actually happens. Reliability means consistent behavior across shifts, languages, edge cases, and outages—not just a high average score.
Trust is also organizational, not psychological. Teams need audit trails, predictable escalation, and clear boundaries on what the system is allowed to do. Responsibility has to land somewhere: if an AI approves a refund, submits a claim, or flags a transaction, who owns the decision and the remediation when it’s wrong? Raising trust usually costs money and speed—more logging, more evaluations, tighter permissions, slower rollouts—and those trade-offs often determine whether “AGI-like” capability becomes deployable value.
Generalization vs. specialization: what do people actually prefer?
Watch what teams buy when budgets, compliance, and deadlines are real: they rarely ask for a system that can “do anything.” They ask for software that does a narrow job predictably—draft this type of response, extract these fields, reconcile these numbers, route this case—because predictable interfaces, permissions, and failure modes are easier to govern. A broadly capable model can still be the engine underneath, but it gets wrapped in specialization: fixed tools, constrained outputs, and domain-specific checks that make behavior legible to reviewers and auditors.
Generalization matters most at the seams: when the input is messy, the policy changes, or the workflow spans departments. But the preference curve flips quickly once errors have a cost. Specialization usually wins because it lowers variance, reduces review time, and makes responsibility clearer. The practical constraint is maintenance: every wrapper, rule, and scenario suite must be updated as products, regulations, and edge cases evolve.
Measuring progress with scenarios, not slogans
A product lead hears “we’re close to AGI” and still has to decide whether to ship an autopilot for refunds, a coding assistant, or a triage bot. The workable question is: in a defined scenario, does the system complete the job with the right constraints? That means measuring end-to-end behavior: it asks the missing question, uses the approved tool, cites the right policy version, and stops when permissions are unclear. A single accuracy number hides the trade-offs between speed, escalation rate, and the cost of human review.
Scenario-based evaluation also makes progress legible across teams. You can say “handles 92% of common billing cases, but only 40% of cross-border exceptions without manual intervention,” and tie that to staffing and risk. The scenarios must stay fresh as policies change, attackers adapt, and tool APIs drift. Treat the suite like production infrastructure, not a one-time benchmark.
A practical way to define “AGI enough” for your context
Imagine you’re deciding whether to let a model operate as an “agent” inside a real workflow—issuing refunds, submitting claims, changing configs, or drafting text that will be sent unedited. “AGI enough” is the point where, for a bounded job, the system hits a pre-declared operating envelope: task coverage (which case types it can handle), reliability (error rate and variance across edge cases), and supervision (what requires approval, what auto-executes, and how fast a human can review). Define the envelope in terms the organization already uses: allowed actions, permission scopes, required evidence (citations, logs, tool traces), and a maximum acceptable loss per month from mistakes.
Then run a gate: a scenario suite that mirrors production, plus a “break glass” drill for outages, policy updates, and adversarial prompts. If you can’t afford to maintain that suite—or to staff the review and incident response it implies—you don’t have an “AGI” problem; you have an operations budget problem, and the target should shrink until it’s governable.
Conclusion: progress accelerates when the target becomes human-sized
The fastest way to make “AGI” debates useful is to stop treating them like a referendum on intelligence and treat them like a deployment decision. When the target becomes human-sized—this workflow, these tools, this policy surface, this error budget—you can argue about concrete trade-offs instead of vibes: more autonomy versus more review, broader coverage versus lower variance, speed versus auditability.
That shift also clarifies why progress can feel uneven. Model capability may improve, but the binding constraints are often scenario coverage, integration work, evaluation upkeep, and who is on call when it fails. If you can name the job, bound the envelope, and price the responsibility, “AGI enough” stops moving and starts shipping.
