From AI headlines to defensible decisions.
An AI headline can report an accurate number and still encourage a conclusion the evidence does not support. The number is rarely the problem. The leap from the number to a decision usually is.
Two steps, one honest conclusion
The BBGK Evidence Meter is an editorial method for closing the gap between a number and a decision. It has two steps: identify what kind of claim is being made, then test how far the evidence behind it can carry. The output is a verdict on one specific proposition, the scope it covers, and a separate decision boundary.
What this is, and is not. The meter is a decision-support method, not a validated scientific or statistical score.
Identify the claim
Four recurring claim types appear across AI-impact reporting. They are common types, not a complete map of all evidence. The research design behind a claim determines how confidently it can be interpreted.
Observation
A measured outcome, trend or relationship in defined data and a defined period. It can be descriptive or causal, depending on the design.
Does not establishCausation or generality, unless the design supports them.
Exposure estimate
Tasks or occupations estimated to be susceptible to AI capability.
Does not establishActual adoption, replacement, unemployment or timing. Exposure is not displacement.
Forecast or scenario
A conditional projection built on stated assumptions, a model and a horizon.
Does not establishA settled fact or a guaranteed date.
Expert judgment
A person’s or institution’s interpretation, concern or expectation.
Does not establishAn observed outcome. Judgment is not proof, though it may signal what that person or institution will do.
Name the design too. Separately from the claim type, note the research design where you can identify it: descriptive, correlational, comparative or quasi-experimental, randomized experiment, simulation or model, or systematic synthesis. The claim type says what is being asserted. The design says how far the method can support it.
The most common error is reading one type as another: an exposure estimate read as a job-loss count, or an executive’s expectation read as a forecast.
Run five evidence checks
Each unanswered or weakly supported check may limit the conclusion. How much it matters depends on the claim and the study design.
Source and interests
Who produced the data, and can the original be inspected? Are the methods disclosed? What incentives, funding or commercial interests could shape the interpretation?
Measurement and uncertainty
What are the numerator, denominator, baseline and time window? What exactly is being measured? Are sample size, exclusions, effect size and uncertainty reported?
Scope and transferability
Who or what was measured: which firms, roles, countries and periods? How comparable is that context to your decision?
A single case can document an occurrence or suggest a possible pattern. It cannot, by itself, establish how widespread that pattern is.
Attribution or model logic
For a causal claim: does the design separate the effect attributed to AI from plausible competing explanations, using methods and assumptions suited to the question? For a forecast: are the assumptions, horizon, calibration and sensitivity explicit?
Corroboration and disagreement
Has independent evidence tested the same proposition? Corroboration strengthens confidence only when studies address the same question and do not simply reuse the same data. Are credible counterfindings explained?
Ten articles about one report are still one source.
A missing answer is not automatically a failed study. Record it as a limitation or “not reported”, then state how it narrows the conclusion. Not every design needs every check in the same way.
Verdict, scope and decision
The two steps produce a verdict on one specific proposition, never on a whole report. A single report often holds several claims that earn different verdicts.
- Well-supportedRobust, appropriately scoped evidence supports the proposition, with material limits disclosed.
- Provisionally supportedCredible evidence points toward it, but important uncertainties remain.
- Limited evidenceEvidence is early, narrow or too constrained by method for a broad conclusion.
- Insufficient evidenceThe available material does not establish this proposition.
- Contested or mixedCredible sources or methods conflict in ways that change the interpretation. This describes a conflict, not a point between weak and strong.
BBGK does not turn verdicts into numeric scores. A verdict never stands alone. Each is published with five things:
- iThe exact proposition
- iiThe verdict
- iiiThe population and period it covers
- ivIts major limitations
- vThe evidence that would change it
The decision boundary
Evidence confidence and action are separate judgments. Map a verdict to a decision only after weighing potential harm, cost, reversibility, the cost of delay, and the alternatives.
- ActThe decision has adequate support for its specific stakes and consequences.
- PilotThe evidence is promising, and a reversible test can resolve the decisive uncertainty.
- MonitorThere is a credible signal, but no broad change is justified yet.
- HoldThe premise or the proposed action lacks adequate support, or the risk is out of proportion.
A cheap, reversible safeguard can justify acting on limited evidence. Strong evidence can still call for holding when the proposed intervention goes far beyond the finding.
Anthropic’s junior hiring share
The junior share of technical hires at Anthropic fell from 27% to 9%.
Source. Nathan Wilmers, Regina Catipon, Maxim Massenkoff and Peter McCrory, How employment is changing at Anthropic (opens in a new tab), The Anthropic Institute, 16 October 2026. The report draws on internal employment and hiring records, AI-use data, output metrics, employee surveys and 17 interviews. It is a company case study, not a representative sample of employers.
Junior share of new technical hires. The report text does not state the endpoint month.
Junior share of technical staff, since early 2025.
The report also says technical hiring “has continued growing in absolute terms”, and that around a third of machine-learning managers and a fifth of software-engineering managers reported hiring less due to AI.
Step 1: Claim type
Observation, descriptive design. The company reports its own records for a period that has already passed.
Step 2: The five checks
- SourceAnthropic reporting on itself, and it sells the AI linked to the change. Methods are described, but the underlying records are not published for outside audit.
- Measurement27% to 9% is a fall of 18 percentage points in a share of hires. It is not an 18% reduction and not a count. The report gives no junior-hire counts.
- ScopeOne unusually AI-intensive company. The authors describe explosive business growth and say many of its practices cannot be replicated across the economy.
- AttributionManager reports and the timing of adoption make an AI role plausible. Rapid scaling and a changing business mix also move hiring shares. The hiring series does not isolate AI’s contribution.
- CorroborationPress coverage repeats the same company report. That is distribution, not independent verification.
Why the denominator matters
A hypothetical firm. Arithmetic only, not Anthropic’s actual counts.
Verdicts
Anthropic’s junior share of technical hires fell from 27% to 9%.
Provisionally supportedA company-reported, descriptive finding for Anthropic’s technical hiring. The records are not independently audited.
AI was one cause of that shift.
Limited evidencePlausible, backed by manager accounts and adoption timing. AI’s contribution to this specific decline is not isolated.
AI is ending junior tech jobs across the economy.
Insufficient evidenceThe report covers one rapidly expanding, AI-intensive firm.
What would change the assessment: junior hiring shares from comparable firms, absolute junior-hire counts, or a design that compares AI-intensive teams with similar teams that adopted AI less.
Track the junior share of your own hiring, absolute entry-level hires, role mix and internal career ladders. Investigate local causes before changing hiring budgets or entry-level pathways, since those changes are hard to reverse.
The one-page worksheet
Use this in a leadership review before an AI statistic becomes a reason to act. Short entries are fine. Write “not reported” where the source is silent.
| ClaimExact proposition, unit, number and date | |
|---|---|
| Primary sourceURL, author, date, version | |
| Claim type and research design | |
| Source incentivesAccess to methods or data | |
| MeasurementMetric, numerator and denominator, baseline, sample, time window | |
| UncertaintyError ranges, missing data, limitations | |
| Context and transferabilityHow like our decision is it? | |
| Attribution or model logicAlternative explanations, assumptions, horizon | |
| CorroborationIndependent support, counterevidence, shared-data risk | |
| Verdict and scopeThe narrowest defensible conclusion | |
| DecisionAct, Pilot, Monitor or Hold, with stakes and reversibility | |
| Action triggerThe evidence or milestone that would change the decision |
The last two rows matter most. If the team cannot fill them in, the claim is context, not a reason.
What the meter does not do
The meter grades evidence for a proposition, not truth. A claim with limited evidence can turn out right, and a well-supported one can be overtaken by new data. It is a discipline for deciding how far to act, not a verdict on the people who made the claim.
It does not replace expertise. Reading a labour study’s identification strategy or a model’s assumptions still takes domain knowledge. The meter makes sure the right questions are asked before a number reaches a slide.
Where it sits among BBGK frameworks
The Evidence Meter works upstream. It asks whether a claim about the world is supported. Other BBGK frameworks take over from there.
Editorial standard and sources
BBGK is adopting the Evidence Meter as an editorial standard for new AI-impact analyses, with transparent judgments, source links and corrections when warranted. It applies equally to optimistic productivity findings, risk findings and vendor claims. Where it matters, BBGK labels observed facts, the researchers’ interpretation, BBGK inference and BBGK recommendation separately.
If you think a verdict here is wrong or too generous, contact BBGK. Corrections are made on this page and recorded in the version log.
Disclosure. This article was drafted with AI assistance and edited by the author. The drafting tool, Claude, is made by Anthropic, which also published the report used in the worked example. The relationship is disclosed here. It is not treated as evidence of bias in either direction.
Sources
- PrimaryNathan Wilmers, Regina Catipon, Maxim Massenkoff and Peter McCrory, How employment is changing at Anthropic (opens in a new tab), The Anthropic Institute, 16 October 2026.
- Coverage onlyAnthony Ha, Anthropic says its workforce has become less junior and is spending more time in meetings (opens in a new tab), TechCrunch, 17 October 2026.
