B2B Agency Vetting Frameworks

B2B Marketing Agency Vetting Frameworks for Pipeline and Revenue Proof

The B2B Agency Vetting Framework Catalog is a structured methodology for de-risking marketing agency selection in complex, industry-specific buying cycles where generic credentials are worthless and only CRM-backed, deal-size-relevant proof counts. It organizes the entire due diligence process into six named frameworks across three purposes: Diagnostic (does this agency fit my industry and motion?), Evaluation (can they prove what they claim?), and Verification (will the proof hold up under scrutiny?). You don't get fired for a bad agency deck. You get fired for unprovable pipeline.

Most agency vetting fails the same way. A marketing leader collects case studies, runs reference calls, scores chemistry, and signs. Six months later, pipeline hasn't moved, attribution is murky, and the agency blames sales enablement. The problem was never the conversation. It was the absence of a structured proof standard going in. If your agency can't prove pipeline in your CRM, you're not hiring a partner. You're buying a story.

This catalog prevents three failures: wrong vertical fit, unprovable attribution, and contracts that reward activity instead of revenue. The sequencing matters: diagnosis prevents false positives, evaluation scores the claims, verification stops the cherry-picking. Yes, chemistry matters. No, chemistry doesn't create pipeline. Hiring an agency without CRM proof is like buying a car based on a photo of the speedometer.

We've watched this movie for 25 years. The ending is always the same when proof is optional. Most agency-selection advice, including widely cited guidance from BCG, MarTech.org, and MarketVeep, presents single-lens advice. None of it sequences the diagnosis-evaluation-verification arc. None of it specifies recommended proof bars that must be calibrated to your economics. None of it differentiates vetting for SaaS, fintech, or manufacturing, where sales cycles, compliance burden, and deal sizes diverge dramatically. The Starr Conspiracy built this catalog to force agencies to show the system behind the results, not just the results. Learn more about how this connects to broader B2B marketing operations and the Ten Demand States framework that anchors several of the diagnostics below.

A note on thresholds throughout: treat them as minimum bars we recommend, not universal truths. Calibrate CAC payback, ACV bands, committee size, and cycle length to your model and vertical. When CRM data is messy, require source definitions, sample reports, and signed third-party validation. When references must be anonymous, demand a signed reference call and redacted screenshots, not just a logo grid.

The Six Frameworks at a Glance

Diagnostic (fit before evaluation):

1. Industry-Fit Diagnostic. Does the agency know your vertical's economics and committee?
2. Demand-State Coverage Audit. Can they operate across the states your buyers actually move through?

Evaluation (score the proof):

1. Case Study Scoring Rubric. Rate every case study on eight dimensions, not vibes.
2. CRM-Backed Pipeline Proof Standard. The "CRM-or-it-didn't-happen" bar for revenue claims.

Verification (does the proof survive contact with reality):

1. Retention and Attribution Verification. What reference calls reveal when you structure them.
2. Revenue Accountability Contract Review. Do they sign for what they pitched?

Read the summaries, pick the framework that matches your risk, then run the components like a checklist.

The Three Categories

Diagnostic frameworks answer a fit question before you evaluate anything else. If an agency fails diagnosis, do not advance them. Period.

Evaluation frameworks score the proof an agency presents. This is where marketing theater dies and revenue accountability starts. If they lead with impressions, you're funding content, not revenue.

Verification frameworks test whether the proof survives contact with reality. Reference calls without structure are vibes. Vibes don't move revenue.

This order matters: diagnosis prevents false positives, evaluation scores the claims, verification kills the cherry-picked anecdote.

The Six Frameworks

1. Industry-Fit Diagnostic

The Industry-Fit Diagnostic is a screening framework developed by The Starr Conspiracy for filtering agencies before any evaluation begins. It covers five components: vertical sales cycle length match, deal size band match, buying committee composition fluency, regulatory and compliance literacy, and channel mix relevance. Use it when evaluating agencies for verticals with sales cycles exceeding 6 months or buying committees with 5 or more stakeholders.

Check these five fit signals:

• Vertical execution within the past 24 months with documented pipeline outcomes, not logos.
• Reference deals within your ACV (annual contract value) band, recommended within 50% above or below your median.
• Fluency in the 4 to 7 roles on your typical buying committee, named without prompting.
• SOC 2, GDPR, PCI literacy and constraints-aware execution where the vertical demands it.
• Channel mix that matches where your buyers actually consume, not where the agency likes to work.

Vertical calibration: manufacturing buying committees include distributors and channel partners whose influence rarely shows in CRM. Fail any two of five and do not advance them.

When to use: Run this first on every longlist agency in SaaS, fintech, or manufacturing before reviewing a single case study.

2. Demand-State Coverage Audit

The Demand-State Coverage Audit tests whether an agency can operate across the demand states your buyers move through, not just the narrow band where they happen to be strong. It covers four components: demand state range, motion fluency, creative-to-conversion linkage, and vertical pattern recognition. Use it when your gap is unclear or when you suspect an incumbent is over-indexed on top-funnel awareness.

• Demand state range across the full Ten Demand States, with named work in each they claim.
• Motion fluency across inbound, outbound, ABM, and partner, with proof of which they've actually run.
• Creative-to-conversion linkage showing how assets ladder to pipeline mechanics, not just engagement.
• Vertical pattern recognition. Can they describe how demand states behave differently in your industry?

Vertical calibration: SaaS demand states are dominated by category education and competitive displacement; fintech is dominated by trust and compliance signals.

When to use: Run after the Industry-Fit Diagnostic to confirm capability range before requesting case studies.

3. Case Study Scoring Rubric

The Case Study Scoring Rubric scores agency case studies on eight dimensions instead of accepting narrative claims at face value. Use it on every case study an agency presents in a finalist round.

Score these eight dimensions, 0 to 3 each:

• Client industry match. Same vertical, same motion, same buyer economics.
• Deal size relevance within your ACV band, not a small-business win dressed up for enterprise.
• Baseline disclosure of the pre-intervention number, not just the post-intervention claim.
• Intervention specificity. What they actually did, not what they "led."
• Time-to-impact transparency tied to a realistic sales cycle for that ACV.
• Attribution methodology that names the model and its known limits.
• Named CRM or analytics source: HubSpot, Salesforce, or the client's system of record.
• Post-engagement retention data showing the client kept buying.

A case study scoring below 16 of 24 is marketing theater. Below 12, it's fiction. If they can't name the report, they didn't pull the number.

When to use: Apply to every finalist case study before reference calls begin.

4. CRM-Backed Pipeline Proof Standard

The CRM-Backed Pipeline Proof Standard (internally: the "CRM-or-it-didn't-happen" bar) separates revenue-accountable agencies from impression-accountable ones. Use it as the minimum bar for any agency claiming revenue impact.

Six proof components:

• Sourced pipeline in dollars, not MQLs and not "engaged accounts."
• Influenced revenue with the attribution model named, governed, and defensible.
• CAC payback under a recommended bar of 18 months for SaaS, 24 months for fintech and manufacturing, calibrated to your model.
• Win rate lift on agency-sourced opportunities compared to baseline.
• ACV trend on sourced deals over the engagement window.
• NRR contribution for SaaS with NRR reporting in place.

Non-negotiable CRM proof artifacts to demand on screen-share:

• Report name and where it lives in the CRM.
• Date range and opportunity stages included.
• Source and influence definitions written down, not improvised.
• The person who pulled the report, live in the meeting.

A real agency can pull this report in 2 minutes and explain every filter. Watch for attribution gaming: self-reported influence, last-touch cherry-picking, and "engaged account" pseudo-metrics. We've sat through finalist presentations where a vendor claimed $4M in sourced pipeline, then couldn't name the Salesforce report, the attribution model, or the date range. The "proof" was a slide. Make them screen-share. If they can't, the claim does not exist. Deck proof is not system proof.

Vertical calibration: fintech requires compliance-aware reporting; manufacturing requires distributor influence visibility; SaaS requires NRR linkage.

When to use: At finalist presentation and on every incumbent review. If you use one thing from this page, make it this framework.

5. Retention and Attribution Verification

The Retention and Attribution Verification framework is a reference-call structure that tests whether case study claims survive after the agency relationship ends or matures. Use it on every finalist's references, including the references they didn't volunteer.

• Client retention length. Under 18 months is a flag; over 36 months is a signal.
• Scope expansion pattern. Did the client buy more, buy less, or rebid?
• Attribution method audit. Did the client's RevOps team independently validate the reporting?
• Offboarding context. Why did the relationship end, and what did the client do next?

Stop rewarding stories. Require structured answers. Anonymous reference? Require a signed call and redacted screenshots, not a logo.

When to use: On every finalist before LOI, and on every incumbent at renewal.

6. Revenue Accountability Contract Review

The Revenue Accountability Contract Review tests whether an agency will sign for the outcomes they claimed in the pitch. Use it at the LOI or MSA stage, before signature.

• Pipeline-tied compensation component with a defensible measurement floor.
• Sourced-versus-influenced split defined in writing, not assumed.
• QBR cadence with revenue data, not activity slides.
• Exit clause tied to performance thresholds you can actually invoke.
• Joint attribution governance shared with your RevOps team.

Have counsel review the terms. An agency that talks revenue in the pitch and refuses revenue terms in the contract told you everything you need to know.

When to use: At LOI or MSA stage, before signature, for every finalist.

How to Sequence the Catalog

For a full procurement cycle, run the frameworks in order: Industry-Fit Diagnostic to screen the longlist, Demand-State Coverage Audit to confirm capability range, Case Study Scoring Rubric on finalist materials, CRM-Backed Pipeline Proof Standard during finalist presentations, Retention and Attribution Verification on references, and Revenue Accountability Contract Review at commercial close.

For incumbent review, start with the CRM-Backed Pipeline Proof Standard. If the proof isn't there, the rest of the catalog is academic.

A real agency can explain your category narrative and your demand states, then prove pipeline impact in your CRM. If they can't do both, they're not strategic. They're decorative.

Run Framework 4 on your current agency this week. If they can't screen-share the proof, stop debating and start replacing. Every quarter you wait is another quarter you can't explain to the CFO. If you want us to run this vetting with you before your next renewal or agency search kickoff, talk to The Starr Conspiracy. We'll help you force proof before you sign, with CRM-backed standards and a vetting sequence that holds up in the boardroom. No more vibes-based procurement. Make them prove it, or move on.