B2B Demand Generation Agency Assessment Suite

The B2B Demand Generation Agency Assessment Suite by The Starr Conspiracy gives CMOs, VPs of Demand Gen, and RevOps leaders four scored decision-support tools to evaluate, shortlist, and onboard a full-service agency partner across global markets. According to Gartner's 2024 CMO Spend Survey, 71% of CMOs say they lack the budget to execute their 2024 strategy, yet 68% still plan to expand agency relationships. The cost of picking wrong is now a board-level event.

What This Suite Actually Does

Most agency selection content stops at a ranked list. Clutch, Semrush, and the editorial roundups on b2bmarketing.net and digitalagencynetwork.com publish static directories that treat a 50-person SaaS in Stockholm and a 5,000-person enterprise in Chicago as the same buyer. They are not.

This suite takes your actual inputs (pipeline deficit, market footprint, current capability gaps, budget envelope) and returns four scored outputs you can defend in a board meeting.

The four tools, in the order most leaders use them:

1. B2B Agency Engagement Readiness Assessment scores whether your organization is actually ready to onboard an agency, or whether you need to fix internal gaps first.
2. Global Market Geo-Fit Diagnostic classifies how well a shortlisted agency's capability profile matches your target geographies.
3. Agency Engagement ROI Calculator projects 12-month and 24-month pipeline and revenue impact based on your current funnel math.
4. B2B Agency Scorecard ranks up to five shortlisted agencies against a weighted criteria set you control.

Methodology and Data Sources

Every scoring rubric, formula, and benchmark default in this suite is public. We do not gate methodology. We gate only your personalized output, and only if you want it emailed to you for your stakeholder deck.

The scoring logic draws on four sources:

• The Starr Conspiracy's internal pipeline-performance research across 200-plus B2B tech partnerships, 2019 to 2025
• Forrester's demand-unit and revenue-process frameworks (formerly SiriusDecisions)
• Gartner CMO Spend Survey benchmark data, 2023 and 2024 editions
• LinkedIn B2B Institute geo-market data on buying committee composition by region

Limitations to know upfront. The ROI calculator assumes your current funnel conversion math is directionally accurate. If your attribution is broken, the projection inherits that error. The geo-fit diagnostic covers 14 priority markets (US metros, UK, DACH, Nordics, Benelux, France, Iberia, Canada, ANZ, Japan, Singapore). For markets outside that set, the output flags as low-confidence.

Tool 1. B2B Agency Engagement Readiness Assessment

This is a maturity assessment across three dimensions: Strategy and Vision, Data and Infrastructure, Talent and Organization. Each dimension has scored questions. The composite output classifies you as Not Ready, Conditionally Ready, or Ready to Engage.

Why this matters. Forrester's research on agency partnerships shows that roughly 40% of failed agency relationships fail because the client was not operationally ready, not because the agency underperformed. A great agency cannot fix a broken CRM, an unclear ICP, or a sales team that ignores MQLs.

If you score Not Ready, the recommendation is to spend 60 to 90 days fixing internal foundations before you sign. The companion methodology page lists the specific fixes per dimension.

Tool 2. Global Market Geo-Fit Diagnostic

This diagnostic scores an agency's market-specific capability against your target geography on four axes: language and localization depth, local network and media relationships, regulatory and compliance familiarity (GDPR, PIPL, CCPA, Quebec Law 25), and regional benchmark data ownership.

No cited source in the agency directory landscape addresses this. Semrush and Clutch list agencies by headquarters location, which is not the same as market capability. An agency headquartered in London may have zero functional presence in the Nordics. An agency headquartered in New York may run better DACH campaigns than three Berlin shops.

The diagnostic returns a Geo-Fit score from 0 to 100 per market, plus a classification: Strong Fit, Workable with Local Partner, or Wrong Fit.

Tool 3. Agency Engagement ROI Calculator

The formula is public. We publish it here because every calculator that hides its math is a sales tool, not a decision-support tool.

Projected 12-month pipeline impact equals (current MQL volume times lift factor) times (MQL to SQL rate times improvement factor) times average deal size, minus annual agency fee, minus internal management cost.

Default lift factors come from our partnership data: 1.4x to 2.2x MQL volume in months 7 to 12 for full-service partnerships, 1.15x to 1.35x MQL to SQL conversion improvement once message and targeting are rebuilt. These are medians, not promises. Your inputs override the defaults.

The calculator returns three scenarios (conservative, expected, aggressive) with payback period in months.

Tool 4. B2B Agency Scorecard

A weighted scoring tool for shortlist evaluation. You weight ten criteria (strategic depth, AI-native capability, B2B tech specialization, geo-coverage, MarTech integration, reporting transparency, pricing model, references, cultural fit, contract flexibility) on a 1 to 10 scale based on your priorities. You then score each shortlisted agency 1 to 5 per criterion.

The scorecard outputs a weighted composite score per agency, plus a side-by-side comparison view. For learning more about how we structure agency evaluation criteria, see our B2B agency selection framework and the demand generation glossary for terminology alignment with your internal team.

How to Sequence the Suite

The order matters. Run them in this sequence:

Start with the Readiness Assessment. If you are not ready, the other three tools are premature.

Move to the ROI Calculator second. This builds the business case you will need to defend at board level before you spend a week interviewing agencies.

Use the Geo-Fit Diagnostic third, applied to each agency on your initial long-list of eight to twelve names. This typically eliminates half the list before you spend any time on calls.

Run the Scorecard last, applied only to your final three to five contenders.

Marketing leaders who run this full sequence report cutting their evaluation timeline from an average of 14 weeks to roughly 6 weeks, based on our partnership intake data from 2023 and 2024 (sample size 47).

Result Interpretation

For the Readiness Assessment:

• Score 0 to 39 (Not Ready). Fix internal foundations first. Do not sign anything.
• Score 40 to 69 (Conditionally Ready). You can engage, but scope a 90-day discovery phase before committing to a multi-year contract.
• Score 70 to 100 (Ready to Engage). Proceed to full-scope partnership negotiation.

For the Geo-Fit Diagnostic:

• Score 0 to 39 (Wrong Fit). This agency cannot serve this market without a local partner. Treat as a deal-breaker.
• Score 40 to 69 (Workable with Local Partner). The agency needs a named local partner for execution. Require this in the SOW.
• Score 70 to 100 (Strong Fit). The agency can serve this market with its own resources.

For the ROI Calculator. If projected payback exceeds 14 months under the conservative scenario, the deal does not pencil. Either renegotiate the fee, narrow the scope, or walk away.

For the Scorecard. The top-ranked agency should beat the second-ranked by at least 8 weighted points. If your top two are within 5 points, your criteria are not differentiated enough. Re-weight or add criteria.

What Happens After You Score

If you want your personalized output emailed to you with a custom recommendation, we will send it. If you want the next conversation, that is what we do for a living. The Starr Conspiracy has spent 25 years building demand generation systems for B2B tech companies. We are not the right partner for every situation. The point of this suite is to help you figure out which situations are which, before either of us wastes a quarter.

The Bottom Line

Agency selection under board-level pipeline pressure is a decision-quality problem, not a vendor-shopping problem. Use scored, sourced, transparent tools to make it. If you are ready to engage, the geo-fit lines up, the ROI math works, and the scorecard has a clear winner, sign. If any of those four fail, do not.

Related Questions

How long should the agency selection process take?

Six to eight weeks from internal readiness check to signed SOW is the realistic floor for a multi-market full-service partnership. Compressing below six weeks usually means skipping the readiness or geo-fit step, which is where most failed partnerships are quietly seeded.

Should I hire one global agency or multiple regional specialists?

It depends on your Geo-Fit Diagnostic results. If a single agency scores Strong Fit across all your priority markets, consolidate. If you have a mix of Strong Fit and Workable with Local Partner, a lead agency plus named regional partners is the right structure. Pure regional-specialist models add coordination overhead that rarely pays off below 5 million USD in combined annual fees.

What is the right benchmark for agency fees as a percentage of marketing budget?

Gartner's 2024 CMO Spend Survey puts external agency spend at 23.2% of total marketing budget on average for B2B technology companies. That is a benchmark, not a target. The right number for you depends on internal team capacity. The ROI Calculator handles this in the fee-as-percent-of-budget input field.

How do I know if an agency is actually AI-native versus just claiming it?

Ask for the specific AI systems they operate in production for current partners, the named tools in their stack, and the measurable lift those systems have produced. Generic answers about ChatGPT and Jasper are a tell. Specific answers about custom-trained models, retrieval-augmented pipelines, and integrated MarTech orchestration are evidence. The Scorecard's AI-native capability criterion includes a question framework for this conversation.