Does ChatGPT Search Break Your AI Visibility Strategy?

ChatGPT's product recommendations changed 80.2% when search was enabled, according to a study of 20,000 responses by Jeff Oxford, founder and CEO of Visibility Labs. Only 19.8% of products recommended without search also appeared in recommendations generated with search enabled.

What Happened

Visibility Labs founder Jeff Oxford ran 1,000 product-recommendation prompts through ChatGPT 10 times with search enabled and 10 times with it disabled, generating 20,000 responses. The result: an 80.2% change in recommended products between the two modes. Even products that appeared in 100% of search-disabled responses showed up only 15.8% of the time when search was on. Search mode also narrowed picks to 5.2 products per response, down from 6.2.

The Numbers in Context

The headline figure is the 19.8% overlap between search-on and search-off recommendations. Compare that to the assumption most B2B marketers have been operating under, that getting named in model training data secures durable AI visibility. A 0.4 Pearson correlation between cited-source mentions and recommendation frequency suggests citations now drive a meaningful share of what surfaces in live search mode, even if causation isn't proven.

Why This Matters for HR Tech and FinTech Marketers

If your AI visibility playbook assumed ChatGPT would consistently recommend you once you cracked the training data, that assumption is broken. Search mode is becoming the default user experience, and it pulls from a different pool. For HR Tech buyers comparing HRIS platforms or FinTech leaders evaluating treasury tools, the recommendation set your prospects see is volatile and tied to what gets cited in current web sources. You can be the most-named partner in static recall and still vanish when a buyer toggles search on. That changes where your earned media, analyst coverage, and review-site presence have to land.

The Starr Conspiracy's Take

This study confirms what we've been telling clients: AI visibility is a two-front war. You need brand presence baked into model weights, and you need to be cited in the live sources ChatGPT pulls when search runs. Most B2B marketers are still treating this as one motion. It isn't. Our work on answer engine optimization for B2B brands walks through how to structure content, citations, and third-party mentions so you show up in both modes. The partners winning this in 2026 are the ones treating cited-source visibility as a distinct discipline.

What to Watch Next

Expect ChatGPT search to become the default mode for free and paid users within the next two quarters. Watch for follow-up studies measuring whether Perplexity, Gemini, and Claude show similar overlap gaps. The decision point for your team: are you funding cited-source presence with the same rigor as traditional SEO?

Related Questions

How is AI search visibility different from traditional SEO?

Traditional SEO optimizes for ranked links on a results page. AI visibility optimizes for whether your brand gets named inside a generated answer. The mechanics overlap, structured content and authoritative citations matter for both, but the measurement and the competitive set are different. See our breakdown of how AI changes B2B demand generation.

Should we optimize for ChatGPT with search on or off?

Both, but prioritize search-on. Search mode is becoming the default, and it pulls from live cited sources you can actually influence through PR, review sites, and authoritative third-party content. Search-off recommendations rely on training data you can't update in real time.

What does a 0.4 Pearson correlation actually mean here?

It's a moderate positive relationship. Products mentioned more often in ChatGPT's cited sources tended to get recommended more often, but the correlation isn't strong enough to call cited-source presence the single dominant factor. Treat it as one of several inputs worth investing in.