{"id":200,"date":"2024-12-13T07:00:23","date_gmt":"2024-12-13T07:00:23","guid":{"rendered":"https:\/\/freeoseocheck.com\/blog\/uncovering-chatgpt-search-sources\/"},"modified":"2024-12-13T07:00:23","modified_gmt":"2024-12-13T07:00:23","slug":"uncovering-chatgpt-search-sources","status":"publish","type":"post","link":"https:\/\/freeoseocheck.com\/blog\/uncovering-chatgpt-search-sources\/","title":{"rendered":"Uncovering ChatGPT Search Sources"},"content":{"rendered":"

It’s been almost two years since generative AI burst onto the radars of tech and digital professionals worldwide. Since then, the conversation hasn\u2019t stopped, and the developments keep coming.<\/span><\/p>\n

Not even 18 months ago, we found that <\/span>ChatGPT\u2019s local search capabilities<\/span><\/a> were pretty poor. Fast-forward to now, and ChatGPT-4 enables users to search the web in real time, producing local search results similar to Google or Bing.<\/span><\/p>\n

Hot on the heels of our recent <\/span>Business Listings Visibility Study<\/span><\/a>, where we analyzed the types of results Google surfaces for local queries, we got straight to work replicating the study for ChatGPT Search. So, ready to dive in?<\/span><\/p>\n

\n

Notes On ChatGPT and ChatGPT Search<\/strong><\/p>\n

At the end of October 2024, OpenAI (the organization that owns the product ChatGPT) announced ChatGPT Search<\/a>.<\/p>\n

This new feature means that, based on the user’s prompt or question, ChatGPT can automatically search the web in real-time<\/strong> to support the answers it provides. Users can also toggle the functionality when entering a query.<\/p>\n

This means that, compared to previous and free versions of the ChatGPT product, ChatGPT Search can now provide answers using up-to-date information, whereas ChatGPT’s foundational data training is limited to 2021.<\/p>\n

Currently only available to ChatGPT-4 Plus account holders (via a paid subscription), OpenAI has said the feature will be rolled out to all users in the coming months.<\/p>\n

Where does ChatGPT source business information from?<\/strong><\/p>\n

Although described as being built with a ‘variety of search technologies<\/a>‘, we also know that ChatGPT is mostly powered by Bing’s Index.<\/a><\/p>\n

However, ChatGPT does also source information about local businesses using review information from Google and Google Maps.<\/p>\n<\/div>\n

Methodology and Terminology Used<\/h2>\n

In this study, we conducted 800 manual local business web searches in ChatGPT, recording the first ten search sources ChatGPT displays in its local results’ \u2018Sources.\u2019 It\u2019s important to note that while Google displays results in the hundreds, ChatGPT\u2019s \u2018Sources\u2019 can vary significantly\u2014sometimes showing only a handful of sources and other times showing more than ten.<\/span><\/p>\n

The image below highlights the difference between what ChatGPT displays as results (or answers) to the user query, and the sources for its answers.<\/p>\n

\"Chatgpt<\/p>\n

We used the exact search terms used within the <\/span>Business Listings Visibility Study<\/span><\/a> so that we could provide a direct comparison of the results. As we did in that study, we used one ‘transactional’ keyword (e.g. “coffee shop”) and one ‘informational’ keyword (e.g. “best coffee shop”) in searches across 20 different business verticals, using 20 different US cities.<\/span>
\n<\/span><\/p>\n

\n

Considerations When Using ChatGPT Search<\/strong><\/p>\n

It\u2019s important to note that tools like ChatGPT are constantly learning and evolving. The data collection for this study was carried out in November 2024, so the results and analysis are specific to that time.<\/p>\n

This study analyzes the types of search results ChatGPT displays in Sources, but it does not provide commentary on the accuracy of these results or its knowledge of the searcher’s precise location. To mitigate location considerations, we also included the US state for each city.<\/p>\n<\/div>\n

Understanding Definitions Within This Study<\/h3>\n

For every search result source, we recorded the website name and classification of each website\u2019s \u2018type\u2019 using the following criteria:<\/p>\n\n\n\n\n\n\n\n\n
Term<\/th>\nDefinition<\/th>\n<\/tr>\n<\/thead>\n
Directory (D)<\/td>\nA formally recognized business listing site where NAP is present, such as Yelp, Tripadvisor, Facebook, BBB, and Yell.<\/td>\n<\/tr>\n
Forum (F)<\/td>\nA forum or discussion website, such as Reddit or Quora.<\/td>\n<\/tr>\n
Mentions (M)<\/td>\nWebsites and resources where businesses are mentioned, but that might include something other than business address or phone number. Examples include Wikipedia, Time Out, Eater, and Forbes.<\/td>\n<\/tr>\n
Business website (B)<\/td>\nTypically, a local business website related to the search term (e.g., “Chicago Toyota” for a ‘car dealership Chicago’ search term), but also all other websites that do not fit the above criteria.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

<\/p>\n

\"Chatgpt<\/p>\n

Search Results Compared<\/h2>\n

Nearly three-fifths (58%) of all the sources for ChatGPT Search results were business websites. Typically, these were local business websites directly related to the business type or business vertical contained within the search term (e.g. “coffee shop chicago”, “hair salon boston”).<\/span><\/p>\n

Business websites were also the top result type in the Business Listings Visibility Study (47%). As we noted then, you would generally expect to see business websites dominating search engine results based on the interests and intent of users, so it follows that ChatGPT Search might mimic a similar number of results if it understands intent in the same way. <\/span><\/p>\n

\"Final<\/p>\n

Business mentions made up more than a quarter (27%) of ChatGPT Search sources across the local searches we conducted. This is significantly chunkier than the 16% we saw for Google\u2019s search engine results. As we\u2019ll get to later on, Wikipedia surprisingly dominates the business mention category throughout this study.<\/span>
\n<\/span><\/p>\n

Meanwhile, business directories only made up 15% of ChatGPT Search\u2019s sources. It\u2019s very interesting to note the difference in both the presence and breadth of directory results within this study compared to the business listings study. Not only do directory results show up less for ChatGPT searches than Google searches, but some of the most prominent directories, like Yelp and Facebook, do not appear at all.<\/span><\/p>\n

The most prominent directory found as a source throughout this study was Three Best Rated, making up almost a quarter of all directory sources (24%), followed by Expertise (18%). Although it seems odd that some of the best-known business directories are missing from the first ten sources, it serves as a reminder that brands and businesses should take the opportunity to boost visibility in all the relevant places, and not just the most well-known ones.<\/span><\/p>\n

When we look at the breakdown of results for transactional and informational search terms, the pattern stays the same.\u00a0<\/span><\/p>\n

\"Final<\/p>\n

However, informational search terms see more of a level split between business website results (43%), business mentions (35%) and directories (22%), while transactional search term sources are largely skewed towards business websites (72%).\u00a0<\/span><\/p>\n

Healthcare and Wellness<\/h3>\n

Searches for healthcare and wellness businesses predominantly surface business website results. It largely makes sense when you consider the general nature and urgency of a person needing to find a healthcare business. For more urgent care, they’re less likely to want to spend time comparing business information, and will just want to see which businesses are closest to them on a map.<\/span><\/p>\n

\"Final<\/p>\n

There are just four terms in this group where business website results appear for fewer than 50% of the first ten ChatGPT sources: \u2018best chiropractor,\u2019 \u2018best day spa,\u2019 \u2018best dentist,\u2019 and \u2018best gym.\u2019<\/span><\/p>\n

In these cases, we see more directory sources, yet there does not seem to be a clear reason for this\u2014particularly when Three Best Rated appears as a directory source for so many different business searches.\u00a0<\/span><\/p>\n

It does make sense that \u2018best day spa\u2019 and \u2018best gym\u2019 would have a higher percentage of business mentions within ChatGPT Search sources. After all, these are the more wellness-focused businesses, for which potential customers might want to read different types of articles and reviews before making a decision.<\/span><\/p>\n

Hospitality<\/h3>\n

The hospitality searches are dominated by a range of business mention results. These include well-known hospitality publications like Thrillist, Eater, and The Culture Trip, while local blogs and news sites like Secret Seattle and San Diego Magazine appear for specific location terms.<\/span><\/p>\n

\"Final<\/span><\/p>\n

It is surprising that directories are not more prominent for hotel searches. While giants like Tripadvisor, Expedia, and Booking.com did appear in source lists, they are largely overshadowed by business mentions. If you think about the results you would see in Google\u2019s SERPs, hotel results tend to be very transaction-led, and travel directories themselves have become commercial comparison or booking platforms\u2014so it could be that this is intentional on ChatGPT\u2019s part.<\/span><\/p>\n

However, the most common business mention source was Wikipedia, which also seems an odd result for hotel searches. It\u2019s not exactly the glossy, photo-laden review content you might expect from Cond\u00e9 Nast or similar.<\/span><\/p>\n

Services and Trades<\/h3>\n

A typical search engine user might expect to see many directory results in the Services and Trades group, such as Thumbtack, Angi, BBB, and so on. It\u2019s surprising, then, that only two of the search terms generated a significant number of directory results in the source lists (“best electrician” at 62%; “best storage facility” at 35%).<\/span><\/p>\n

It\u2019s also not clear why there is such a difference between the number of directory results appearing for \u2018best electrician\u2019 and \u2018best storage facility\u2019 searches.<\/span><\/p>\n

\"Final<\/span><\/p>\n

As with the healthcare group, sources for Services and Trades searches are mostly in the business website category. Arguably, there is also some degree of urgency attached to the intent behind local services searches, which might explain why the results are mostly business websites.\u00a0<\/span><\/p>\n

Then again, because ChatGPT Search is still so new, we don\u2019t know enough about how well it understands web search intent.<\/span><\/p>\n

\u2018Your Money or Your Life\u2019 (YMYL)<\/h3>\n

The \u2018Your Money or Your Life\u2019 group concerns businesses that can impact clients’ lives, health, and finances. Here, directory results take a stronger share of the sources.\u00a0<\/span><\/p>\n

It makes sense that someone searching for these types of businesses would want to compare professionals and businesses by using directories. Directories provide the opportunity to compare customer reviews and read professional profiles in more of a side-by-side view than simply flicking between business websites.<\/span><\/p>\n

It\u2019s also notable that these percentages tend to increase for informational terms, where a user is more likely to be looking for a comparison.<\/span><\/p>\n

\"Final<\/p>\n

Entertainment<\/h3>\n

The Entertainment group is another interesting one where directories are surprisingly absent, although the findings are similar to those of our business listings study.<\/span><\/p>\n

Both \u2018betting store\u2019 search terms see a very large number of business mentions, which, as we discovered in the Business Listings Visibility Study<\/a>, is likely due to gambling regulations varying so much from state to state. Therefore, local and trade publications are likely to cover the topics.<\/span><\/p>\n

\"Final<\/p>\n

Notable Findings<\/h2>\n

Mentions on Wikipedia<\/h3>\n

As we touched on in the hospitality and entertainment sections above, Wikipedia makes a surprising number of appearances throughout this study. Generally, where local business searches are concerned, Wikipedia is an uncommon result\u2014particularly to appear within the first ten results. This is likely because, as a largely educational platform, it does not match the \u2018informational\u2019 intent behind searches such as \u2018best coffee shop\u2019 or \u2018best hotel.\u2019<\/span><\/p>\n

Wikipedia probably appears as a source in ChatGPT Search because the platform is trained on Wikipedia\u2019s articles and media. Although this suggests that LLMs might still have some learning to do regarding search intent, it does highlight the value of a Wikipedia presence\u2014something many business owners and brands might not have previously considered.<\/span><\/p>\n

The below screenshot highlights an example of a local business with its own Wikipedia entry.<\/p>\n

\"Chicago<\/p>\n

If you’re interested in\u00a0<\/span>learning more about the relationship between Wikipedia entries and search engine sources,\u00a0this guide<\/a> explains how managing brand entities via Wikipedia can inform Google and influence<\/span>\u00a0features like knowledge graphs.<\/span><\/p>\n

No Forum Presence in ChatGPT<\/h3>\n

One area we haven\u2019t touched on yet is forums. We first conducted a SERP study to analyze result types in May 2024, when Reddit\u2019s exclusive partnership with Google dramatically altered the <\/span>types of results<\/span><\/a> being surfaced to users. Since then, the <\/span>Business Listings Visibility Study<\/span><\/a> found that the presence of forum results had decreased significantly, accounting for just 7% of search results displaying for local business searches.<\/span><\/p>\n

In the case of ChatGPT Search, it seems to ignore forums completely for local business searches. Although it seems that ChatGPT <\/span>can<\/span><\/i> access content from Reddit or Quora, based on the response below, there do seem to be limitations preventing ChatGPT from sourcing local business information from it.<\/span><\/p>\n

\"Reddit<\/p>\n

Yet, at the time of writing, it appears that the Google-Reddit exclusivity deal is still in place. And it doesn’t explain why other forums like Quora aren’t present in its source lists, other than that ChatGPT might not deem these results ‘helpful’ enough.<\/p>\n

How to Appear in ChatGPT Search’s Sources<\/h2>\n

ChatGPT Search, in its current form, has only been available since October 2024, so it\u2019s unclear just how mainstream it might become as a search engine. Yet we can\u2019t ignore the fact that over <\/span>300 million users<\/span><\/a> are using the wider ChatGPT platform weekly, with over one billion messages sent a day. So, that\u2019s a hefty number of people that <\/span>could<\/span><\/i> use it for local queries.<\/span><\/p>\n

With that in mind, you want to give your brand the best shot at appearing in ChatGPT Search\u2019s source lists. How?<\/span><\/p>\n

Optimizing for LLMs<\/b><\/p>\n

Crystal Carter recently put a guide together on <\/span>optimizing for large language models<\/span><\/a> (LLMs), so we\u2019d strongly recommend reading this. The document can be quite technical and goes beyond just ChatGPT, but contains actionable insights to consider for ChatGPT, such as:<\/span><\/p>\n