Study Finds ChatGPT Beats Google In Blind Test For Search Queries

A study by ClickPop, a digital marketing consultancy in the UK, found that two-thirds (65.3%) of people selected the websites provided by ChatGPT as a better match for their query than websites provided by Google.

For this study, ClickPop created a tool to compare people’s preferences for website recommendations from Google and ChatGPT. Users were able to perform a search query, and the tool presented two sets of website recommendations (A and B) without revealing which technology generated each set. Users could then vote for the set of websites that best matched their search.

ClickPop’s Google vs ChatGPT comparison tool is available for people to use for free.

Key findings:

• Across all searches, 65.3% of people preferred websites provided by ChatGPT vs 34.7% of people who preferred the websites provided by Google.
• For queries that were identified as being for a local area such as a city or region, ChatGPT was still more popular than Google, with ChatGPT receiving 61.1% of the votes vs 38.9% for Google.
• The preference for ChatGPT’s results applied for searches on both desktop and mobile devices, with ChatGPT receiving 65.7% and 65.1% of the votes respectively.
• Differences were identified in the types of websites provided by Google and ChatGPT. Notably, Google frequently provided more publisher and product comparison websites, whereas ChatGPT was more likely to provide websites from retailers and product brands.
• ChatGPT was also more likely to recommend websites that could be considered as having a strong brand and backlink profile, whereas Google was more likely to recommend websites from lesser-known brands and businesses with a weaker backlink profile.

The test ran from 29 May 2023 to 19 July 2023. 1,674 searches were carried out by users from 84 different countries, with 726 votes being placed. A complete analysis of the results, including the differences between the websites provided by Google and ChatGPT are available at https://www.clickpop.co.uk/google-vs-chatgpt-test-results/

On the findings of this research, David Richter, founder of ClickPop said “The results of our study show that the threat to Google presented by ChatGPT, and AI more generally, is already very real and not just something that might happen in the future.”

“Whether Google remains dominant as a search engine remains to be seen, however, with people’s apparent preference for the websites recommended by ChatGPT for a search query, I expect Google will make significant changes to how it ranks and displays search results in the future.”