Conversational AI-powered search engines are not a new concept
You.com, Andi AI and Perplexity AI are just a handful of companies, alongside the larger players including Bing and Google, that have been mooted around the SEO industry as those capable of changing the way information is found and consumed online. Granted, their existence (the lesser known at least) has remained very much in the periphery of those operating, or with an avid interest in, the SEO and search industry. For others, it may just have been yet another industry where AI is ‘set to make its mark’ and changing things ‘for the better’.
It wasn’t until rumours that OpenAI (powered by Microsoft Bing) was developing a Web Search product with the sole aim of taking on Google that the people really sat up and listened. The giant of search was now being challenged by the legend of AI — if there were two (or arguably three if you count Microsoft too) companies who could make AI web search a possibility — and more than a niche offering — surely it is them? Potentially, but there is much more work to be done if AI web search engines are to become a standalone reality.
It’s not rocket science — it’s all down to the algorithms and content, with a bit of speed thrown in too. After all, if users and the industry are going to adopt AI web search as standard, they need to know that the information provided is better — in terms of the relevance and newness of the content itself, as well as the accuracy and speed of delivery than what they already are being served with or what’s the point? This is not just about Google placing itself at the #1 position for ‘AI search engines’, and presumably intentionally failing to feature Bing on page 1 (which has had ChatGPT integrations for over six months).
It would be fair to say that AI works well with existing search engine technology
In fact, a recent study suggests that generative AI can present even more opportunities to increase search visibility and traffic. This is why Google has been using generative AI within its search engine since May last year. Its Search Generative Experience (SGE) is designed to improve the user experience by answering all questions through a single user interface. SGE essentially uses the traditional index and prompts on a search result to say ‘summarise the top 10 results’ rather than using Large Language Modules (LLMs) to add and remove from its index directly.
AI has already been shown to provide enhanced user features, particularly within ecommerce. Traditional AI experiences have focused on informational queries but the step into Google looks to shake up retail, tourism, recruitment and other industries more directly.
However, when it comes to a full-scale product, what’s important to note is that the LLMs currently used in AI search are based on a large, trained data set which has to re-evaluate every time something new is added to it. This is in contrast with traditional search which is updated in near real-time with billions of pages constantly being added to its index.
LLMs simply can’t re-train at the speed of traditional search algorithms, so they’re less good at data retrieval and recency. This means that traditional optimisation tactics are still important, fundamentally Google wants to ensure it’s able to properly crawl through, render and save webpages and the people who ensure that are SEOs. If they kill organic traffic to websites, they de-incentivise brands investing time to make sure Google gets an accurate picture of the web while ultimately killing its data store.
And then we have the content sources. In preparation for the forthcoming bout with OpenAI, Google is muscling up by doubling down on User Generated Content, which is the area AI chatbots don’t currently factor in when searching. This has been in the works for a while since the launch of the Experience factor in it is E-E-A-T guidelines (Experience, Expertise, Authoritativeness, Trustworthiness), but has accelerated since Q3 2023.
Added to this Google has also started to promote Reddit and user-generated content in general through search more than historically, this has come off the back of the last six months of algorithm changes. It has also cut a deal with Reddit to harvest more data. This may very well be the unique search proposition for traditional search engines.
So, are we there yet, with AI web search?
There is certainly plenty of scope but much more needs to be done in terms of algorithms, data sets and content, which OpenAI is certain to be working away on before its full product is officially, or BETA launched. What will be interesting to see is if there is an OpenAI/Bing version of Google’s SGE, or will it take AI web search to the next level? With the current limitations, alongside the cost of AI versus traditional search — it’s hard at present to justify a full-scale free product.
AI will certainly be a feature within search. But what’s becoming more obvious is that the power of AI will be its integration with products that can be monetised more directly, by being used for bespoke long-form content, where traditional search will still play a useful role in the retrieval of general information. It’s been said before, but it’s worth saying again, that AI is too substantial to ignore. However, there’s a long way to go and even then, we may still be ‘googling’.
Featured image: Sarah Blocksidge / Pexels
