June 7, 2023
4 Reasons Why Google’s SGE Generative AI Search is…. Disappointing
Generative AI is all the rage. Google made a big announcement in May that it was integrating it into its primary search interface, starting with a beta version it calls SGE. After a couple weeks on the waitlist, I got access to it a couple weeks ago. Most reviews are positive. But I’ve never been one to shy away from an unpopular opinion. Many see this as the future of search, and even I have written about the tenuous future of SEO and traditional content marketing in the face of AI search.
So, I’ll admit it. At least in its current form, I think Google SGE is disappointing. Here’s why:
1. The primary issue is I don’t trust it. We all know generative AI is prone to hallucination, or making up information to sound plausible. But I’ve found it gets even very basic math wrong. Here’s an extremely standard loan calculation that it just completely flubs (the correct monthly payment is $1710.98).
Once I lose faith in something, it’s hard to get it back. I find myself sneaking a glance lower on the screen to the traditional search results to check what the generative AI response is telling me, even if the generative AI result seems strong initially. If that’s the case, why not just skip right to the traditional results?
2. Is it really better than Featured Snippets? In recent years, Google has made increasing use of Featured Snippets, in which it shows a snippet from a web page at the top of search results in an attempt to provide users with an easily accessible answer to their query. When I first envisioned Google SGE, I imagined it to be a massive step up from Featured Snippets.
But… is it? Here’s an example in which the generative AI response is virtually identical to the Featured Snippet. Both clearly source from the same article, and the answer to my question is the same. What’s the point?
3. The results can be irrelevant. Here is an example in which I search for tires (yes, I’m actively in the market for tires for my car), and 2 of the suggested options are actually rims, not tires.
If I scroll down just 2 more inches, I find a human-curated well-researched list that exactly answers my question, which is superior to the generative AI answer. So in this case the AI search result is just a distraction.
4. The results are inconsistent. If I do the exact same search a second time, I get a completely different response, in this case an excellent list of options with good explanations, which is exactly what I was looking for. How can the result be so different to the exact same question just seconds from each other?
This list is not meant to suggest that AI search is a bust, or that it is never useful. That’s definitely not true. Sometimes the results are great (as in #4 above), and I have no doubt the tool will get better quickly. But we are all going to need a little patience and understanding along the way.