And though the Gemini-powered technology has improved its accuracy dramatically over the last two years (unfortunately for publishers), AI Overviews still gets basic questions wrong. And that includes spelling tests.
Google's AI tools remain abysmal at answering questions about spelling, having gone viral two years ago for responding to the question "how many r's are in the word strawberry?" incorrectly. But it's still bad. Last 26 May, X user Naomi Rohatyn tested the large language model's (LLM) current ability to answer to a spelling question.
"There are exactly 2 'e's in the word "astronomical" (a-s-t-r-e-n-o-m-i-c-a-e-l)," replied AI Overview.
How many Ps are in Google? According to Google, there are two.
There’s also is also "exactly 1 'r' in the word 'poop'," Google’s AI Overview says, as well as two ‘d’s in the word journalism, yet spelled it: j-o-u-r-n-a-d-i-s-m. Google did at least identify that there is one P in the last name of the U.S. president, but spelled it as t-r-p-u-m.
Considering users are less likely to click on links when an AI summary appears in the results, surely the information provided in AI Overviews should be accurate. But it's complicated.
AI chatbots need exact context and specifics to answer as well as they can, so surely spelling words within their training data seems easy. However, things get knotty when you ask an LLM to consider words letter-by-letter, as the model will process text in chunks rather than individual characters (it's called tokenisation).
"Counting within words has been a known challenge for LLMs, and we’re working to fix this particular issue," Google told TechCrunch in an emailed statement.
These basic spelling errors may seem familiar. LLMs, the kind of artificial intelligence that powers chatbots and other text-generators, are not built to understand spelling. It’s been a running joke for years that whenever a company unveils a new AI model, you should ask it how many 'r's are in the word strawberry. These AI models — which can code an app in seconds, or solve problems that have stumped mathematicians for decades — are about as good as a kindergartener at spelling.
Google’s AI overview woes reach beyond silly spelling mistakes though. Google already patched an issue from last week in which searching the word "disregard" would yield what looked like a dictionary definition of the word, only the definition was shown as, "Understood. Let me know whenever you have a new prompt or question!" But these spelling errors have remained amusing because they’re so difficult to quash.
"LLMs are based on this transformer architecture, which notably is not actually reading text. What happens when you input a prompt is that it's translated into an encoding," Matthew Guzdial, an AI researcher and assistant professor at the University of Alberta, told TechCrunch. "When it sees the word ‘the,’ it has this one encoding of what ‘the’ means, but it does not know about 'T,' 'H,' 'E.'"
The token-based architecture that powers LLMs like Google’s AI overview is inherently limiting, and researchers haven’t been optimistic that they can solve the spelling problem.
Read More




