Google’s somewhat muted announcement of a major upgrade to its search algorithm, named “Hummingbird,” is a much bigger deal than it looks, and much more of a milestone in our relationship to computing than we might think. So what’s going on here?
Just the Facts, Ma’am
People like getting answers, fast: the Internet was, to a degree, built for this originally. Over its first two decades, it boomed, busted, and boomed again on the premise we’d want to act (and transact) on what we found out. This is what the e-commerce folks labeled, prosaically, ‘conversion’. Consider some quotidian musings like these: “Can I afford a vacation?”, “How can I make my retirement last?”, or (I confess it’s dinnertime as I write this): “How do I make spaghetti carbonara?”. Of course, at some point we will search online for answers, and we almost certainly use Google.
Let’s look at how this usually worked, say since about 1998 or so, if not earlier.
To an extent, we have been trained by the desktop/browser context of the pre-mobile era, and years of experience with search engines (and their example phrases) to enter individual word strings, (aka ‘keywords’) to get the most exact result:
‘cheap flight Hong Kong’
‘retirement programs 401k IRA’
‘carbonara recipe easy’
But, with the increasing ‘naturalness’ of computing experiences, the intimacy and directness of mobile devices (originally designed, after all, for talking, not for keyboarding), and the proliferation of ‘answer’ sites in recent years, the ‘keyword’ way of searching increasingly began to change to the ‘conversation’ way of phrasing your search:
‘How can I fly to Asia for under a thousand dollars?’
‘What retirement programs offer the least risk?’
‘What can I make with bacon and a box of spaghetti?’
Conjunction Junction
Syntax is what connects words together. In the ‘keyword’ days of search, this wasn’t an issue. Meanings were more or less known, referenceable, and trackable as individual keywords appearing on webpages, and thus were born the strategies of the simpler days of search engine optimization (SEO): keyword optimization and various other techniques to ensure that certain words or word combinations were readily findable and relevant in a ‘natural’ search result. The business side of this, and how Google became the juggernaut it is today, is known as search engine marketing (SEM): figuring out the best keywords and keyword strategies to bid on in order to generate paid search displays and clicks.
Now, while understanding natural language sentences like the ones above is child’s play to a native speaker, creating a program to parse context and semantics together to understand the speaker’s true intention (‘semantic search’) is devilishly difficult. This has been a computing struggle since the days of ELIZA, which mostly responded generically, and often comically, when it couldn’t process the language. To be sure, Siri more than occasionally does this even today when the topic veers from more easily parsable “Where is, What is, Who is” topics (what linguists call, aptly enough, “Wh-questions”).
Speaking of Siri: understanding the same spoken utterance as one which is written down is qualitatively more complex on a massive scale. Why? Because what we write isn’t always exactly how we say it. Say the words “This year”. Say them again a bit faster—and listen. Comes out kind of like “the shear”, doesn’t it? That’s because individual sounds adapt to the contexts they’re in, and the sounds around them—and the faster we talk, the more pronounced this ‘assimilation’ process is (not that we notice the difference: we pretty much always hear what we think is “this year”, and don’t think the interlocutor is talking about gardening implements). Therein lies the challenge for natural-language processing algorithms.
What Did You Say?
In short, there is a complex interplay of natural sound variation, meaning, and context that lets us know what is meant. More and more, as people interact more naturally with computing devices, asking questions in natural language, and speaking directly to the device, programmers will be diving ever deeper into the mystery of language itself. And SEO/SEM specialists will be following very closely behind…
All just so I can find that perfect spaghetti carbonara recipe.
Sean Ketchem, PhD, is a branding and content strategy consultant based in San Francisco. You can reach him at csketchem@gmail.com.
