Everything you need to know about natural language search

Search engines like Google, Bing and others are making efforts to bring searching for information in line with everyday conversation with a type of search called ‘natural language search’.

This development is a move away from the type of searching that has dominated the web since the advent of search engines in the 1990s. It is part of an attempt to make searching faster and more effective by understanding searcher intent and more complex, multi-part queries.

Natural language search is also key to a number of advancements currently taking place in technology, including voice search, digital assistants and smart home hubs. But what exactly is it, and how is it going to affect the way that we look for information online?

What is natural language search?

Natural language search is search carried out in everyday language, phrasing questions as you would ask them if you were talking to someone. These queries can be typed into a search engine, spoken aloud with voice search, or posed as a question to a digital assistant like Siri or Cortana.

This is as opposed to keyword-based search, which is what most people who are used to using web search engines still default to. Keyword-based search is an attempt to break down a query into the most important terms, getting rid of unnecessary connecting words like “how”, “and”, “the”, and so on.

So if you wanted to know how high the Empire State Building was, a keyword-based search query for that information might be “Empire State Building height”. But if you were searching using natural language, you would phrase your query as, “How high is the Empire State Building?”

“Don’t speak in these weird haikus.” CollegeHumor’s ‘If Google Was a Guy’ series satirises keyword-based search queries by placing them in a life-like context. 

Natural language search has always been around – think of Ask Jeeves, the 1990s search engine which encouraged users to phrase their queries in the form of a question. But Ask Jeeves was ahead of its time; keyword-based searching was the norm then, and Jeeves found itself out-competed by more powerful keyword engines like Google.

Several years too late for Ask Jeeves, search trends are coming back around towards natural language search. This is the result of a number of different developments in search and technology coming together.

The trend towards natural language

First of all, search engines – particularly Google – have improved their search capabilities so much over the years that people expect to find exactly what they’re looking for on the first try.

There’s a reduced patience for sitting and trying different keyword combinations; people are searching on their mobiles, on the go, and they want to be able to ask a question, get the answer, and move on. And search engines have worked hard to meet this expectation, so that people will feel satisfied with the service they provide instead of frustrated by it.

Secondly, search technology has improved to the point where we can begin to teach search engines to understand longer, more complex queries, with different components that modify each other and can’t operate independently.

Google recently published a blog post welcoming “complex questions” and illustrating how its search engine can now understand superlatives (tallest, largest, oldest) and “ordered items”, such as a list of the largest cities in a given state, in order of area.

Google now also has an improved ability to interpret specific dates, and complex, multi-part queries like “Who was the U.S. President when the Angels won the World Series?”

You’ll notice from the screenshot that Google doesn’t just aim to serve the right answer to that question, but to display it within the Knowledge Graph at the top of the screen, eliminating the need to even click on another site in order to find the answer.

The third key component contributing to the development of natural language search is the rise of voice search and digital assistants. As Rob Kerry noted in his presentation on the future of search at Ayima Insights digital marketing conference, “It’s becoming a lot more common for people to search by talking into their phone.”

Amazon’s Alexa, Apple’s Siri, Microsoft’s Cortana, Google Now – these AI assistants are being adopted more and more widely, and their voice activation capabilities increasingly integrated into technology.

The technology might have had its stumbling blocks at first, but as with Google, it’s improving steadily, leading people to become much more used to speaking commands and queries aloud in everyday, natural language and expect an accurate response in return.

Will Oremus of Slate put the progression to natural language search best when he described it as a move from us speaking the language of computers in order to communicate with them, to us teaching them our language.

“In the beginning, computers spoke only computer language, and a human seeking to interact with one was compelled to do the same. First came punch cards, then typed commands such as run, print, and dir.

The 1980s brought the mouse click and the graphical user interface … the 2000s, touch screens; the 2010s, gesture control and voice. It has all been leading, gradually and imperceptibly, to a world in which we no longer have to speak computer language, because computers will speak human language—not perfectly, but well enough to get by.”

It’s not just Google

As the world’s most popular search engine, it’s not really a surprise that Google is leading the pack in natural language search advancements. But it hasn’t been the only search engine to do so by any means.

In 2014, Microsoft made updates to Bing smart search which improved its parsing of natural language queries. A few months later, it developed on this even further by introducing the ability to “continue the conversation” after asking a question in search. In other words, you can ask a follow-up question which depends on the previous one for context, and Bing will understand what you mean.

This works, and it’s quite impressive – ask Bing who the “President of America” is, and then in a separate query, ask “how tall is he?” and you’ll get the right answer, with the height of the First Lady and a couple of other presidents thrown in just in case.

Of course, it depends on the first question being one that Bing can answer, which mostly restricts it to simple “who is…” or “how is…” questions. Still, not even Google, which always treats individual searches as a new query, can do this, and it’s a big step towards the kind of frictionless, conversational searches that natural language search aspires to.

Newer search engines are also making natural language search capability their goal. Plonked, a niche business-focused search engine which launched in March, aims to provide its users with a natural language interface in order to keep up with the level of searching offered by Google and other major search engines.

I also made an unexpected discovery when researching this article, which is that Ask Jeeves is not the only natural language question and answer service left over from the 90s. START is a “natural language question answering system” developed by the InfoLab Group at MIT, and it has been online since 1993.

START functions more like a reference book than a search engine, designed to give factual answers to questions in fields like geography, science, history and culture.

It also might be a little out of date. But it has an ability to puzzle out the different components of a complex query in a way that Google could stand to learn a thing or two from – and by the look of things, has been doing it for much longer than Google has.

Where next for natural language search?

We’ve published a number of articles on where Google is going with search, from using Hummingbird to better understand searcher intent, to employing RankBrain to guess at the meaning of never before seen questions, to making strides towards semantic search.

Natural language search is bound up with all of these, since these are all capabilities that would allow Google to better interpret and respond to search queries in everyday language. So I think it’s fair to say that we can expect much better and more accurate natural language responses from Google as these algorithms learn, develop and have their limits tested.

But as we’ve established, it’s not just about Google. There’s a possibility that we’ll see natural language search developing in a few different directions as Bing furthers its ‘conversational’ search style, and other search engines play to their own strengths.

The kinds of natural language search queries that a niche engine like Plonked needs to interpret and respond to could be very different from those put to a general search engine, which could lead to some interesting advances in unexpected areas.

Screen capture via GV Commerciais

There are even bigger developments taking place in the ‘digital assistant’ field, which is heavily tied together with voice and natural language search as digital assistants handle search queries along with a myriad of other tasks. Siri, Cortana, Alexa and others are all threats to Google’s dominance of search, and there are rumours that Google is planning to develop its own voice-controlled assistant device to take on Amazon’s Echo in the smart home space.

Natural language search is not only a direction that search engines are overwhelmingly moving in in order to better understand the goals and desires of searchers online, but also a key component of some of the most important – and, let’s be honest, futuristic – developments currently happening in the field of technology.

The article Everything you need to know about natural language search was first seen from https://searchenginewatch.com