NLP and Search: Optimizing for Natural Language Processing

Google’s Language Systems in Search

Google’s ranking systems guide describes the systems that read language:

So meaning-based matching predates BERT: RankBrain lets Google “better return relevant content even if it doesn’t contain all the exact words used in a search”. When Google announced BERT for Search in October 2019, its announcement gave examples where small words change the meaning, such as a query about a Brazilian traveler to the U.S., where “the very common word ‘to’ actually matters a lot”. The history of these models is in BERT and MUM.

Writing That Language Systems Read Well

Say things in full. A sentence that names its subject and states its point is easier for readers and models than a fragment that relies on a heading for context.

Use precise terms, and define them once. Introduce “Interaction to Next Paint (INP)” before using “INP”. Use the same name for the same thing across the page.

Keep pronouns close to what they refer to. “It” three sentences after the noun leaves the reader, and any system, guessing.

Give each paragraph one job. Passage ranking looks at sections of a page, so a passage that answers one question clearly can stand on its own.

Write the words people use. If searchers say “cost” and the industry says “pricing”, use both where they fit naturally, and explain the trade term.

What doesn’t help: repeating a phrase, inserting lists of synonyms, or padding a page to a target length. Keyword stuffing is one of the practices listed in Google’s spam policies, and its helpful content guidance asks, as a warning sign, “Is the content primarily made to attract visits from search engines?” See helpful content.

Entities in the Text

Language systems recognize people, organizations, places and products in text. Help them, and readers, by:

• introducing an entity with what identifies it, such as a role and organization for a person, or a city and state for a place;
• stating relationships explicitly, for example “PageSpeed Insights, which runs Lighthouse for its lab test”;
• using structured data where a page describes an organization, product or person, so the same facts appear in machine-readable form.

Entity work across a site is covered in entity SEO.

Matching the Query’s Intent

Understanding a query means understanding what the searcher wants to do. A page meant for “how to change a tire” and one meant for “tire change cost” use similar words for different needs. Decide the intent before writing, make the page’s opening answer it, and give mixed-intent queries a page structured for both needs, with each part clearly labeled. Intent types are covered in search intent, and topic coverage in semantic SEO and topical authority.

Structure That Helps Extraction

• A direct answer early in the section that asks the question.
• Headings that say what the section answers.
• Lists and tables for steps, options and comparisons, with the explanation in sentences around them.
• A question-and-answer pattern where readers have specific questions.

This structure also makes passages easier to quote in features such as featured snippets; see featured snippets.

Analysis Tools, and Their Limits

Google’s Cloud Natural Language API offers sentiment analysis, entity analysis and syntax analysis; its entity analysis “inspects the given text for known entities” and returns information about them, including a salience value. It’s a cloud product for developers, not a view into Search. Use it as a rough check that a page’s main entities are the ones you intended, not as a score to raise.

Content-scoring tools compare your page with ranking pages and list terms; read their suggestions as prompts to check coverage, and keep only what the reader needs.

When a tool suggests a term, add it only where a reader of that section would expect it, such as the word searchers use for the thing you describe; a term that appears only because ranking pages use it stays out.