Understanding Site Search — Part Three: How Will You Organize Results?

Someone is visiting your site, but they cannot find what they’re looking for. They browse the navigation. They click around a bit. Finally, they pull the emergency cord: they enter their request into the search box, and they select “submit.”

So. What are they going to see?

A search engine does not understand human syntax or context. It’s a program running a crawl and searching indexed pages for common terms. It does not see words: it only sees a series of characters with which it must match.

Which means, to this search engine, there is no difference between a relevant page and a non-relevant page if they both contain the requested query. We have to provide this context and syntax for the search engine.

In this post, we’re going to talk about how to plan and create a search implementation strategy around delivering the right results. It’s a little bit of programming, a little bit of design, and a decent amount of editorial work.

This is part three of a four-part series on understanding how to plan for site search on your website:

• Introduction: Understanding Site Search
• Part One: How Does Search Work?
• Part Two: What Content Will You Search?
• Part Three: How Will You Organize Results?
• Part Four: Choosing the Right Solution

How will you organize those results?

When you think about “organizing” search results, think back to our initial metaphor of the search algorithm as a complex math equation, with words being worth a specific “amount.”

It’s not really an analogy at all — search algorithms really are complex math equations. And while we can do a lot to tweak how the algorithm finds content, we can also get a lot of distance tweaking the “value” of the terms that show up within the algorithm. Additionally, we can provide filters after the fact: once the results have been delivered, we can tell the search engine to remove non-relevant results.

This section will focus on:

• How are results organized on the search results page?
• How are results honed in after the initial query?
• How do we weigh and influence results to push the correct content to our users?

Search result layout.

The basics of the search result page are simple:

• A search field that includes the current query.
• A list of the content that’s been found, with links to the individual pages.
• Pagination of those results if the list extends past a certain number.

Of course, this is the bare minimum. Layout of search results should probably provide more context, both for the page itself and the list of results as a whole. This includes:

• Summary of each individual page result.
• Any related preview images, if added to the page.
• Content or file type, if applicable.
• The URL or site section, if applicable.
• Publish date, if applicable.
• The number of total results.

There’s a lot of data there, and that’s by design: each of these helps provide context to the page results, allowing users to better understand what they’re clicking as they dive deeper into results. How this page is designed can go a long way in providing usable search results.

Filters, facets, and honing results.

Your search results don’t have to be law. Depending on how your content is set up, you may want to provide users with additional ways to filter down results. Enter the world of filters and facets.

Filters and facets serve similar functions: they provide opportunities to hone results through use of categories or metadata. In fact, the difference between the two is largely a UI decision — filters are often dropdowns, and they are based on a category or other singular common attribute. Facets are more relative, and live along the side of the search results.

From a usability standpoint, the filters and facets come at things from a different angle.

Filters often also allow you to take control of the results as you’re entering the query. For example, on South Dakota State University’s programs search page, you can pre-filter your results based on one of four dropdown — interest area, degree type, locations, or academic department. In this case, you’re actually doing some of the search engine’s work for it — you’re helping hone results before they’re delivered.

When people talk about “facets,” they’re most often talking about filters aligned in the same way as Amazon, showing the entire realm of potential filters at once along the left or right side of the page. From a UI standpoint, this is a bit messier, and it takes up more room, but it’s also an effective form of discovery. If we take Amazon as an example, facets are only going to show you what you can do with the existing results. Rather than pre-selecting a filter and hoping there are results, facets are going to tell you what options are still available.

Weighting and influencing results.

Filters and facets allow your users to control their results around their own set of parameters, but you can also tweak and adjust your results algorithm within your search tool itself, allowing the results page to be more indicative of your users’ needs by weighting and influencing those results.

Remember: a search engine doesn’t fully understand the content it’s working with — it only knows what we tell it. Part of what we “tell” it manifests as a search algorithm — which ends up being a somewhat complex scoring system — in which we identify what parts of a page are important for results, and which ones are not.

For example (and this is a grossly oversimplified example) — we can tell a search engine to “weigh” words in the title field double what they would weigh them in the main body. We can also tell it to weigh any assigned category terms at a higher level. What this does is make a somewhat obvious assumption — that if a word is in a title, or assigned to an article as a category, it’s more important than if that word was just randomly found within the main body.

Then, because of some kind of complex math, those posts appear higher for the related query.

Balancing the weight of these fields is a part of tweaking search results, and it’s not something that can be done right away and then left forever. While there are obvious standard weights — the title field and categories, for example — the rest might depend on what we’re looking for.

In addition, you may make editorial updates to certain pages to not just improve the weight, but to outright influence that page’s standing in your internal search. For example:

• You might implement a keyword field. While the META keyword field provided by most CMSs has been devalued on Google, you may still try to use it to influence your internal search. If you weigh that field at some extravagant level — say, three times the normal weight — you can almost guarantee that the page will appear at the top of the list for the related search.
• Maybe you want to force pages to the top of search results without even worrying about weight. These are often called “Best Bets,” and they are assigned to a specific search query — and, they appear _separate_ from the main search, above the search or in a side bar. This allows you to take common terms — like “biology” — and force a specific page to the top — such as the Biology program page vs. articles posted by the Biology department
• If you have an advanced personalization strategy in place, you might artificially weigh terms that have been recently searched, rather than just the current query. This helps string the past search history into helping find more relevant results.

In essence, what we’re doing in all of these cases is making assumptions on behalf of our users. This can be helpful if your team is active in keeping track of site search analytics — common terms can be fast tracked, or common misspellings can be understood — but too much of this can lead to results that are more in sync with your marketing needs and less in sync with your users’ expectations.

Autocomplete vs. autosuggest.

Finally, understand that search goes beyond the results page itself. Integration of a kind of search pre-filter — such as autocomplete or autosuggest — can help users narrow down what they’re looking for before even getting to the search results page. What’s more, they can provide guidance among a complex set of terms.

There are really three separate concepts here that often get bundled into the same category:

• Autocomplete / type-ahead — As the user types, suggested terms will appear based on the current string. Auto-complete will show all terms with the current query — either those with the query at the start, or those with the query anywhere in the word. For example, if a user types “dar,” it will show terms like “dark” or “daring,” or it could also show terms like “radar” or “standard.”
• Autosuggest — As the user types, suggested terms will appear based on the current string. Auto-suggest will predict terms based on the current string, with suggestion data pulled from an auto-suggest source. For example, on a healthcare site, autocomplete may see “heart” as “heart disease” or “heart attack,” but it could also see it as “cardiac” or “
• Predictive search — As the user types, we skip past the term and begin simply providing results. This is similar to what Google provided with “Instant Search” before they removed the feature in 2017. While this feels helpful, it actually hurts performance and can be very distracting as you jump from term to term as you type. 

Autocomplete and autosuggest are still pretty widely used — the difference is that autocomplete tries to “finish” your word within its existing set of terms, while autosuggest tries to give you word suggestions regardless of their application within your site.

So. Now What?

We’ve focused on both what to index and how to display it. But this may not answer your real question: what search engine too do I choose?

In our final post, we will take a look at how to compare and contrast your options, including a look at two search engines we commonly implement: Solr and Episerver Find.