Since we launched Swiftype back in 2012, we’ve worked tirelessly to build the best website search product around. Our founders, Matt & Quin, started Swiftype after discovering site search has been broken for a long time. Of course, the single most important element to search from the searcher’s perspective is relevance of the results. Many of our key features are centered around making it easy to ensure your website’s search results are relevant, from the search engine we build for you providing relevant and up to date results from the moment of your first crawl to allowing heavy customization of your results if you feel they aren’t quite right (by the way – keep an eye out, we’ve got some really exciting stuff coming on the customization front).
However, just because we’ve built a great product doesn’t mean we’ll quit making it better. Quin, our CTO has mentioned that one of the things that makes search so interesting is that he considers it a moving target – something that can always be improved on. In that spirit, we’re incredibly excited to announce a major update to our search algorithm, which now features phrase scoring! This means that our algorithm will now focus on better matching of your search results to the meaning of your entire query, rather ranking based on relevance to each individual keyword in the query. It’s a bit of a subtle difference, but it has some major implications on search results relevance.
A common misconception about most site search products is that the search engines generally understand queries as humans do. For example: if I told you I needed to buy a pair of dress shoes, you would immediately recommend a shoe store, not a dress shop. However, most website search engines use simple keyword matching, which means they rank documents based on their relevance to the individual keywords in the query, rather than the meaning of the query as a whole. In the given example, this likely wouldn’t pose a huge problem, as the most relevant results would be the ones that included both dress and shoes – although there might be some unexpected products further down your result set! However, imagine if you had searched for “black dress shoes.” Without phrase scoring, black dresses are about as likely to show up as dress shoes.
For another illustration, let’s go through an example that we’ve seen in some of our clients.
A very common relevance signal considered in ranking documents against queries is recency. For an illustrative example, consider TechCrunch. Some of their most popular content is news about the iPhone. As TechCrunch is a news site, recency is one of the most important relevance factors, so a search for iPhone will rank news on the new iPhone 6 highly. However, what if you are researching the history of the iPhone, and are looking for press on the iPhone 3? Since most site searches don’t offer phrase scoring, they’ll likely rank results based on relevance to each individual keyword, as well as recency. Since 3 is an extremely common keyword, the results will again be heavily weighted towards recent stories on the iPhone.
Now, with Swiftype, performing a search for “iPhone 3” will consider the sets and order of words. Our results ranking is based on how closely the result matches the set of words, so the results will be much more likely to be relevant to the actual meaning of the search. We’ve been beta testing this feature for a few months with select clients, who have seen improved click-through rates, conversion rates, and retention rates, as their visitors are finding what they are searching for much more often.
Now that we’ve rolled the new algorithm out to all our users, you’ll see immediate improvements to your results. The impact will be felt most by websites that contain large amounts of text, as this is when considering the meaning of the entire query is most meaningful, but all our users should see improvements. For an example of the impact Swiftype’s phrase scoring can have on search results, compare TechCrunch’s search to Mashable’s search. Since those are live search links, I went ahead and took screenshots in case your results look a bit different:
All of TechCrunch’s results are clearly about the internet of things (and check out the great faceting they’ve implemented). However, none of the top results on Mashable are at all related to the internet of things, instead being ranked on relevance to the term “internet,” or if I had grabbed the sixth result (The 7 Most Worthless Things at Your Garage Sale), the term “things” (by the way, if anyone from Mashable reads this, go ahead and contact us, we’d love to help).
This is just one example of how we’re continuing to innovate and build in the website search world. This isn’t a feature widely available in website search solutions, or in the out of the box search you get from a third party website platform. We’re looking to fix site search for the broadest possible audience, which is very difficult and complex task. Sometimes it means building a drag-and-drop results customization tool, other times it means releasing feature updates for those of you using our WordPress Plugin. But at its core, search is about relevant results and ensuring your users find what they are looking for, and we are constantly working to make sure we offer the best solution you can find.
To find out more about how Swiftype can fix your search problems, drop us a line.
