The Swiftype Blog / Category: Support

Questions about your Swiftype installation? Our Support Team has answers.

Congrats on your Swiftype implementation! You’ve already started delivering a powerful site search experience for your visitors. Trust me, they’ll thank you for it.

We’ve been listening to our community and we’re here to answer 4 of the most commonly asked questions from Swiftype customers.

1. Why do my results all look the same?

When Swiftype’s crawler, lovingly called “Swiftbot”, crawls a domain, there’s potential for repetitive template elements to be indexed with the page body. Most notably the navigation/header, sidebars, and/or footer content. With all the great, meaningful content encased in template noise, the quality of the customer’s search experience can be negatively affected.

For example, Apple.com has these elements on their website:

support_blog_img1

Since these elements appear on all pages, when their site it indexed, in the Swiftype dashboard, it looks like this:

support_blog_img2

You can easily clean this up by using our Content Inclusion/Exclusion tag recognition. By adding Swiftype specific meta tags (data-swiftype-index=’true’) to the HTML container(s) that holds the primary page content, it’s possible to instruct Swiftbot to index only those sections of the page body.

The best practice is to set the main content container to true. If you want to further refine what’s indexed from that section, you can add additional tags with a value of false to containers nested within.

Example

<body>
    <nav>Blah Blah blah</nav>
    <div id="main_content" data-swiftype-index='true'>
         <p>All of my sweet, sweet 5/7 content is going to go in here.</p>
             <div id="ad_widget" data-swiftype-index='false'>This bit really isn't as important which is why I'm going to add a 'false' exclusion parameter.</div>
         <p>This bit will be indexed though, because it’s still within the ‘main_content’ div that’s set to 'true'. Everything outside of the ‘main_content’ div container will be ignored, yo.</p>
             </div>
    <footer>Copyright Attempting to Sound Official© 2016</footer>
          </body>
        

2. How do I prevent Swiftype from indexing certain pages of my site?

For crawler based engines there are three approaches you can take to determine what pages are indexed from your domains: URL path rules, a customized robots.txt file, and robots meta tags.

Path Rules:

From the Manage > Domains section of the Swiftype customer dashboard, there’s an option for each domain that will allow you to ‘Manage Rules’ for that domain. From here, you can define specific paths to include (Whitelist) or exclude (Blacklist) when crawling your site.

Examples of common cases are the exclusion of /category/ paths for ecommerce sites, so the focus is exclusively on crawling product pages. For other CMS based sites, you’d likely see paths to login or administrative pages excluded, as well as dynamically generated content, like pages based on tags or categories. More examples and tips on using this feature can be found here.

Robots.txt Files:

A robots.txt file is a plaintext document that you can upload to the root directory of your website’s domain. With the robots.txt file, you can define URL path exclusion rules for all or only specific web crawlers to follow. Many websites will commonly have a robots.txt file already in place, and it’s presence is one of the first things Swiftbot will look for when starting a crawl process.

Check out our Robots.txt documentation to learn how you can leverage this with Swiftype.

Robots meta tags:

If you need to exclude content in a more precise way (page by page or page template basis), we recommend and fully support robots meta tags. We adhere to the robots tag standard that’s a companion to the aforementioned robots.txt file.

This means that we’ll pass over any page we attempt to crawl that contains a meta tag:

<meta name="robots" content="noindex">

You can also configure these tags so they only apply to Swiftype’s web crawler:

<meta name="st:robots" content="noindex">

Similar to the robots.txt file, meta tags are configured and managed outside of the Swiftype dashboard via their webhost/CMS.

You guessed it. We’ve got documentation of robots meta tags support here.

3. “Why are pages missing from my search engine?”

Here are 3 reasons why Swiftbot, our crawler, may not be able to locate and index pages on your site:

A. We’re unable to find the content because it’s not linked to from other pages.

When spidering a domain, the Crawler will examine all links within a page to discover URLs that are part of the domain submitted to the engine and also adhere to any configured path rules (see question 2 above). If content exists within a site that is not linked to from another known page, within the site’s navigation menu, or listed on a domain’s sitemap, chances are Swiftbot will not be able to locate it.

One of the best ways to ensure our Crawler is able to index all desired content is to include a current sitemap. Sitemaps are files that are typically stored at the root of a URL domain and contain a list of links to pages on their site(s) that are available for crawling. Our documentation for sitemap support and installation notes can be found here.

B. Improperly configured canonical URL elements/tags.

A canonical link element is often used like a meta tag, in order to prevent duplicate content indexing issues by pointing web crawlers to the preferred (or canonical) URL version of a web page. A scenario that can occur is that a customer’s site content is configured with static canonical tags that all point to the root domain URL.

Due to this misconfiguration, even if the crawler finds links to all the content on the site, it is being given instructions that all pages on the site are another version of the home page. With that directive in place, only the home page will be indexed. If the customer re-configures or removes those elements, a recrawl will be able to index their content successfully.

For best practices on canonical elements, you can refer to Google’s documentation here.

C. The content is being excluded by one of the methods noted in question 2.

Just as with misconfigured canonical link elements, misconfiguration or conflicts in a customer’s path or robots rules can cause pages to be skipped over.

4. My site is password protected / behind a firewall / hosted on our company’s intranet.

It is possible for Swiftbot to crawl secured content, but you’ll first need to make minor configuration changes to your web or intranet site’s host server.

All Swiftype accounts have an account specific User-Agent ID string. By whitelisting this identification string with your server, you can allow, or disallow, crawlers access to your site’s content.

Swiftype has a unique security feature where we encode our crawler’s User-Agent with a secure key that is uniquely tied to your Swiftype account. This approach enables you to limit access solely to Swiftype’s crawler, and is an extra level of security many customers enjoy.

If you’re interested in using the Swiftbot web crawler to access your secured content, please contact our support team and we’ll be happy to supply you with your account specific User-Agent ID string.

Hopefully these answers to commonly asked questions will point you in the right direction. If you ever have questions, suggestions, or feedback, you can always email [email protected] to reach our team. We’re happy to help!

Meet Zhen Liu: Got Questions? We have Instant Answers!

HowCanIHelp

We live in a world where the solution to nearly any problem can be solved instantly. When you need the answer to a question, there’s Google, or Siri. If your refrigerator is empty at dinnertime, you have Postmates or Munchery to the rescue. So when you need help for a product you purchased or service you subscribe to, waiting is not an option. As a Customer Success Manager at Swiftype, I help our clients build upon the success of their own customer service departments through the business of instant answers.

Many of Swiftype’s customers use our search software for their Knowledge Base. Knowledge Bases can vary from a FAQ or Help section provided for end consumers as a first stop in the customer support chain, to internal documentation for developers using a product. No matter what the final application looks like, help centers are becoming an integrated part of the product experience today.

Customers are searching now more than ever
Thanks to Google, we’ve all been trained to search. Customers and employees alike expect a self-help channel to service themselves prior to getting on the phone for support. In fact a Forrester survey shows that for the first time in the survey’s history, customers of all ages are using the FAQ pages on a company’s website more often than speaking with an agent on the phone.

This change in user behavior has generated a significant need for our customers to have not only a robust Knowledge Base that they themselves can navigate and find answers within quickly, but the ability for their end customers to quickly search those databases and find answers to their questions instantly.

The need for instant answers is not confined to one industry or solution. Speed and accuracy are critical in a search solution within knowledge bases across the board.

Providing Instant Answers Three Ways
Customer Support teams are using Swiftype as a means to aggregate content in one location. A customer of ours has content that sits in multiple subdomains of their site for developers, designers, their sales team, within their blog, training center and help center. As each individual portal grew, it became increasingly difficult to quickly find the information you were looking for. They’ve successfully leveraged Swiftype to search across all content types, creating a one-stop locale for all internal users when they are unsure of where to locate content.

Another way Swiftype has been implemented by customers internally is to better support call center teams. We are the internal search engine for a global enterprise organization with many call center teams who need to be able to answer questions for their end customers quickly and accurately. If decreasing call time and increasing customer satisfaction are critical metrics for your team, consider ways to make your employees more successful with a customizable search solution.

Lastly, we’ve even been leveraged for instant answers before your customer is even actively searching. Swiftype results are shown to users in a right-hand drawer that slides in when users are in their dashboard for our customer’s product, suggesting answers that are related to the page they are on.

Maintaining a best-in class customer support system is critical to businesses. Knowledge bases are a key component to providing an excellent experience. Gone are the days where your customer is willing to wait 10 minutes on hold with a customer service representative, and another 20 minutes to explain their issue, and hopefully receive a solution. The faster you can provide an answer to your customer, the happier and more satisfied they leave.  And I’m in the business of making happy customers.

New Help Center Search Implementation Checklist

Adding powerful search to your company knowledge base or help center is an essential step toward helping users resolve issues without filing tickets, but the process of implementation can often be a daunting first step. To make this process easier, we added a new resource to our website for customers looking to improve search on their help center or knowledge base.

15 Steps to Help Center Search Implementation

Last week, we added a new resource to our website for customers looking to improve search on their help center or knowledge base. Written with a non-technical audience in mind, 15 Steps to Help Center Search Implementation introduces readers to the most important questions that companies should familiarize themselves with before getting started, including:

  • What are the pros and cons of building search internally?
  • How long will implementing search take?
  • What team members are required to implement new search?
  • How can I evaluate the performance of a new search experience?
  • What ongoing work is required after implementing search?

With a step-by-step checklist that clearly lays out each phase of implementation, this resource will be an invaluable guide. Implementation is often an intimidating prospect with any new software, but with this guide in hand, your team will start with a much clearer sense of what needs to be accomplished and what team members will be involved.

To access the checklist, follow the download link below. To receive a personal demo of what Swiftype can do for your knowledge base or help center, contact us today.

New: Knowledge Base Guide to Search Analytics

While personal customer support remains an essential ingredient for attracting, retaining, and creating loyal customers, this level of individualized attention is expensive to support difficult to scale as your company grows. In response to this challenge, creating a comprehensive user-facing knowledge base that enables customers to resolve support issues without contacting your team can save companies time and money while also improving the overall user experience.

Read Swiftype's new knowledge base guide to search analytics.

Once in place, any steps that a customer support team can take to optimize this knowledge base and create a more efficient user experience can substantially decrease inbound support volume. In this pursuit, creating a centralized search experience that allows support center visitors to quickly and easily find the content they are looking for is vitally important.

Still, while the importance of search across knowledge bases is fairly self-explanatory, customer support teams often overlook the valuable insights available from user search behavior—insights which include:

  • What are the most pressing issues my customers are facing?
  • What issues are users unable to resolve on their own?
  • What new support content needs to be created?
  • How relevant are the results for users who search across my knowledge base?

To give support teams a clearer sense of precisely what information they should be looking for from their search analytics and to provide actionable recommendations about how they might improve their on site search, Swiftype has created the Knowledge Base Guide to Site Search Analytics.

To access this guide and learn how customer support teams can leverage knowledge base search analytics, follow the download link below.

Teaching Swiftbot to Intelligently Index Images

When creating search engines, the first and arguably most important step is indexing website information in a structured format that is optimized for a specific search algorithm. The specific information you index and the structure by which you organize this information (also known as the schema) dictates how your search engine will determine relevance, what your users can search by, and what information you can display in search results.

How does indexing work?
While there are numerous ways to customize and control the information you index in your Swiftype search engine (for example, via our API or one of our platform integrations) we aim to make this process as simple as possible for non-technical users by automatically indexing website information with Swiftbot—our high performance web crawler designed to index information from a specific URL.

Swiftbot allows non-technical users to get up and running with a working search engine in minutes by simply entering their website URL and letting Swiftbot index their website for them. A major component of Swiftbot’s technology is the logic that our engineering team has built in to parse website HTML and index it in a structured format that works with Swiftype’s advanced search algorithm and information retrieval method. (To learn more about the technical challenge of building a search engine, read our white paper on the subject, written for a non-technical audience).

Building an intelligent web crawler
Because almost every website is built and structured in a different way, teaching Swiftbot how to effectively read, sort, and organize information from a website’s HTML base is an ongoing challenge. While we do allow site owners to completely customize the default information Swiftbot indexes from your website with custom <meta> tags, not all users have the technical resources or knowledge to do this on their own, so Swiftbot is also built to make many of these indexing decisions on its own.

HTML windows

With every website structured differently, how do we teach Swiftbot to intelligently index this information?

Still, with websites differing so dramatically from one another, indexing the right information in the right format from each page is no easy task. In particular, identifying the most important image from a web page and associating that image with a search result is a multifaceted problem, since there are many images on every page and these images often have different filename structures and/or occupy different locations on a page.

images in search and autocomplete

Adding images to search results pages and autocomplete menus can create a much more engaging search experience.

Nevertheless, indexing images allows site owners to create much more engaging search experience, adding thumbnails of varying sizes to their autocomplete and search results that let users see a preview of the page content before selecting a result. So, in a recent update to Swiftbot, we’ve built in conditional logic that automatically indexes images from your website pages (provided there are no Swiftype specific image tags already in place).

How does Swiftbot decide which image is “best”?
To teach Swiftbot how to index the “best” image from web pages, we had to build in logic that would overcome a series of challenges that result from the varying nature of website pages.

  1. As a starting point, we decided to leverage existing open graph <meta> tags (such as Facebook and Twitter <meta> tags) that many site owners use to prepare their content for sharing on social media platforms and other content distribution networks. By teaching Swiftbot to obey these <meta> tags if no Swiftype specific <meta> tags exist, we created hierarchical indexing logic that more intelligently sources images from existing website metadata.
  2. Secondly, we know that many websites have a large number of images that repeat across many, if not every page on their website (for example: a company logo, images in the header, footer, and sidebar, author headshots, ads, etc.). To ensure these images are not considered the “best” image for a specific document, we built in logic that identifies and rules out these repeating elements as candidates. Similarly, we do not want to index advertisements, so we run any images on the page against an ad server blacklist to ensure these remain out of consideration.
  3. Thirdly, we compared data in the alt attribute of each <img> with the url and <title> of that page, assigning a relevance score to those images based on how closely the alt description matched this page information.
  4. Lastly, Swiftbot looks for common CSS classes and id’s to locate the main content area of each page—another step that helps rule out extraneous information such as the header, footer, and sidebar.

Taking all these pieces of information together, Swiftbot assigns the images on the page a relevance score and indexes the image it judges to be the “best” image for that document. As this new indexing process gains wider use and we gather feedback from customers, we will continually work to improve our image extraction technology over time.

Adding these images to search
Once these images are indexed from your website and in your search engine, the question becomes: how do I display these image thumbnails in my search results and autocomplete dropdown? While there are many ways to style your autocomplete and search results (including using Swiftype’s web components or jQuery library) the best choice for users with very little technical experience is the Result Designer, which allows users to style their search results entirely from the Swiftype dashboard without writing any additional code. To learn more about the Result Designer, watch our dedicated webinar explaining this tool and offering best practices advice from the Swiftype customer success team.

What Are Swiftype Web Components?

I do a lot of Swiftype integrations with enterprise customers. Search applications are like christmas trees – everyone knows what they look like, but have very specific ideas about how to customize them. For developers, having the right resources to meet this expectation, in the desired project timeline, is a must. That is why we’ve launched Swiftype Web Components, a resource rich website built to support developers who are responsible for integrating search for their company or client.

Web Components is an emerging standard for developing more complex HTML elements. Swiftype Web Components is based on the React javascript library, but it’s not dependent on React. So, we’ve actually built a library that supports React, but can also support other web component libraries; such as Riot, Polymer or webcomponents.js, which follow Web Component design patterns. For those unfamiliar with React, it is a very popular JavaScript library that came out of Facebook.

The idea of web components is that you can take a complicated application, like a search results page, and break it up into a bunch of different components that all operate independently. In our catalog, which is the first place to start, we have about 30 to 40 components that all can sit on the page together and work to provide a powerful search interface. Each component is optional and can be composed easily to fit the desired experience. For example, some customers might not want faceting so they don’t have to include that component. Other things like pagination, most people are going to want and they’re going to put on every search results application.

The component catalog is broken up into categories for organization. Some categories are sorting, faceting, filtering, links, how to handle misspelling, pagination etc.. All these items are components, that put together, deliver a really powerful search application. Our goal is to standardize these components to make everybody’s search results powerful, easy to use, and based on the same flexible core library.

A developer who’s learning about how to use components can begin by clicking the Get Started Now button, which leads you to our install tutorial. We’ve built a starter kit, which brings you to a small search application that we’ve built. The nice thing about what we’ve built is that you can actually edit the search application and start working with the code from your browser without having to set up a development environment or anything. When you’re happy with your changes you can share the link with others and download the customized files.

Our site also includes technical documentation for the library and examples that demo how components can be composed to build different search applications.

We’ve love to get your feedback. So please reach out to us and let us know what other resources you need to be able available to build search very quickly and easily.

How to Index Thumbnails for Crawler Based Engines

As you’re getting started with Swiftype, you may be wondering how to index thumbnails from your website and serve them to users in your search results. The answer to this question lies in using Swiftype’s custom <meta> tags, which allow site owners to pass detailed web page information directly to Swiftbot, our web crawler, as it moves across your site. As Swiftbot encounters these custom Swiftype <meta> tags, it indexes their content and incorporates that information in your search engine index schema.

To index thumbnails from your website, all you need to do is add a Swiftype image <meta> tag to the <head> section of your website template that indicates where images are located on your various page types. For illustration purposes, the Swiftype image <meta> tag is formatted like this:


Swiftype recommends placing these <meta> tags at the template level of your website to ensure that image files are dynamically populated within the tags, rather than being added manually for every page on site.

NOTE: the value of the “content” attribute must be HTML encoded. For more information see this guide.

Alternatively, you can wrap images with a body-embedded Swiftype image <meta> tag to avoid changing your website <head>. For example, Swiftbot will index example.jpg into the image field from the HTML below:

<body>

Hello world

 

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed ut risus sed ante dignissim pharetra aliquet a orci. Maecenas varius.

 

In in augue molestie, bibendum velit vel, luctus erat. Curabitur cursus, tellus at feugiat lacinia, tellus est suscipit lectus, non commodo diam elit sit amet justo.

http://fullurl.com/example.jpg </body>

It is important to note that in both the <head> and <body> embedded <meta> tags, you need to specify the data-type attribute as enum. For images, this will always be the case. For any other custom meta tags you choose to define, each attribute must be a valid, Swiftype-supported field type, which you may read about here.

Once you index thumbnails from your website, you can easily customize your search results and autocomplete to feature thumbnails in a range of shapes and sizes with the Swiftype Result Designer.

To learn more about using custom Swiftype <meta> tags to refine your search engine index, check out our tutorial. As always, if you need help or have any questions, feel free to reach out to us at [email protected].

Swiftype for Zendesk

Zendesk_logo_RGB

As we’ve written before, search is a critical tool for self-service support centers. Users who enter the documentation or support section of a website generally have a clearly defined issue in mind, and a powerful search bar provides these users with a clear path to the content they are looking for.

To help more support centers bring great search to their users, Swiftype is excited to announce our integration with Zendesk Help Centers, making it easy for knowledge bases built with Zendesk to deliver the search experience that customers expect. With Swiftype powering search on your Zendesk Help Center, site owners have the power to customize results for individual queries, ensuring that users always find the most helpful content. Furthermore, Swiftype provides detailed analytics on what your users are searching for, providing key insight into user issues and identifying content gaps by populating a list of top queries that return no results.

With these tools and more, site owners can redefine their Help Center user experience by providing powerful and customizable search without the need to consult engineering. Get started by creating account today.

New feature: Live Index Preview tool

Today we are excited to announce a new dashboard feature for site owners with crawler based engines: the Live Index Preview tool. Some of you may have noticed this feature since we rolled it out a month ago, but we announce it today in conjunction with our new meta tags tutorial, which walks site owners through the steps required to implement Swiftype meta tags on your website template.

Now when you log into your Swiftype account and visit the Content tab, you have the ability to compare the live state of your site’s content with what Swiftype presently has indexed for your search engine. To do this, click on the Details button for a page document from your main Content tab view, or look up a specific URL from the search box.

On the Details page, switch from the indexed page overview to the live view by clicking on the bolt icon. Once you switch to live view, you can see what Swiftype currently has indexed in the left column, side by side with your live site offerings in the right column.

Compare your search engine index directly with your live site content with the Live Index Preview tool.

Why is this tool useful? This tool allows site owners to make changes to their site (modifying content or adding custom meta tags) and immediately see how their index will change during the next recrawl. Furthermore, from the Options drop down it’s possible to trigger instant recrawls of specific pages.

The Live Index Preview enables a much greater degree of flexibility when making changes to your website, allowing you to immediately see the impact of those changes in your search engine index without having to wait for a Swiftype recrawl to take place. To view our new meta tags tutorial, click on the link below.

Improve your site search today with these quick tips

On the modern web, search bars are everywhere. Google, the modern address bar doubling as a search bar, and the prevalence of search tools throughout the web have trained users to expect a search option when they have a clear idea of what they are looking for. This clear idea—intent—makes visitors who use search one of the most valuable online audiences.

The power of Google’s web search and of major internet players’ such as Amazon’s site search often lead to an assumption that all search solutions offer the same experience that your visitors expect. However, if you’ve used many search functions, you know this is not true. While a comprehensive, simple solution to robust site search is the best way to ensure your site doesn’t disappoint your visitors, we’ve put together a list of steps that site owners can take to improve their search experience right away.

  1. Use your search analytics. A search bar is an extremely unfiltered form of customer feedback—a free text box that asks the searcher to type in exactly what they want. This unfiltered feedback should be valued for what it is: a goldmine of analytics on your visitors. Be sure to listen to this feedback, checking for popular searches, searches that return no results, autocomplete selections where applicable, and any other information about what your visitors are looking for. You should then pipe these learnings into your marketing and product strategy. For example, an ecommerce company who sells shoes might notice that their visitors are searching for a brand they do not carry, suggesting that they should consider adding the brand.
  2. Ensure your search is ready for mobile. The share of searches of occurring on mobile websites and in mobile apps is steadily increasing. As users begin to expect their experience to be seamless across their devices, a well-designed and powerful mobile search feature is becoming as important as a desktop search solution. Good mobile search generally includes a prominent, easily accessible search bar, as navigational elements such as breadcrumbs are often difficult to display clearly for mobile visitors. Also, be sure to optimize your results page, autocomplete dropdown, and product thumbnails for the smaller screen space available to mobile users.
  3. Feature your search box prominently and intuitively. Visitors who start with search are your most valuable visitors—as much as 70% more likely to convert than those who avoid search. A search indicates a visitor came to the site for a reason, and a strong search experience moves that visitor to their desired destination quickly and painlessly. Featuring search also encourages your visitors to tell you what they are looking for, critical to an effective marketing and product strategy. To encourage searches from your visitors, make sure your search bar is prominent and intuitively located so that your visitors naturally use the feature. For inspiration, we’ve highlighted a few of our favorite search-driven site designs.
  4. Offer autocomplete results. For each additional page a visitor sees on a website, there is an associated visitor drop-off. Offering autocomplete results in a dropdown reduces the time a searcher needs to spend waiting for a results page to load. This streamlined experience will delight your visitors and reduce friction in their experience.
  5. Don’t ignore misspellings. We’ve all made typos when searching for something, either inadvertently or because we don’t know how to spell a tough word. The success of companies such as Google and Amazon in typo tolerance leads the average internet user to expect search tools to protect them from their mistakes. However, most do not come nearly close enough to Google or Amazon in typo protection, causing situations like this:
    Misspellings can cause frustrating no result pages.
    As the screenshot shows, a missed space between words—the most common typo—in a query that should have results can return none. As modern searchers expect typo protection, not accounting for misspellings can lead visitors to believe their intended query has no results, driving them away from your site. Be sure to pay attention to the most common misspellings, and take steps to ensure your visitors do not mistakenly see a no results page.

Many of these issues are best addressed by using a third-party tool that has developed easy tools for managing issues like results reordering, spellcheck, synonyms, and autocomplete. However, each of these issues represent common problems with native search tools, and each can be rapidly improved simply by paying attention to them. We’ll be featuring more tips for improving your search in the future, so be sure to check the blog often. Finally, if you are looking for a solution to each of these issues that you can get up and running quickly, sign up for a Swiftype account and create your first engine.