The Swiftype Blog

Analytics is changing how editors and publishers build experiences for their audience online. Check out this webinar to learn why.

600x240-banner

Creating an engaging user experience is paramount to the success of digital publishing. A primary component of any engagement strategy is helping your users find content that is timely, interesting, and relevant to their interests. But how can you do that? Your first consideration should be your data and analytics.

Join Ben Lack, Senior Director of Marketing and Business Development for Swiftype, and Clare Carr, Director of Marketing for Parse.ly, as they cover the best questions to ask of your data and analytics to find out what your readers want and how to get it to them. See how top media brands and digital publishers are using the answers they find about their audience to power their editorial and product strategies.

  • What metrics really mean and what they’re telling you about your audience
  • 5 Questions about what your readers want that anyone on your team should be able to answer
  • Examples of how premium publishers are using analytics to grow their loyal audiences
  • Insights from Swiftype’s billions of search queries and Parse.ly‘s billions of network page views about what audiences want from all publishing sites.

So sign up today, then tune in on August 19 for a deep dive on how your editorial and publishing teams can improve your users online experience. Invite your colleagues to join, too!

Promoting Content Marketing with Custom Result Ranking

For any business pushing a content marketing strategy, every white paper download, webinar registration, and video view is critical to generating quality leads and achieving a positive ROI. While SEO, email marketing, and paid acquisition are all useful strategies to drive users to this content, at Swiftype we also use our own tool to promote content based on user search activity. Here’s how.

Swiftype offers site owners a unique tool to promote content for visitors who use site search: Custom Result Ranking. This tool lets site owners customize the order of search results for a specific query, dragging and dropping existing results to a new order, adding new results that don’t appear by default, or eliminating results they don’t want displayed.

Improve content marketing efforts by pinning gated content at the top of site search results.

With this tool, site owners can “pin” specific pieces of content at the top position for related searches to drive users to these high value landing pages. For example, we have pinned our white paper Understanding Ecommerce Site Search Analytics, as the top result when people search “ecommerce.” With this new result order, we see a significantly higher number of white paper downloads each week that result from site search.

Adding marketing automation landing pages
However, in many cases (including our own) these landing pages are hosted on a different domain than your main website, generated by the specific marketing automation platform your team uses (such as info.website.com, go.website.com, or many others). Because these pages exist on a separate domain, Swiftbot will not automatically discover and index these pages on the initial crawl of your website—meaning they will not initially be discoverable through search. To add these external landing pages to their search engine, Swiftype users can use the Domains tab in their Swiftype Dashboard. Let’s take a look at this process in action.

To access the Domains tab, simply click on the icon from your Dashboard home page or navigation bar. Here you can manage the domains that comprise your search engine index. For example, we combine our main site (https://swiftype.com) and our blog (http://blog.swiftype.com) in one search engine to let users search across both properties at the same time.

Add new domains to your search engine directly from the Swiftype Dashboard.

To add your landing pages, you simply have to add your landing page URL as a new domain. To do so, paste the complete URL of any landing page in the new domain box, then click verify to have Swiftbot index the page and add it to your search engine. Note that because these landing pages often don’t contain links to other pages of the same domain (which Swiftbot uses to create an index from a single url) you may see an alert warning you that Swiftbot will not be able to index additional pages beyond this one landing page. However, this will not cause any problems, and you can simply click Add Domain to proceed.

adding-url-zoom

From here, you can add additional landing pages by managing your domain rules, accessible by clicking “Manage Rules” next to the domain you just added. To add another landing page, simply click “Add URL” in the top right corner and then paste the unique URL of each landing page you’d like to add. Once these pages become part of your search engine, you can add them as a top result (or any other result) to any query you would like from the Rankings tab.

As your users interact with your on site search engine, you’ll be able to see the number of clicks these pinned results receive for specific queries in the Rankings tab, as well as the top referring queries and autocomplete prefixes that lead users to this page by looking at the Details section of each page in the Content tab.

With this tool, Swiftype allows site owners to personally curate search results, giving marketers and site owners yet another way to promote content and ultimately drive their lead generation efforts. For help with this process, or to get in touch with a member of the Swiftype team, email [email protected].

How to Index Thumbnails for Crawler Based Engines

As you’re getting started with Swiftype, you may be wondering how to index thumbnails from your website and serve them to users in your search results. The answer to this question lies in using Swiftype’s custom <meta> tags, which allow site owners to pass detailed web page information directly to Swiftbot, our web crawler, as it moves across your site. As Swiftbot encounters these custom Swiftype <meta> tags, it indexes their content and incorporates that information in your search engine index schema.

To index thumbnails from your website, all you need to do is add a Swiftype image <meta> tag to the <head> section of your website template that indicates where images are located on your various page types. For illustration purposes, the Swiftype image <meta> tag is formatted like this:


Swiftype recommends placing these <meta> tags at the template level of your website to ensure that image files are dynamically populated within the tags, rather than being added manually for every page on site.

NOTE: the value of the “content” attribute must be HTML encoded. For more information see this guide.

Alternatively, you can wrap images with a body-embedded Swiftype image <meta> tag to avoid changing your website <head>. For example, Swiftbot will index example.jpg into the image field from the HTML below:

<body>

Hello world

 

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed ut risus sed ante dignissim pharetra aliquet a orci. Maecenas varius.

 

In in augue molestie, bibendum velit vel, luctus erat. Curabitur cursus, tellus at feugiat lacinia, tellus est suscipit lectus, non commodo diam elit sit amet justo.

http://fullurl.com/example.jpg </body>

It is important to note that in both the <head> and <body> embedded <meta> tags, you need to specify the data-type attribute as enum. For images, this will always be the case. For any other custom meta tags you choose to define, each attribute must be a valid, Swiftype-supported field type, which you may read about here.

Once you index thumbnails from your website, you can easily customize your search results and autocomplete to feature thumbnails in a range of shapes and sizes with the Swiftype Result Designer.

To learn more about using custom Swiftype <meta> tags to refine your search engine index, check out our tutorial. As always, if you need help or have any questions, feel free to reach out to us at [email protected].

The Evolution of Search on Amazon

This post was initially published by Internet Retailer—a leading source of ecommerce news and publications.

One of my favorite tools for online research is the Internet Archive—a 501(c)(3) non-profit devoted to the digital preservation of the web as it evolves, crawling the web and capturing how various web pages have looked at different points in time. As someone who spends time researching and writing about web design, I’ve relied on this tool to trace the evolution of numerous websites—Amazon.com being one of the most fascinating websites to study in light of its ever increasing power and influence over the web, particularly the realm of ecommerce.

While Amazon’s homepage design has evolved in response to a wide range of influences and company goals, one of the most striking changes has been the increasingly prominent role that the search bar plays in the user experience. Viewed in the context of the internet’s larger history, this evolution makes logical sense: Google and Amazon were both founded around the same time (1994 and 1998, respectively), and while Google has made more of a name for itself as the innovator in search technologies, search technology at Amazon has had to evolve alongside Google because of the tremendously important role it plays in ecommerce.

Considering the tremendous strides that Google has made in search technology over the last fifteen years, it should come as no surprise that Amazon has had to work equally as hard to keep pace (today still, Amazon has an entire section of its job site devoted to Search and Discovery Technologies positions). As Amazon’s search has bar has become more useful and more powerful, it has become increasingly valuable to users. And the more valuable it has become, the more devoted real estate it has received across the homepage and website.

Let’s trace this evolution with a series of screenshots from Amazon’s homepage over the last fifteen years, focusing on the changing size and position of the search bar.

May 10, 2000: Search begins somewhat obscurely tucked in the top left corner.

May 10, 2000: Search begins somewhat obscurely tucked in the top left corner. Amazon provides the option to filter searches to a specific subsection of the site and a suggested query to inspire users, but the sidebar below the search bar remains a primary navigation tool. This design remains largely unchanged for the next six years.

 June 2, 2007: Search occupies a much more prominent space in the header of the page, but the sidebar remains equally prominent as a critical navigation tool.

June 2, 2007: Search occupies a much more prominent space in the header of the page, but the sidebar remains equally prominent as a critical navigation tool.

June 15, 2012: Amazon eliminates the sidebar, offering instead a single search bar to serve as the primary portal for users entering the site.

June 15, 2012: Amazon eliminates the sidebar, offering instead a single search bar to serve as the primary portal for users entering the site.

June 9, 2015: Not only does search remain the most prominent feature in the header of Amazon’s homepage, but the search bar now sticks to the top of the page as shoppers scroll.

June 9, 2015: Not only does search remain the most prominent feature in the header of Amazon’s homepage, but the search bar now sticks to the top of the page as shoppers scroll.

The trend across these images is clear. As search technology has improved on Amazon, the search bar has become a more valuable asset in the user experience and has accordingly taken a more prominent position the homepage. At the same time, with the dominance that Google exercises across the web, user expectations for search have steadily risen as well—a development which undoubtedly contributed to Amazon’s shift towards such a search-centric user experience.

Another major takeaway from this evolution is the massive amount of data that Amazon is able to collect as a result of their massive search volume. On site search queries are arguably the clearest expression of user intent available to site owners, and Amazon continually leverages this information to intelligently promote products across their website—a subject and design evolution that merits its own post entirely.

To learn more about best practices for site search design and see example implementations, read our post on Swiftype’s favorite implementations from 2014.

Styling Search Without Code: The Swiftype Result Designer

Since our founding in 2012, Swiftype has worked hard to let site owners install powerful, user-friendly search on their website without writing code. To take this vision even further, we are excited to announce a new feature this week: The Swiftype Result Designer.

blog-post-desk-dash-mobile

The Swiftype Result Designer allows site owners to style their search results and autocomplete directly from the Swiftype dashboard, making it easy to integrate Swiftype search with the existing look and feel of your website. For a brief overview of how the Swiftype Result Designer works, watch the screencast below.

With the new Result Designer, Swiftype users can

  • Choose between multiple search result display methods
  • Add thumbnails to search results and autocomplete suggestions
  • Decide which page attributes to display in search results
  • Customize search result text and snippet highlighting colors
  • Change their search result and autocomplete design as frequently as they would like, without ever touching their code base

blog-post-color

For new Swiftype users, the Result Designer can be accessed under the INTEGRATE tab, on the INSTALL SEARCH page. For Swiftype users who created their engines before this new installation process was launched in mid-April and would like to transition to this new javascript embed, follow the instructions on the INSTALL page in your Swiftype dashboard or visit our tutorial here.

This is the first in a series of exciting updates that the Swiftype team is making to the installation process. Stay tuned for more updates over the next few months, including new tools to custom style your own facets, filters, and sorting tools—all without writing a single line of code.

Try out the Result Designer today by creating a free account.

Swiftype Shopify Search App Update

We’ve made some exciting new changes to our Shopify Swiftype app that give store owners an unprecedented level of control over how their search results and autocomplete drop-down menu appear to users.

Before these changes, the Swiftype Shopify app offered limited control over how search results appeared within your selected theme. However, after releasing our second edition of the Shopify app last week, the default result display mode is highly flexible and Shopify store owners can even fine tune the look and feel of their search experience through the Swiftype dashboard. Let’s explore these new changes in detail.

Installation

Installing the Swiftype app on your Shopify store works much the same as it always has. Simply visit the Swiftype app listing, click “Get,” and enter your Shopify store URL.

app listing

From here, Swiftype will automatically index your products and create a search engine for you. Depending on the size of your product set, this process can take a few minutes. Once complete, you will be directed to the Swiftype dashboard, where you can customize the look and feel of search (details below), gather detailed analytics on customer search behavior, and fine tune your search results to promote specific products. Let’s explore these features in turn, starting with the autocomplete and search result styling process.

Styling the search results and autocomplete

To customize the style of your search results and autocomplete drop-down, go to the “INSTALL SEARCH” page under the “INTEGRATE” tab. On this page, click “START INSTALLATION” to get started. For Shopify app users, you will only need to make changes on the “APPEARANCE” section of the installation process. In this section, you can control

  • how search results look
  • how many search results appear on the results page
  • the color of the search results text
  • the style and color of the autocomplete drop-down menu

Specific instructions for all of these pieces can be found within the “APPEARANCE” section of the install page.

result-designer-blog-width

Once you have configured the search results, autocomplete, and the color of the results text, hit “SAVE & PREVIEW” on the “COLORS” page. To preview your changes before activating them on your site, simply type in a query and hit enter or click “SEARCH.” The styling you selected will preview directly in the dashboard, which you can go back and edit if you’re not happy with the appearance.

preview-blog-width

Once you are satisfied with the appearance of search results and autocomplete, click “SAVE.” This will navigate you to to the “activate” section where you can review your configuration before clicking “ACTIVATE SWIFTYPE” to push these changes to your live site.

Swiftype Dashboard: Analytics and customization

Once installed on your Shopify store, Swiftype collects detailed analytics about your website search traffic, the most popular searches on your site, what users are searching for but not finding, and more. Learn more >

Leverage detailed analytics to customize search on your Shopify store.

Leverage detailed analytics to customize search for your website.

With this detailed information in hand, you can customize search results to drive more sales on your Shopify store. Swiftype users have three tools to customize search results:

  1. RESULT RANKINGS allows you to drag and drop to reorder search results for individual queries. You can also add in product results that don’t appear by default or eliminate products that you don’t want displayed for that query. Swiftype recommends you customize results for the top ten or twenty queries to fine tune results for popular searches. It is also a good idea to customize results for top queries which return no results, so users hit less dead ends on your site. Learn more >
  2. WEIGHTS allow you to fine tune your relevance model and control how various attributes in your Shopify index impact search results. This tool can be used to promote products with high ratings, high popularity, or any other trait you choose. Learn more >
  3. SYNONYMS allow you to synchronize search results for different words of the same meaning. This is especially useful for stores with high volumes of international traffic, since shoppers from different countries often use different words for the same product. Learn more >

For help installing Swiftype on your Shopify store, email [email protected].

Building an Asynchronous API to Improve Performance

One of the challenges we’ve had to deal with at Swiftype is that we have had customers pushing a lot of search and indexing traffic from very early on. When a customer is pushing hundreds of index updates per second, it’s important to respond quickly so we don’t start dropping requests.

In order to do that, we’ve built bulk create/update API endpoints to reduce the number of HTTP requests required to index a batch of documents and moved most processing out of the web request. We’ve also invested in front-end routing technology to limit the impact customers have on each other.

However, we were not satisfied. Sometimes when a large customer was indexing a huge number of documents, our front-end queues would still back up.  In the pursuit of even better response times for our customers, we’ve built an asynchronous indexing API. Our goals in creating the new API were high throughput, supporting bulk requests for all interactions, and excellent developer ergonomics. We wanted an API that was fast and easy to use.

Here’s how it works.

async_bulk_API_vertical_2.28.39_PM

First, customers submit a batch of documents to create or update. The request for this looks just like our pre-existing bulk create or update API, but goes to a new endpoint.

When our server receives the response, it performs a quick sanity check on the input, without hitting the database. If all the input parameters are present and validly formatted, we create two records in our database for each document that was submitted: a document creation journal, and a document receipt.

For performance, we insert these rows using activerecord-import. This is a great library that uses a single INSERT statement with multiple rows. This results in a massive speed improvement compared to standard ActiveRecord when saving a large number of records. We also generate the IDs ahead of time using BSON. By generating the IDs ahead of time, we don’t need to get them from the database after inserting, and using BSON lets us encode a timestamp in the ID at the cost of a larger ID column.

Once created, we enqueue a message for each document creation journal onto a queue that is read by a pool of loops workers. Loops is a dead-simple background processing library written by our Technical Operations Lead, Oleksiy Kovyrin. It makes it easy to write code that does one thing forever, in this case, reading messages off the queue and creating the associated document in the database.

The response to the API request includes a way to check the status of all the document receipts. To make the API easy to use, we’re including URLs to the created resources. Though we’re not following all its precepts, this approach is inspired by the idea of the hypermedia API. These URLs make it easy for both humans and computers to find the resource.

Since the API is asynchronous, users must poll the document receipts API to check for the status of the document creation. We’ve built an abstraction in our Ruby client library that allows developers to simulate a synchronous request, although we recommend that only for development.

By pushing all work except for JSON parsing and the most minimal input validation to the backend, we’re able to respond to these API requests very quickly. On the backend, the loops workers read messages off the queue and create documents. When a loops worker attempts to create a document, it updates the document receipt (either with the status of “complete” and a link to the created/updated document, or with the status “failed” and a list of error messages) and deletes the document creation journal.

This brings us to one final aspect of the asynchronous API: how we make sure it keeps working. If our loops workers started failing, the document creation journals would back up without being processed, and no documents would be created/updated. To guard against this, we have built a monitoring framework that alerts us when the oldest record in the table is older than a certain threshold.

This solution has been successful for us in beta tests with our largest API users, and we have now rolled it out to everyone.

We hope this helps you build out your next high-throughput API. If this is the kind of thing you’re interested in, we’re hiring engineers for our core product and infrastructure teams.

Swiftype Announces $13 million Series B Funding Round

Today we’re excited to announce that we’ve raised a $13 million Series B financing led by our investors at NEA. We’ve actually worked with NEA since our company was only a few weeks old — first as a part of the Seed round we raised during Y Combinator and again when they led our Series A in September of 2013 — and we’re delighted to further our partnership with them today. We’re also excited to officially welcome NEA Partner Chetan Puttagunta to the Swiftype board.

Swiftype-Office

Swiftype has come a long way since graduating from Y Combinator in March of 2012. Quin and I started the company alone in an apartment in Mountain View, CA, adding Luke Francl and Oleksiy Kovyrin as quickly as we could convince them, and then the team stayed that size for a while. But today we’re a team of over 25 incredibly talented individuals, and our business has grown similarly. Today we power billions of queries every month across hundreds of thousands of websites, mobile applications, and SaaS applications. Along the way we’ve had the privilege of working with many fantastic companies, including Qualcomm, CloudFlare, Shopify, and Twitch. It has taken a tremendous amount of dedication — and a lot of hard work — from our entire team to get to where we are today, and I couldn’t be more proud of the progress we’ve made so far.

Looking ahead, these new funds will fuel our continued growth as we look to bring on more search engineers, expand our sales and marketing efforts, and establish a stronger position of leadership within the search industry.

If you’re interested in joining a quickly growing, fast-paced organization, we encourage you to check out our jobs page. We’re always excited to meet new people who are interested in joining the Swiftype team, and we’d love to hear how you can contribute.

Swiftype for Zendesk

Zendesk_logo_RGB

As we’ve written before, search is a critical tool for self-service support centers. Users who enter the documentation or support section of a website generally have a clearly defined issue in mind, and a powerful search bar provides these users with a clear path to the content they are looking for.

To help more support centers bring great search to their users, Swiftype is excited to announce our integration with Zendesk Help Centers, making it easy for knowledge bases built with Zendesk to deliver the search experience that customers expect. With Swiftype powering search on your Zendesk Help Center, site owners have the power to customize results for individual queries, ensuring that users always find the most helpful content. Furthermore, Swiftype provides detailed analytics on what your users are searching for, providing key insight into user issues and identifying content gaps by populating a list of top queries that return no results.

With these tools and more, site owners can redefine their Help Center user experience by providing powerful and customizable search without the need to consult engineering. Get started by creating account today.

New feature: Live Index Preview tool

Today we are excited to announce a new dashboard feature for site owners with crawler based engines: the Live Index Preview tool. Some of you may have noticed this feature since we rolled it out a month ago, but we announce it today in conjunction with our new meta tags tutorial, which walks site owners through the steps required to implement Swiftype meta tags on your website template.

Now when you log into your Swiftype account and visit the Content tab, you have the ability to compare the live state of your site’s content with what Swiftype presently has indexed for your search engine. To do this, click on the Details button for a page document from your main Content tab view, or look up a specific URL from the search box.

On the Details page, switch from the indexed page overview to the live view by clicking on the bolt icon. Once you switch to live view, you can see what Swiftype currently has indexed in the left column, side by side with your live site offerings in the right column.

Compare your search engine index directly with your live site content with the Live Index Preview tool.

Why is this tool useful? This tool allows site owners to make changes to their site (modifying content or adding custom meta tags) and immediately see how their index will change during the next recrawl. Furthermore, from the Options drop down it’s possible to trigger instant recrawls of specific pages.

The Live Index Preview enables a much greater degree of flexibility when making changes to your website, allowing you to immediately see the impact of those changes in your search engine index without having to wait for a Swiftype recrawl to take place. To view our new meta tags tutorial, click on the link below.

Subscribe to our blog