Full-text search with Gatsby and Decap CMS

Full-text search refers to techniques for searching a single document or a collection in a full-text database. Full-text search is distinguished from searches based on metadata or on parts of the original texts represented in databases (such as titles, abstracts, selected sections, or bibliographical references).

Full-text search allows users to retrieve higher quality results than using other methods. During the full-text search, a search engine examines all of the words in every stored document as it tries to match search criteria.

Implementing a full-text search into Gatsby is quite painless to set up. I've experimented with different plugins but decided to use Flexsearch (easiest to implement and set up).

Prerequisites

In this example, I'm presenting an implementation for Gatsby's website using Decap CMS to manage content. When using other CMS, the implementation may vary.

First, we need to install gatsby-plugin-flexsearch

$ yarn add gatsby-plugin-flexsearch

After that, we have to initialize the plugin in gatsby-config and configure which attributes we want to include.

{
      resolve: 'gatsby-plugin-flexsearch',
      options: {
        languages: ['en'],
        type: 'MarkdownRemark',
        fields: [
          {
            name: 'id',
            indexed: false,
            resolver: 'id',
            store: true,
          },
          {
            name: 'html',
            indexed: true,
            resolver: 'internal.content',
            attributes: {
              encode: 'balance',
              tokenize: 'strict',
              threshold: 0,
              resolution: 3,
              depth: 3,
            },
            store: true,
          },
          {
            name: 'title',
            indexed: true,
            resolver: 'frontmatter.title',
            attributes: {
              encode: 'extra',
              tokenize: 'full',
              threshold: 1,
              resolution: 3,
            },
            store: true,
          },
          {
            name: 'description',
            indexed: true,
            resolver: 'frontmatter.description',
            attributes: {
              encode: 'icase',
              tokenize: 'forward',
              threshold: 2,
              depth: 3,
            },
            store: true,
          },
          {
            name: 'type',
            indexed: false,
            resolver: 'frontmatter.type',
            store: true,
          },
          {
            name: 'slug',
            indexed: false,
            resolver: 'fields.slug',
            store: true,
          },
          {
            name: 'layout',
            indexed: false,
            resolver: 'frontmatter.layout',
            store: true,
          },
        ],
      },
    },

In the example above, I declared custom attributes to be indexed by flex search (id, layout, and slug) that are important for displaying results snippets and have to be available from the search component.

After restarting the development server flex search index and the store will be attached to the window object of your browser.

Search component with snippets

Right now, our Gatsby website will have flex search enabled. So far, so good! Now we have to enable users to perform searches against our content.

To do that, we need a search component! The most important part is the search function itself. If you want to check out the full component take a look at this gist

getSearchResults(query) {
    var index = window.__FLEXSEARCH__.en.index
    var store = window.__FLEXSEARCH__.en.store
    if (!query || !index) {
      return []
    } else {
      var results = []
      Object.keys(index).forEach((idx) => {
        results.push(...index[idx].values.search(query))
      })

      results = Array.from(new Set(results))

      var nodes = store
        .filter((node) => (results.includes(node.id) ? node : null))
        .map((node) => node.node)

      return nodes
    }
  }

To search our documents, we can create a simple function that takes the user's input and performs a search using the Flexsearch index and store.

First, we create an array for results. Then we can check each index for the querying user searched for and return the matched values. Then we're using the results array to filter Flexsearch store and return objects that are matching our query. The function is returning nodes containing information about documents that matched the query.

These nodes can then be used to render snippets for each search result. You can check out the full implementation on Dionysus website we worked on last month

Psst! If you're looking for a great Gatsby starter with Tailwind and Decap CMS you have to check out Henlo