Vendure Advanced Search Plugin

This plugin enables lightning-fast, memory-efficient search backed by Typesense, as well as powerful search analytics.

Features

Typo tolerance: Handle spelling mistakes with configurable typo tolerance.
Tunable weightings: Define what relative weight to give title, description and SKU.
Geopoint search & filtering: search by distance from a given location.
Result pinning: Pin specific results to a particular position e.g. for merchandising.
Synonyms: Define synonyms to increase accuracy of search results
Custom search indexes: index any dataset into Typesense and expose via Vendure APIs.
Advanced analytics giving insights into popular searches, click-through rate, result rate etc.

Setup

You’ll need an instance of Typesense. E.g. using the official Docker image:
```
docker pull typesense/typesense:0.22.2
```
Or you can set up an account with Typesense Cloud for a hosted solution.
For the best support for search analytics, we strongly recommend Clickhouse:
```
docker pull yandex/clickhouse-server:21-alpine
```
If you do not want to use Clickhouse, we also support SQL-based analytics which will store the analytics data in your database. It is also possible to use this plugin without analytics - see section below.

Add the plugin to your VendureConfig:

import { AdvancedSearchPlugin, ClickhouseAnalyticsStrategy, SqlAnalyticsStrategy } from '@vendure-plus/advanced-search-plugin';

export const config = {
  //...
  plugins: [
    AdvancedSearchPlugin.init({
      typeSenseClientOptions: {
        apiKey: process.env.TYPESENSE_API_KEY as string,
        nodes: [
        {
          host: process.env.TYPESENSE_HOST as string,
          port: 8108,
          protocol: 'http',
        },
        ],
      },
      analytics: {
        analyticsStrategy: process.env.USE_CLICKHOUSE
          ? new ClickhouseAnalyticsStrategy({
              url: process.env.CLICKHOUSE_URL as string,
              port: 8123,
              database: 'vendure_search_analytics',
              debug: false,
            })
          : new SqlAnalyticsStrategy(),
      },
      bufferUpdates: true,
    }),
  ],
};

If not already installed, install the @vendure/ui-devkit package.
```
npm install @vendure/ui-devkit
```

Add the UI extensions. This plugin requires the ui-devkit in order to compile a custom Admin UI including the advanced search extensions. See our guide to extending the Admin UI.

import { compileUiExtensions } from '@vendure/ui-devkit/compiler';
import { AdvancedSearchPlugin } from '@vendure-plus/advanced-search-plugin';

// ...
plugins: [
  AdminUiPlugin.init({
    route: 'admin',
    port: 3002,
    app: compileUiExtensions({
      outputPath: path.join(__dirname, '../admin-ui'),
      extensions: [AdvancedSearchPlugin.uiExtensions],
      devMode: false,
    })
  }),
],

Disabling Analytics

If you do not need the analytics features, they can be disabled with the following configuration (since v1.4.0):

import { AdvancedSearchPlugin } from '@vendure-plus/advanced-search-plugin';

export const config = {
   //...
   plugins: [
      AdvancedSearchPlugin.init({
         typeSenseClientOptions: {
            // ...
         },
         // Set to false to disable analytics on the server
         analytics: false,
         bufferUpdates: true,
      }),
      AdminUiPlugin.init({
         route: 'admin',
         port: 3002,
         app: compileUiExtensions({
            outputPath: path.join(__dirname, '../admin-ui'),
            // Use the `uiExtensionsNoAnalytics` variant to remove the "analytics" menu item
            // from the Admin UI.
            extensions: [AdvancedSearchPlugin.uiExtensionsNoAnalytics],
            devMode: false,
         })
      }),
   ],
};

Shop API

The same search query is used to search in the Shop API, but the input object is extended as follows:

extend input SearchInput {
    "Whether to log this search in the analytics store. If true, the result will include a queryId"
    logAnalytics: Boolean
    "Whether to apply curations to the results"
    applyCurations: Boolean
    maxFacetValues: Int
    "Allows filtering by stock status"
    inStock: Boolean
    "Allows filtering by price range"
    priceRange: PriceRangeInput
    "Allows filtering by price range (including taxes)"
    priceRangeWithTax: PriceRangeInput
    "If true, will use the search term as a prefix. Intended in live search (autocomplete) use cases."
    prefixMode: Boolean
    "Sample to top collections from the first n results, defaults to 10"
    topCollectionsFromTop: Int
    "Supply custom Typesense filter strings to apply to the search"
    filterBy: [String!]
    "Allows filtering by facetCode and facetValueCodes to get a subset of facets in the facets object of the response to easily further refine the search"
    facetsFilter: FacetsFilterInput
}

The search response is also extended to include data on top facets, top collections,

type ResultHighlight {
  field: String!
  matchedTokens: [String!]!
  snippet: String!
}
type FacetCountItem {
  count: Int!
  highlighted: String!
  value: String!
}
type FacetCountStats {
  avg: Float!
  min: Float!
  max: Float!
  sum: Float!
}
type FacetCountData {
  fieldName: String!
  counts: [FacetCountItem!]!
  stats: FacetCountStats
}
type SearchResponsePriceData {
  range: PriceRange!
  rangeWithTax: PriceRange!
}
type GeoDistance {
  field: String!
  distanceInMeters: Float!
}
extend type SearchResult {
  id: String!
  inStock: Boolean!
  collectionSlugs: [String!]!
  highlights: [ResultHighlight!]!
  geoDistance: [GeoDistance!]
}
type TopCollectionResult {
  collection: Collection!
  score: Int!
}
extend type SearchResponse {
    queryId: String
    facets: [FacetResult!]!
    facetCounts: [FacetCountData!]!
    prices: SearchResponsePriceData!
    topCollections: [TopCollectionResult]!
}
type FacetResult {
    facet: Facet!
    facetValueCounts: [FacetValueResult!]!
}

Custom Mappings

Custom mappings allow you to index arbitrary data alongside your product variants.

AdvancedSearchPlugin.init({
  // ...
  customMappings: {
    reviewRating: {
      graphQlType: 'Float',
      valueFn: ({ variant }) => variant.product.customFields.reviewRating,
    },
    reviewCount: {
      graphQlType: 'Int!',
      valueFn: ({ variant }) => variant.product.customFields.reviewCount,
    },
    facetValues: {
      graphQlType: 'String!',
      searchable: true,
      hydrateRelations: ['product.facetValues'],
      valueFn: ({ variant, languageCode }) =>
        variant.product.facetValues
          .map((fv) => translateDeep(fv, languageCode).name)
          .join(' '),
    },
    featuredAssetName: {
      graphQlType: 'String!',
      hydrateRelations: ['product.featuredAsset'],
      valueFn: ({ variant }) => variant.product.featuredAsset?.name ?? '',
      searchable: true,
    },
    discountPercentage: {
      graphQlType: 'Float',
      valueFn: ({ variant }) => variant.customFields?.discountPercentage,
      outputFn: (values: number[], groupByProduct: boolean) => {
         if (groupByProduct) {
            return Math.max(...values);
         } else {
            return values[0];
         }
      },
    },
  },
}),

Primitive & Object custom mappings

Primitive custom mappings return a primitive value (a GraphQL scalar or list of scalars such as Int or [String!]). The examples above are of this type, and these are suitable for most cases.

Sometimes, however, you may want to return a more complex type from a custom mapping. In this case, you can use an object custom mapping, which can return any kind of GraphQL object type.

An object custom mapping must:

specify a typesenseType which is one of the Typesense field types. This is required because we cannot infer the Typesense type from the GraphQL type, since the GraphQL type can be anything.
specify an outputFn which has the task of transforming the data stored in Typesense into the correct shape according to the chosen graphQlType. The values it receives will be an array, and in the case that the search has been performed with groupByProduct: true, it will contain all the values for each variant in the given product. The groupByProduct argument will be true if the search was performed with groupByProduct: true, otherwise it will be false.
If a GraphQL type has been used which does not yet exist in your schema, you can define it using the graphQlSchemaExtension property.

In this example, we have defined a custom rrp (recommended retail price) field on the Product, and we want to expose it as a min/max range:

AdvancedSearchPlugin.init({
    // ...
    customMappings: {
      rrp: {
        graphQlType: 'RRPRange',
        graphQlSchemaExtension: `
          type RRPRange {
            min: Int!
            max: Int!
          }
        `,
        typesenseType: 'int32',
        valueFn: ({variant}) => variant.customFields?.rrp,
        outputFn: (values, groupByProduct) => {
          return {
            min: Math.min(...values),
            max: Math.max(...values),
          };
        },
      },
    },
}),

Sortable Fields

By default, the search query results can be sorted by price or name. It is also possible to define extra fields by which to sort, by using the sortableFields option.

AdvancedSearchPlugin.init({
  // ...
  sortableFields: [
    // Here we are specifying that the 'sku' field
    // should also be sortable
    { name: 'sku' },
    // Assuming we have defined a customMapping for reviewRating
    // (as in the example above), we can also make this sortable.
    {
      // This name must match the field name internally in TypeSense,
      // which, for customMappings is always of the format
      // `customMapping_<name>`.
      name: 'customMapping_reviewRating',
      // We can alias this with a more friendly name that will be used
      // in the GraphQL API "sort" input.
      alias: 'rating'
    }
  ],
});

With the above configuration, we will then be able to perform queries such as:

query {
  search(input: {
    sort: {
      rating: DESC
    }
  }) {
    totalItems
    items {
      productName
      customMappings {
        reviewRating
      }
      # ...
    }
  }
}

Filtering

Filtering of the search results can be done via the built-in filter inputs of the SearchInput object such as collectionSlug, facetValueFilters, inStock, priceRangeWithTax. For even more control over filtering you can use the filterBy input to provide expressions directly as Typesense filter_by parameters.

Custom mappings can be filtered by prepending the name with customMapping_:

query {
  search(input: {
    groupByProduct: true,
    filterBy: [
      "inStock: true",
      "collectionIds:= [`345`, `525`]",
      "customMapping_reviewRating:[2..5]"
    ],
  }) {
    items {
      inStock
      productName
      customMappings {
        reviewRating
      }
    }
  }
}

Faceted Search

Faceted search is a powerful feature that allows users to refine search results by applying multiple filters. This can be implemented using the facetsFilter argument of the SearchInput object. This argument accepts the code of the facet and the codes of the selected facet values, and returns the updated facets with the remaining counts of their facet values in the facets property of the SearchResponse object.

By setting includeNullValues: true in the facetsFilter argument, the response will also include facet values with a count of zero.

Here’s an example of a GraphQL query implementing a faceted search:

query {
    search(input: {
      facetsFilter: {
        includeNullValues: true,
        facets: [
          {
            code: "category",
            facetValueCodes: ["footwear"]
          },
          {
            code: "brand",
            facetValueCodes: ["nike", "adidas"]
          }
        ]
      }
    }) {
      totalItems
      items {
        productName
        productId
      }
      facets {
        facet {
          code
        }
        facetValueCounts {
          count
          facetValue {
            id
            name
          }
        }
      }
    }
}

This query will return a list of products filtered by the selected facets, along with the total number of items and the updated facets with their counts.

Document Overrides

It is possible to override the indexed value of the built-in fields on the SearchResult type. For example, let’s say you have a customField which stores a “sale price” which you want to use in your search results. Here’s how you can configure it:

AdvancedSearchPlugin.init({
  // ...
  documentOverrides: {
    priceWithTax: ({ variant }) => variant.customFields.salePrice,
  },
}),

The override function will receive an object as it’s single argument which looks like this:

export interface DocumentContext {
  ctx: RequestContext;
  languageCode: LanguageCode;
  channel: Channel;
  variant: ProductVariant;
  productTranslation: Translation<Product>;
  variantTranslation: Translation<ProductVariant>;
  optionTranslations: Array<Translation<ProductOption>>;
  collectionTranslations: Array<Translation<Collection>>;
}

Geosearch

Typesense supports powerful geosearch capabilities, which you use to sort results based on distance from a given point. Here’s an example of how this would work.

First, we need to store coordinates somewhere. In this example, we will store them in a custom fields on the Product entity:

import { VendureConfig } from '@vendure/core';

export const config: VendureConfig = {
  // ...
  customFields: {
    Product: [
      {
        name: 'latitude',
        type: 'float',
        defaultValue: 0,
      },
      {
        name: 'longitude',
        type: 'float',
        defaultValue: 0,
      },
    ],
  },
};

Next, we need to define a custom mapping for the latitude and longitude fields:

AdvancedSearchPlugin.init({
  // ...
  customMappings: {
    location: {
      graphQlType: 'Geopoint',
      valueFn: ({ variant }) => {
        // A "Geopoint" type will expect a tuple of [latitude, longitude]
        return [
          // You can avoid "as any" by typing your custom fields,
          // see https://www.vendure.io/docs/developer-guide/customizing-models/#typescript-typings
          (variant.product.customFields as any).latitude,
          (variant.product.customFields as any).longitude,
        ] as [number, number];
      },
    },
  },
  sortableFields: [
    {
      // By making this customMapping sortable, we can sort results by distance
      name: 'customMapping_location',
      alias: 'location',
    },
  ],
}),

Finally, we can use the sort input to sort by distance from a given point, filter by distance from that point, and use the geoDistance field to get the distance in meters:

query {
  search(input: {
    groupByProduct: true,
    sort: {
      location: {
        latitude: 48.205809,
        longitude: 16.366315,
        sort: ASC
      }
    }
    filterBy: ["customMapping_location:(48.205809, 16.366315, 1.5 km)"],
  }) {
    items {
      productName
      customMappings {
        location { latitude longitude }
      }
      geoDistance {
        field
        distanceInMeters
      }
    }
  }
}

External Indexes

An external index allows you to index and search any dataset alongside your product data. This data can come from any source, such as within Vendure itself or from an external system.

Example: Indexing collections

Here’s an example where we index Vendure collections, so we can display them in the search results:

import { ExternalIndex, TypesenseDocument } from '@vendure-plus/advanced-search-plugin';
import {
   ChannelService,
   Collection,
   CollectionEvent,
   CollectionService,
   EventBus,
   RequestContextService,
   TransactionalConnection,
} from '@vendure/core';
import { filter, map} from 'rxjs/operators';

interface CollectionDocument extends TypesenseDocument {
   name: string;
   slug: string;
   breadcrumb: string[];
   rank: number;
}

export const collectionExternalIndex = new ExternalIndex<CollectionDocument>({
   name: 'collections',
   getAllIds: async (injector) => {
      const connection = injector.get(TransactionalConnection);
      return connection.rawConnection
              .getRepository(Collection)
              .createQueryBuilder('collection')
              .select('collection.id')
              .where('collection.isPrivate = :isPrivate', {isPrivate: false})
              .getMany()
              .then((collections) => collections.map((c) => c.id));
   },
   fields: {
      id: {
         type: 'string',
         facet: false,
      },
      name: {
         type: 'string',
         facet: false,
      },
      slug: {
         type: 'string',
         facet: false,
      },
      breadcrumb: {
         type: 'string[]',
         facet: false,
      },
      rank: {
         type: 'int32',
         isDefaultSortingField: true,
         public: true,
         facet: false,
      },
   },
   createDocuments: async (ctx, injector, ids) => {
      const channelService = injector.get(ChannelService);
      const requestContextService = injector.get(RequestContextService);
      const collectionService = injector.get(CollectionService);
      const defaultChannelCtx = await requestContextService.create({
         apiType: 'admin',
         channelOrToken: await channelService.getDefaultChannel(ctx),
      });
      const collections = await collectionService.findByIds(defaultChannelCtx, ids);
      return Promise.all(
              collections
                      .filter((collection) => !collection.isPrivate)
                      .map(async (collection) => {
                         const breadcrumb = await collectionService
                                 .getBreadcrumbs(defaultChannelCtx, collection)
                                 .then((result) => result.slice(1).map((b) => b.name));
                         return {
                            id: collection.id.toString(),
                            name: collection.name,
                            slug: collection.slug,
                            breadcrumb,
                            // We weight the rank by the depth of the 
                            // collection in the tree
                            rank: 10 - breadcrumb.length,
                         };
                      }),
      );
   },
   createTypesenseSearchParams: (ctx, injector, input) => {
      const per_page = input.take ?? 10;
      const page = input.skip && 0 < input?.skip ? Math.ceil(input.skip / per_page) + 1 : undefined;
      return {
         q: input.term,
         query_by: ['name', 'slug'],
         prefix: input.prefixMode ?? false,
         sort_by: 'rank:desc,_text_match:desc',
         // We can support arbitrary Typesense filter expressions by
         // using the `filter_by` parameter and passing it any input
         // which has been provided in the `filterBy` input field
         filter_by: input.filterBy?.join(' && '),
         per_page,
         page,
      };
   },
   updateStream: (injector) => {
      const eventBus = injector.get(EventBus);
      return eventBus.ofType(CollectionEvent).pipe(
              filter((event) => event.type !== 'deleted'),
              map((event) => [event.entity.id]),
      );
   },
   removeStream: (injector) => {
      const eventBus = injector.get(EventBus);
      return eventBus.ofType(CollectionEvent).pipe(
              filter((event) => event.type === 'deleted'),
              map((event) => [event.entity.id]),
      );
   },
});

Example: Indexing blog posts

As another example, you might want to index blog posts or information pages. This is done by defining an ExternalIndex instance. Here’s an example which indexes articles from a CMS:

import { ExternalIndex, TypesenseDocument } from '@vendure-plus/plugin-advanced-search';
import { EventBus, TransactionalConnection } from '@vendure/core';
import { Observable } from 'rxjs';
// Some imaginary SDK from our CMS vendor
import { cmsSDK } from 'cms-sdk';

interface CmsArticleDocument extends TypesenseDocument {
  title: string;
  slug: string;
  breadcrumb: string[];
}

export const cmsExternalIndex = new ExternalIndex<CmsArticleDocument>({
   // A unique name to identify this index. It is used when searching via the
   // `searchExternal` GraphQL query.
   name: 'cms-articles',
   
   // Since v1.6.0, the plugin will dynamically generate a new query for searching
   // this index. By default, it will be `search<Name>`, where `Name` is the name
   // of the index converted to PascalCase. So in this example the default query
   // name would be `searchCmsArticles`. If you want to override this, you can
   // specify the `queryName` property.
   queryName: 'searchArticles',
   
   // This defines the fields that will be indexed in Typesense.
   fields: {
      id: {
         type: 'string',
         facet: false,
      },
      title: {
         type: 'string',
         facet: false,
      },
      slug: {
         type: 'string',
         facet: false,
      },
      breadcrumb: {
         type: 'string[]',
         facet: false,
      },
   },
   // This method should return the ids of _all_ documents from the external
   // system. This is used in doing a full reindex operation.
   getAllIds: async (injector) => {
      return cmsSDK.fetchAllIds();
   },
   // When given an array of IDs, this function fetches the necessary
   // data to populate a CmsArticleDocument in the search index and
   // returns those documents.
   createDocuments: async (ctx, injector, ids) => {
      const articles = await cmsSDK.findPostsByIds(ids);
      return articles.map(article => {
         return {
            id: article.id,
            title: article.title,
            slug: article.slug,
            breadcrumb: article.breadcrumb,
         };
      });
   },
   // This is used to specify how the search term and pagination arguments
   // are converted to an object to pass to Typesese. In general, only the `query_by`
   // should require changes to point to the field(s) that are to be searched.
   createTypesenseSearchParams: (ctx, injector, input) => {
      const per_page = input.take ?? 10;
      const page = input.skip && 0 < input?.skip ? Math.ceil(input.skip / per_page) + 1 : undefined;
      return {
         q: input.term,
         query_by: ['title'],
         prefix: input.prefixMode ?? false,
         per_page,
         page,
      };
   },
   // This method should return an Rxjs observable stream of IDs
   // which emits a value whenever the external source is updated
   // and requires reindexing.
   updateStream: (injector) => {
      return new Observable((subscriber) => {
         cmsSDK.on('update', article => {
            subscriber.next([article.id]);
         })
      });
   },
   // This method should return an Rxjs observable stream of IDs
   // which emits a value whenever the external source is deleted
   // and requires removing from the index.
   removeStream: (injector) => {
      return new Observable((subscriber) => {
         cmsSDK.on('delete', article => {
            subscriber.next([article.id]);
         })
      });
   },
});

You then pass this ExternalIndex instance to the AdvancedSearchPlugin:

import { VendureConfig } from '@vendure/core';
import { AdvancedSearchPlugin } from '@vendure-plus/advanced-search-plugin';

import { cmsExternalIndex } from './cms-external-index';
import { collectionExternalIndex } from './collection-external-index';

export const config: VendureConfig = {
  // ...
  plugins: [
    AdvancedSearchPlugin.init({
      // ...
      externalIndexes: [collectionExternalIndex, cmsExternalIndex],
    }),
  ],
};

Searching external indexes

For each external index defined, the plugin will automatically generate a new query in the GraphQL API. The name of the query will be search<Name>, where Name is the name of the index converted to PascalCase. This can be overridden by specifying the queryName property when defining the ExternalIndex.

query SearchCollections($input: ExternalSearchScopedInput!) {
   searchCollections(input: $input) {
      items {
         slug
         name
         breadcrumb
         highlights {
            field
            snippet
         }
      }
   }
}

{
   "term": "foo",
   "skip": 0,
   "take": 5,
   "prefixMode": true,
   "filterBy": ["rank:>8"]
}

Searching multiple external indexes

It is also possible to search multiple external indexes in a single query using the searchExternal query. The response will be a union type of all the external indexes you have defined. The plugin will automatically create a GraphQL type for your external index, which will be named as a PascalCase version of the name property with a Response suffix. In the example above, the name is 'cms-articles', so the resulting GraphQL type would be CmsArticlesResponse.

query SearchExternal($externalInput: ExternalSearchInput!) {
   searchExternal(input: $externalInput) {
      ... on CollectionsResponse {
         items {
            breadcrumb
            highlights {
               field
               snippet
            }
            name
            slug
         }
      }
      ... on CmsArticlesResponse {
         items {
            breadcrumb
            highlights {
               field
               snippet
            }
            title
            slug
         }
      }
   }
}

The ExternalSearchInput object must then specify the name of the external index(es) to search:

{
   "term": "foo",
   "skip": 0,
   "take": 5,
   "prefixMode": true,
   "indexes": ["collections", "cms-articles"],
   "filterBy": {
      "collections": ["rank:>8"]
   }
}

In this example we are also providing some custom filterBy data, which allows your external index to support arbitrary filtering expressions supported by Typesense.

Search Analytics

When performing a search, setting the logAnalytics field to true will record that search in the analytics store.

Note that the analytics engine will automatically “debounce” intermediate searches during an autocomplete, so that only the final search is recorded. For example, if the user types “red”, then “red sh” and then “red shoes”, only the “red shoes” search will be recorded. The window of time during which intermediate searches are ignored can be configured via the analytics.aggregationWindowMs option.

In your storefront, you should then implement the following mutations in order to track search result views and click-throughs:

input SearchListViewedEventInput {
  queryId: String!
}
input SearchResultClickedEventInput {
  queryId: String!
  position: Int!
  resultId: String!
}
extend type Mutation {
  logSearchListViewed(input: SearchListViewedEventInput!): Boolean!
  logSearchResultClicked(input: SearchResultClickedEventInput!): Boolean!
}

The logSearchListViewed mutation allows us to register the fact that the search results were viewed. This can be used for both a full results page, or an autocomplete list.

The logSearchResultClicked mutation allows us to register the fact that a specific search result was clicked. This in turn allows the analytics engine to calculate click rate and average click position for each search term.

Vendure Plus Advanced Search Plugin