We tested our search with complex configurations, massive product datasets and tens of thousands of searches per hour. The result? Response times faster than the blink of an eye.

There is a high cost to low performance in any customer-facing system. How efficient and effective a system behaves affects the perception of the overall quality and determines whether the user will stay or leave. In eCommerce, visitors’ first glimpse of your system performance is often the search.  

Your search performance can ensure the staying power of your online shop in a highly competitive market or shorten its lifespan. So, it’s no surprise that online retailers spend countless hours and resources optimizing it. 

This blog will cover eCommerce search performance in-depth, provide realistic benchmarks based on unique business requirements and the performance you can achieve with our solution. 

Expect to learn:

⚡ How we test response times

⚡ FactFinder’s response times for simple to complex shops

⚡ When high speed ≠ high performance

⚡ Factors that affect search performance 

How we test response times 

Let’s start by debunking “average response time” as a reliable metric. It oversimplifies the issue and hides the true performance of larger, more complex systems within the average of many smaller, simpler systems.  

When evaluating search solutions, decision-makers want to ensure that a new solution will match or exceed the capabilities of their current one. At the same time, they want ten additional capabilities they currently do not have. They expect to maintain the current performance and response time while adding these new features.

Imagine driving a sports car and suddenly wanting it to transport five passengers and five pieces of luggage without sacrificing its top speed. That’s the kind of challenge we’re addressing. 

To provide useful benchmarks, we categorized customers into different clusters based on their needs and system complexities. This approach helps us give businesses realistic expectations for performance. 

For our tests, we used an internally developed tool called Go Traffic. It’s designed for flow testing and generating high traffic loads on the system. The tool sends random search terms to FactFinder, which then responds with the corresponding search results.

We ran multiple test scenarios to understand how different configurations and product data sizes impact response time and how the system handles tens of thousands of searches per hour. We specifically measured the time between making a request and receiving a response – so, the processing time for the search system. 

One key point to highlight is that in our tests, we weren’t able to fully leverage the caching options that FactFinder offers. The caching in our system is useful for frequently used search terms. It recognizes these recurring terms and reuses previous search results, delivering quicker performance. While the simpler tests did use our caching, we can no longer use caching once we switch on capabilities such as personalization. This means that in real-life scenarios, our solution could perform even faster than in our tests.  

FactFinder’s response times

We measure response times in milliseconds (ms), where one millisecond is one-thousandth of a second. To put it in perspective, a human blink takes 100 to 150 milliseconds.

For systems with less than 10,000 products and a simple configuration, the response time is an impressive 6 to 7 milliseconds (ms). That’s just 1/21 of the time it takes to blink an eye.

When we scaled up the product data set to 100,000 products and activated more advanced features, we saw response times of 20 ms or about 1/6 of a blink.

We kept increasing the data size to around 750,000 products (a large dataset typical of B2B) along with complex features. FactFinder still delivers highly relevant results in just 50 to 60 ms or 2/5 of a blink.

Even with a massive catalog and multiple features, Factfinder’s “slowest” response time leaves over 900 ms to build the page and deliver a sub-second response to the shopper.  

That is true performance.

Is your current search solution holding you back?

Discover how FactFinder can enhance your online store’s performance. Contact us today to see the difference.

High speed does not always equal high performance

One mistake decision-makers continue making is thinking that high performance solely equals speed. While closely related, they refer to different aspects of search.  

  • Speed focuses on how fast a specific process is completed. That is, how quickly the search query is answered.  
  • Performance includes speed, efficiency, scalability and reliability. That is, how well the search handles high traffic, remains consistent under different loads and latency.  

For online shopping, speed affects user experience in terms of quickness while performance impacts overall satisfaction.

Understanding this distinction is important, especially when evaluating your current search or exploring possible solutions. Some providers boast high search performance, claiming response times are 200 times faster than competitors. Numbers can be impressive, but context is essential to understand what they mean.

An analogy to explain how they test response times is a car race. Imagine two cars: one is highly tuned with customized tires, suspension and engine (their solution), while the other is an out-of-the-box car without modifications (competitors). Saying their tuned car performs faster than the other isn’t false but it’s a narrow view of search focusing solely on speed. Once this “advantage” is marketed, it misleads stakeholders into hyper-focusing on speed, potentially overlooking additional features that enhance user experience. While speed is a crucial component of a car’s abilities, performance determines how well a car can navigate the complexities and demands of a racing environment.

FactFinder wins in both speed and performance. Our search response times remain incredibly fast while delivering advanced capabilities that provide value to eCommerce businesses. Unlike competitors who test on very basic shop setups, our speeds are proven even in the complex configurations typical of B2B shops.

Explore FactFinder features crafted for B2B.

Factors that affect search performance

Shoppers care about the time between clicking or entering and having a full page load. But more moving parts contribute to the page load speed aside from the search, including content management systems, merchandising systems and site navigation. Configurations within the search system itself also significantly impact speed. For example, some online shops apply rules to the order in which results are displayed while others have filters that dynamically adapt to queries. Additionally, there are merchandising features that modify search results by adding or changing products or including extra content like photo banners, text elements or special tiles. Retailers often combine these features with complex capabilities such as personalization, natural language processing and automatic search optimization. 

Then there’s the product data which goes beyond just the number of products. Searching through a set of 5,000 products is much simpler and faster than 750,000. But it’s not just the quantity; the quality and complexity of the data matter too. This includes the number of fields, data structure, content within each product record, customer-specific data and market or seller information. All these factors require more processing power and affect performance. 

You might already know this, but it’s worth mentioning because many decision-makers struggle to grasp the full context of these factors when choosing a search solution.

Let’s go back to our car analogy to explain further. We’ve already tested the cars for speed, but what if aside from arriving you need additional features like passenger capacity, cargo space, towing ability and off-road capability? The same goes for search solutions. Online retailers have certain business objectives that demand more from their search function than simply speed. A typical FactFinder customer has more products, complex data structures and advanced capabilities such as Customer-Specific Info, Geo and 1:1 Personalization.  

As mentioned earlier, advanced capabilities can increase response time. The difference, however, is minimal. If one solution has a response time of 20 milliseconds and another with more capabilities has 50 milliseconds, the latter seems slower. But do you know what 30 milliseconds are? It’s 1/4 the duration of a blink. A blink is already an almost imperceivable action. 30 milliseconds are a fraction of that – a difference beyond human perception. 

We believe the best outcome for shoppers is finding what they want quickly and intuitively. This means fast search responses but also presenting the most relevant items at the top of the search results. If this takes a fraction of a second longer, it’s time well spent.

Search performance that wins customers

We understand that search performance is crucial for your online store’s success. That’s why we’re dedicated to delivering fast, accurate, thorough product discovery, all within a single response. Our search delivers complete search results with full product details and includes related data for filtering and sorting. This means your shoppers have all the information they need at their fingertips, leading to the best possible customer experience.  

We also recognize that every business is unique. That’s why our solution provides eCommerce teams with advanced, easy-to-use tools to tailor search behavior specifically for their shoppers. This way, your customers get the most relevant results, customized to their preferences, using our comprehensive configuration options. Whether you run a B2C or B2B shop, our search capabilities handle any scale without compromising speed or accuracy.