Benchmarking Concurrent Workloads for Operational Analytics and BI Workloads

When hundreds of people open dashboards at the same time — sales teams checking live metrics, finance monitoring daily spend, operations tracking inventory — the system powering the insights for these applications has to be performant, efficient and scale to dynamic user demands.

Why this matters

Our customers power a wide variety of operational analytics workloads with Snowflake — fresh, interactive queries that power dashboards and decision-making across the business. These queries are small and selective, but they arrive in bursts from dozens or hundreds of users. In some cases, analytics applications and dashboards power customer facing products, requiring global scale, running 24/7 for thousands of users.

Understanding and benchmarking highly concurrent workloads helps ensure that as systems grow busier, performance remains predictable. It’s not just about one fast query — it’s about every query completing faster, even during peak windows, and running the entire workload efficiently.

What we benchmark

To design our systems to represent real-world behavior, we build and maintain a wide variety of benchmarks, based on our customers’ workload patterns, and in some cases, based on industry standard benchmarks. One such benchmark uses a data set based on the industry standard TPC-DS benchmark, which models a retail business with sales, customers and products. For our internal runs, we use a 3 TB data set and run on a multi-cluster warehouse, configured to scale to a maximum of 10 clusters.

We run a consistent set of 14 TPC-DS queries that reflect the kinds of operations that dashboards perform every second: Queries 3, 7, 19, 27, 42, 43, 52, 53, 55, 63, 68, 73, 89 and 98. The queries contain typical query shapes for these workloads, such as group-bys, filtered aggregations, window functions, joins, counting and ranking — the building blocks of operational BI. You can download the full list of TPC-DS queries from our documentation site here.

Each simulated user runs these queries in a random order, with different predicate values for different streams. That randomization prevents synchronized execution and mimics real-world user activity.

How we run the benchmark

We start small — a single simulated user — then double the number of users step by step (2, 4, 8, 16, 32, 64, 128 and so on). Each user loops through their 14-query list for 20 minutes. We found that a 20-minute, 32-user benchmark yielded stable results, with no significant changes in numbers. This benchmark runs daily, comparing with past executions in our production environment, allowing us to see the impact of our weekly performance improvements on the same software and infrastructure that our customers use for their most mission-critical workloads.

The key metrics we track are:

Query latency: How long queries take (median and 99th percentile)
Throughput: Total queries completed per second
Scalability: How those numbers behave as concurrency increases

How you can try it yourself

Anyone can reproduce a simplified version of this test using publicly available tools — no internal frameworks required.

Load data: Populate a TPC-DS schema in your Snowflake account (many public data generators are available online).
Collect queries: Use the 14 TPC-DS SQL queries listed above.
Generate load: Use a tool such as Apache JMeter, Locust or k6 to simulate multiple users running those queries concurrently.
Measure results: Capture query times and calculate throughput and latency.

You can containerize your setup with Docker for portability, or run locally from any machine that can connect to Snowflake.

This will allow you to observe how your own configuration scales under BI-style pressure and learn how factors like warehouse size, caching and concurrency-scaling affect responsiveness, and compare this across different analytics platforms.

Continuous benchmarking

We run these concurrency tests every day inside Snowflake. They help us detect performance regressions before they reach customers and reveal new opportunities for optimization. Using the same 14 TPC-DS queries daily provides a consistent baseline for measuring progress and guiding engineering improvements.

Takeaway

Operational analytics is one of the most common and important workloads on Snowflake. Testing concurrency ensures that it stays fast — not just for one analyst, but for everyone using data to make decisions at the same time.

If you’d like to explore how your own environment handles concurrent BI-style workloads, you can use these same ideas and open source tools. Start small, measure carefully, and you’ll learn a lot about how your system behaves under real-world pressure — exactly the insight that drives continuous improvement.

Watch the Platform Connect webinar to see how we enable predictable scale for high-concurrency BI workloads.