org.apache.pinot.tools.perf

Class QueryRunner

java.lang.Object

org.apache.pinot.tools.AbstractBaseCommand

org.apache.pinot.tools.perf.QueryRunner

All Implemented Interfaces:

Command

public class QueryRunner
extends AbstractBaseCommand
implements Command

Field Summary

Fields inherited from class org.apache.pinot.tools.AbstractBaseCommand

DEFAULT_CLUSTER_NAME, DEFAULT_ZK_ADDRESS

Constructor Summary

Constructors

Constructor and Description
QueryRunner()

Method Summary

Modifier and Type	Method and Description
String	description()
boolean	execute()
boolean	getHelp()
String	getName()
static void	increasingQPSQueryRunner(PerfBenchmarkDriverConf conf, Stream<String> queries, int numTimesToRunQueries, int numThreads, int queueDepth, double startQPS, double deltaQPS, int reportIntervalMs, int numIntervalsToReportAndClearStatistics, int numIntervalsToIncreaseQPS, long timeout) Use multiple threads to run query at an increasing target QPS.
static void	main(String[] args)
static void	multiThreadedQueryRunner(PerfBenchmarkDriverConf conf, Stream<String> queries, int numTimesToRunQueries, int numThreads, int queueDepth, int reportIntervalMs, int numIntervalsToReportAndClearStatistics, long timeout) Use multiple threads to run queries as fast as possible.
static void	singleThreadedQueryRunner(PerfBenchmarkDriverConf conf, Stream<String> queries, int numTimesToRunQueries, int reportIntervalMs, int numIntervalsToReportAndClearStatistics, long timeout) Use single thread to run queries as fast as possible.
static void	targetQPSQueryRunner(PerfBenchmarkDriverConf conf, Stream<String> queries, int numTimesToRunQueries, int numThreads, int queueDepth, double startQPS, int reportIntervalMs, int numIntervalsToReportAndClearStatistics, long timeout) Use multiple threads to run query at a target QPS.

Methods inherited from class org.apache.pinot.tools.AbstractBaseCommand

cleanup, printExamples, printUsage

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.pinot.tools.Command

printUsage

Constructor Detail

QueryRunner

public QueryRunner()

Method Detail

getHelp

public boolean getHelp()

Specified by:

getHelp in interface Command

getName

public String getName()

Overrides:

getName in class AbstractBaseCommand

description

public String description()

Specified by:

description in interface Command

execute

public boolean execute()
                throws Exception

Specified by:

execute in interface Command

Throws:

Exception

singleThreadedQueryRunner

public static void singleThreadedQueryRunner(PerfBenchmarkDriverConf conf,
                                             Stream<String> queries,
                                             int numTimesToRunQueries,
                                             int reportIntervalMs,
                                             int numIntervalsToReportAndClearStatistics,
                                             long timeout)
                                      throws Exception

Use single thread to run queries as fast as possible.

Use a single thread to send queries back to back and log statistic information periodically.

Queries are picked sequentially from the query file.

Query runner will stop when all queries in the query file has been executed number of times configured.

Parameters:

conf - perf benchmark driver config.

queries - query stream.

numTimesToRunQueries - number of times to run all queries in the query file, 0 means infinite times.

reportIntervalMs - report interval in milliseconds.

numIntervalsToReportAndClearStatistics - number of report intervals to report detailed statistics and clear them, 0 means never.

timeout - timeout in milliseconds for completing all queries.

Throws:

Exception

multiThreadedQueryRunner

public static void multiThreadedQueryRunner(PerfBenchmarkDriverConf conf,
                                            Stream<String> queries,
                                            int numTimesToRunQueries,
                                            int numThreads,
                                            int queueDepth,
                                            int reportIntervalMs,
                                            int numIntervalsToReportAndClearStatistics,
                                            long timeout)
                                     throws Exception

Use multiple threads to run queries as fast as possible.

Use a concurrent linked queue to buffer the queries to be sent. Use the main thread to insert queries into the queue whenever the queue length is low, and start numThreads worker threads to fetch queries from the queue and send them.

The main thread is responsible for collecting and logging the statistic information periodically.

Queries are picked sequentially from the query file.

Query runner will stop when all queries in the query file has been executed number of times configured.

Parameters:

conf - perf benchmark driver config.

queries - query stream.

numTimesToRunQueries - number of times to run all queries in the query file, 0 means infinite times.

numThreads - number of threads sending queries.

queueDepth - queue size limit for query generator

reportIntervalMs - report interval in milliseconds.

numIntervalsToReportAndClearStatistics - number of report intervals to report detailed statistics and clear them, 0 means never.

timeout - timeout in milliseconds for completing all queries

Throws:

Exception

targetQPSQueryRunner

public static void targetQPSQueryRunner(PerfBenchmarkDriverConf conf,
                                        Stream<String> queries,
                                        int numTimesToRunQueries,
                                        int numThreads,
                                        int queueDepth,
                                        double startQPS,
                                        int reportIntervalMs,
                                        int numIntervalsToReportAndClearStatistics,
                                        long timeout)
                                 throws Exception

Use multiple threads to run query at a target QPS.

Use a concurrent linked queue to buffer the queries to be sent. Use the main thread to insert queries into the queue at the target QPS, and start numThreads worker threads to fetch queries from the queue and send them.

The main thread is responsible for collecting and logging the statistic information periodically.

Queries are picked sequentially from the query file.

Query runner will stop when all queries in the query file has been executed number of times configured.

Parameters:

conf - perf benchmark driver config.

queries - query stream.

numTimesToRunQueries - number of times to run all queries in the query file, 0 means infinite times.

numThreads - number of threads sending queries.

queueDepth - queue size limit for query generator

startQPS - start QPS (target QPS).

reportIntervalMs - report interval in milliseconds.

numIntervalsToReportAndClearStatistics - number of report intervals to report detailed statistics and clear them, 0 means never.

timeout - timeout in milliseconds for completing all queries

Throws:

Exception

increasingQPSQueryRunner

public static void increasingQPSQueryRunner(PerfBenchmarkDriverConf conf,
                                            Stream<String> queries,
                                            int numTimesToRunQueries,
                                            int numThreads,
                                            int queueDepth,
                                            double startQPS,
                                            double deltaQPS,
                                            int reportIntervalMs,
                                            int numIntervalsToReportAndClearStatistics,
                                            int numIntervalsToIncreaseQPS,
                                            long timeout)
                                     throws Exception

Use multiple threads to run query at an increasing target QPS.

Use a concurrent linked queue to buffer the queries to be sent. Use the main thread to insert queries into the queue at the target QPS, and start numThreads worker threads to fetch queries from the queue and send them.

We start with the start QPS, and keep adding delta QPS to the start QPS during the test.

The main thread is responsible for collecting and logging the statistic information periodically.

Queries are picked sequentially from the query file.

Query runner will stop when all queries in the query file has been executed number of times configured.

Parameters:

conf - perf benchmark driver config.

queries - query stream.

numTimesToRunQueries - number of times to run all queries in the query file, 0 means infinite times.

numThreads - number of threads sending queries.

queueDepth - queue size limit for query generator

startQPS - start QPS.

deltaQPS - delta QPS.

reportIntervalMs - report interval in milliseconds.

numIntervalsToReportAndClearStatistics - number of report intervals to report detailed statistics and clear them, 0 means never.

timeout - timeout in milliseconds for completing all queries.

numIntervalsToIncreaseQPS - number of intervals to increase QPS.

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception