tsexperiment

Name

TS Experiment - standalone agent to run TSLoad experiments

Synopsis

$ tsexperiment [global options] subcommand [options and arguments]

Description

Where global options are:

• -e <experiment_path>
  Path to experiment root directory
  
• -d|-t
  Enable debug or tracing
  

Experiment constsists of current config and several "runs", that contain versions of config (maybe with altered parameters) and experiment results. Experiment config is a file named experiment.json Each run identified by it's unique number called runid.

List capabilities

$ tsexperiment info [CAPABILITY] [-x] [-H] [-l] [-j] [NAME]...
Lists supported TSLoad Core capabilities with it's parameters. If NAME arguments are specified, shows only information on desired capabilities.

Options:

• -j - Print in JSON format
  
• -x - Print extended data such as parameters and descriptions
  
• -H - Do not print headers
  
• -l - Print legend
  

Output description for workload types

Output columns for workload types:

• WLTYPE - name of workload type
  
• MODULE - name of the module that provides it
  
• CLASS - bitmask of workload class
  
• C - CPU intensive workloads. May be followed by i or f flags: integer or floating-point operations respectively and m flag for memory-intensive workloads.
  
• M - virtual memory workloads. Only available subflag is a for memory allocation
  
• F - filesystem workloads. Subflag o stands for operations such as making directories, creating/deleting files, etc. while rw subflag means read-write workload.
  
• D - disk workload. Only available subflag is rw for read-write
  
• N - network workload. May be followed by c meaning that actual workload is for some network application
  
• O - miscellanous OS-level workload
  

Output columns for workload params:

• PARAM - name of parameter
  
• SCOPE - scope of parameter (WL for workload, RQ for per-request parameters and OUT for output parameters)
  
• OPT - flag that parameter is optional
  
• TYPE - type of parameter
  
• RANGE - minimum and maximum values for integer and float types, maximum length of string or available values for string sets
  
• DEFAULT - default parameter value (if provided by module)
  

Working with experiments

Show information about experiment

tsexperiment -e <experiment_path> list
Lists experiment runs

Output columns:

• ID - id of experiment or so-called runid
• NAME - name of experiment run
  
• STATUS - status of experiment run. FINISHED - run was successfully finished, ERROR - error was encountered during experiment run, NOTCFG - experiment was not configured
  
• DATE - date and time when workload start was scheduled
  
• HOSTNAME - name of host where experiment was run
  

tsexperiment -e <experiment_path> show [-j] [-l] [RUNID]
Shows experiment run information and config: enlists global options such as runtime, information of host where experiment was run, workloads and threadpools.

Options:

• -j - Show in JSON format
  
• -l - Show as name-value list (useful to create run -s option arguments)
  

Running experiments

tsexperiment -e <experiment_path> run [-n name] [-s param=value] [-b] [-T] [RUNID]
Creates new experiment run and starts it

Options and arguments:

• -n - sets name of experiment
  
• -s - sets variable of experiment config only for current run (see below)
  
• -b - enables batch mode (see below)
  
• -T - enables trace-driven mode. In this mode tsexperiment uses traces generated during previous run (specified as runid). It also changes dispatchers of all threadpools to trace mode. Request arrival time, per-request parameters are used, and also requests are dispatched to same workers it was executed during baseline run.
  

Running experiments in batch mode

Batch mode is intended to run set of experiments from shell-script and enabled by setting -b option. In this mode tsexperiment doesn't print information about course of experiment (it is still saved in <run_directory>/tsexperiment.out file) and only prints runid to stdout so you can use it in your script. For example you may write bash script:

#!/bin/bash
EXPERIMENT=/var/tsload/sample/
RUNID=$(tsexperiment -e $EXPERIMENT run -b)
tsexperiment -e $EXPERIMENT show $RUNID

Also experiment config variables may be altered by using -s option, where param name is path to JSON node inside config. I.e:

run -s threadpools:tp_test:num_threads=10
run -s workloads:workload1:params:cycles:pmap:0:probability = 0.3

Working with experiment results

tsexperiment -e <experiment_path> report RUNID [WL]...
Show some statistics report about experiment run. If WL arguments are specified than tsexperiment reports only particular workloads. RUNID argument is mandatory.

Output columns:

• STEP - id of step
  
• Request numbers
  
• COUNT - requests that were scheduled to be executed during this step
  
• ONTIME - requests that were started their execution according to its arrival time
  
• ONSTEP - requests that were finished before it's step ended
  
• DISCARD - requests that were discarded due to threadpool policy or request scheduler deadline parameter
  
• Times in milliseconds - mean and standard deviation
  
• WAIT TIME - time between request arrival and start of service
  
• SERVICE TIME - time that was spent by worker while executing request
  

tsexperiment -e <experiment_path> export [-d DEST] [-F csv|json|jsonraw] [-o option] [-o wl_name:option] RUNID [WL]...
Export workload measurement data into text files

Options:

• -d - destination directory of output. If not set, then experiment run directory is used
  
• -F - desired format of output.
  
• -o option - options for TSFile backend. See more at tsfutil

Exit status

If command was successfully executed, tsexperiment returns 0 as exit status. If invalid command line was passed to tsexperiment it returns one of the values from range {1..5}. In case of other failures, it will return 30 (CMD_GENERIC_ERROR code), or one of the well known error codes from range {11..14}.

Code	Mnemonic	Description
	Basic errors
0	CMD_OK	Everything went fine
1	CMD_MISSING_CMD	Subcommand is missing from command line
2	CMD_UNKNOWN_CMD	Unknown subcommand
3	CMD_INVALID_OPT	Incorrect option was specified or option needs argument
4	CMD_INVALID_ARG	Invalid option or argument value (like negative integer)
5	CMD_MISSING_ARG	Required option or argument is missing
	Experiment errors
11	CMD_NOT_EXISTS	Needed file not exists
12	CMD_NO_PERMS	Insufficient permissions to read or write on of the files
13	CMD_ALREADY_EXISTS	TSExperiment tried to create file, but it is already exists
14	CMD_IS_ROOT	Some operation that TSExperiment had tried to perform on root experiment only allowed for experiment runs
30	CMD_GENERIC_ERROR	All other errors

Experiment errors will be accompanied with EXPERR error code printed to standard error output.