Command Line Command Usage

Version 2.1
August 9, 2014


Command Line Documentation:

For those Open|SpeedShop users who do not want to use the GUI, or are more comfortable with command line control, that option is available.  In this section we will discuss the various Command Line Interface (CLI)  options, and how to set up basic experiments using them.


For starters, to invoke Open|SpeedShop on the command line, once installed on a system:

%openss -cli


Open|SpeedShop will start up and the user will see the following:

Welcome to OpenSpeedShop 1.9.2
openss>>

At this point, one may begin performance analysis at will.  Below we list all the possible CLI commands, their definitions, and how one may use them.

To begin, should all else fail, the help command is very useful. 
help:
<string> = help [-v <help_modifier_list>] [<string>]

Request information about a topic.

Here are the available help topics:

commands        -> Lists all available commands
grammar         -> Lists command grammar elements
experiments     -> Lists available experiments
man             -> How to access the Open|SpeedShop man page
terms           -> Lists general Open|SpeedShop nomenclature
preferences     -> Lists general Open|SpeedShop preference variables

To get details for a particular topic,
type "help <topic>".
For instance, to get help list of the topic "commands"
      openss>>help commands
To get help on an individual command type, such as "expCreate"
      openss>>help expCreate


To begin, let us look at an exapmle for running a simple pcsamp experiment on an executable called 'mutatee'.
The following snippet demonstrated how the user would set up and run the experiment:

openss>>expcreate -f /home/openss/demo/Simple/mutatee pcsamp
# Below a "1" is returned to indicate the experiment number
openss>>   1
# The next user command is "expGo" which runs the experiment
openss>>expGo
o
penss>>
# The next three lines are output from the executable's execution
Usage: /home/openss/demo/Simple/mutatee <size>
No size argument given.   Defaulting to 250.
/home/openss/demo/Simple/mutatee: successfully completed.
# The next user command tells Open|SpeedShop to print the results of the experiment
# NOTE - the 5 at the end of the stats parameter indicates you want to view the top 5 functions.
# If you had specified 33, as in stats33, you would see the top 33 functions in the performance report.
openss>>expView pcsamp5

# The next four lines are the output of the experiment due to the expView command
  CPU Time
(Seconds)  Function
        
2.090000  f3
        
1.320000  f2
          
0.650000  f1
# Open|SpeedShop prompt for additional command input
openss>>exit

expCreate:
]
<expId> = expCreate [-i  offline] | [ [-i online] [ -v mpi] ] [ <target_list> ] [ <expType_list> ]
                   
Start the process of defining an experiment.

o define a new experiment identifier,

o set the current focus to the new identifier and

o return the experiment identification identifier.

o The experiment will not execute until an 'expGo' command
is issued.

- If the -v mpi option is selected, all the threads that
are part of a running application will be included
in the experiment.

- If the -v mpi option is not present, only those
threads that are running application will be included
in the experiment.

- The <target_list> will associate the specified executable
with the experiment being defined.

- The experiment type argument, <expType_list>, can be
used to specify the types of data that will be collected
during the experiment.

- Missing arguments can be supplied later with the 'expAttach'
command.

- An 'expCreate' command with no arguments will still return
a new <expId> and will set the focus.

Examples:

expCreate -f my_prog usertime

Creates an experiment targeting the application
'my_prog' using the usertime performance measurement
utility.

The next logical action would be to run the experiment
using expGo.

If you need to add more attributes to this experiment
before running it, use the expAttach command.

ExpCreate changes the focus to the newly created
experiment. If you wish to change the focus back to an
existing experiment use the expFocus command.


expGo:

expGo [ -v all || -x <expId> ]

Start execution or continue running the experiment.

- If the experiment has already begun execution,
this command causes execution to continue from
where it was interrupted.

- The experiment will continue to run until either:

* the application terminates, or

* the application executes a programmed halt, or

* the user issues another exp... command that refers
to this experiment Id. or

* the OpenSS tool terminates due to an exit command,
an external interrupt, or abnormal, internal event.

- Except when executed in batch mode, the execution of the
application and the collection of performance data does not,
automatically, prevent other commands from being issued:

A following command that depends on the result of this
experiment may not be able to execute and may block
further commands from being issued.

Independent commands may be issued and completed before
this experiment completes.
The status of any experiment can be determined through the
listStatus command.

Depending on the type of data collection that is taking
place, it may be possible to view the results before the
application completes.

- The absence of the -x <expId> and all keyword will result
in only the focused experiment being run.

- The use of -x <expId> will result in only that experiment
being run.

- The use of the all option will result in every defined experiment
being run.

Examples:

expGo

Run the currently focused experiment.

expGo -x 5

Run experiment #5. If #5 was not the currently focused
experiment this will NOT bring it into focus. Use
'expFocus' to do that.

The next logical action would be to display the
experiment results using expView.

expView:

expView [-v <modifier_list>] [-x <expId>] [<viewType>] [-m
<expMetric_list>] [<target_spec>]

View the experiment's output in text form.

- If this command is issued with no -x <expId>, the
focused experiment is used.

- Output can be directed to a file by the use of '>' or '>>'
followed with a file name, at the end of the command. If
the output is not redirected, it will be sent to stdout.

- If <expMetric_list_spec> is present, only those metrics
in the list will be included in the report.

- If a <target_list> is specified, only the data for the
specified targets will be included in the report.

- The report may also be controlled through <modifier_list>
or <expMetric_list> specifiers. The meaning of these options
is unique to each <viewType>, although the convention is
that the '-v <modifier_list>' will select a version of the
report that will be generated and '-m <expMetric_list>' will
determines which metrics or data items are included in that
report. The specific options available for a view can be
determined by requesting 'help <viewType>'.

- This command does not change the execution state of the
experiment, although the nature of the requested data may
require that the experiment complete execution before the
information can be provided.

- To find the viewtypes available for your experiment use
list -v views.

- To find the all the viewtypes available to you use
list -v views,all.

- To get detailed information for a specific viewtype use:

help <viewType> where <viewType> is the view type.

Examples
:
help mpi
help pcsamp
help hwc

exit:

exit

Terminate the session.

- Suspend execution or all executables

- Suspend data collection.

- Remove instrumentation from all applications.

- Release all applications from control of the OpenSS tool

- Allow all applications to continue executing.

- Close all data base files.

- Delete all temporary files.

- Close the GUI and Command windows.

- Terminate execution of the OpenSS tool.

The commands detailed below (expAttach, expClose, expData, expDetach, expDisable, expEnable, expFocus,
expPause, expRestore, expSave, expSetParam, and expStatus) are all used to control various aspects of the
experiment. Explanations and examples of each command are given.

expAttach:

expAttach [ -v mpi] [ -x <expId> ] [ <target_list> ] [ <expType_list> ]

Attach applications or collectors to an experiment.

The attached applications will not execute until
an expGo command is issued.

- If there is no -x <expId> provided, information is
attached to the focused experiment.

- If the -v mpi option is selected, all the threads that
are part of a running application will be included
in the experiment.

- If the -v mpi option is not present, only those
threads that are running application will be included
in the experiment.

- The <target_list> is used to restrict the knowledge
of the experiment or data collection to certain portions
of an appliaction.

- It is ambiguous to use both the -f and -p options.

The -f option implies that an executable is to be
loaded into the OpenSS tool from a file.

The use of -p option implies that the OpenSS tool
is to attach to a program that is already executing.

- If <target_list> is provided without <expType_list>,
those applications are attached to all the <expType>s
that are already attached of the experiment.

- If <expType_list> is provided without <target_list>,
the specified data collectors are used to instrument
all the executable routines that are already attached
to the indicated experiment.

- If both <target_list> and <expType_list> are provided,
only those portions of the application in the <target_list>
are linked to the specified collectors in <expType_list>.

- If neither <target_list> nor <expType_list> is provided,
the command does nothing.

Examples:

expAttach -x 2 -f my_prog pcsamp
Includes the executable program 'my_prog' into experiment
#2 and adds the performance measurement utility 'pcsamp'
to the experiment. The executable is loaded into the
system and instrumented to provide performance measurements.

Experiment #2 must have been previously defined with an
expCreate or expRestore command.

The program will not start execution until a subsequent
'expGo -x 2' command is issued.

expAttach -x 1 -v mpi -h localhost -p 1234 pcsamp

Which will, for the previously created experiment #1,
find pid 1234 and attach the pcsamp data collector to
all the associated pids that are part of the same mpi
job. The assumption is that the given pid is the
original one that the system allocated for an
'mpirun...' command.

expClose:

expClose [ -v kill,all ]

Close/Terminate the experiment.

The OpenSS tool is NOT terminated. Use exit to
terminate the session.

All data collectors attached to the experiment
are removed from the attached applications.

All attached applications are removed from the experiment.

If '-v kill' is not provided, attached applications
will continue executing.

If '-v kill' is provided, attached applications
will be terminated.

The data base file that was used to save information
about the applications and the collected data, is deleted
if it is a temporary file, or closed if it is a user
defined file. Use 'expSave' for saving information.

- Because important information may accidently be lost,
the absence of any arguments will not cause the focused
experiment to be deleted. This can only be done by
specifying the expId of the focused experiment.

- Use of '-v all' will cause all defined experiments
to be deleted.

Examples:

expClose -x 3

Closes experiment #3.

expClose all

Closes all experiments

In all the above cases, if the target application is
running it will be left running. To kill the
application(s) you will need to add '-v kill'.

This command will also delete all performance measurements
that were not explicitly saved with an 'expSave' command.
This means that the results of the experiment will not be
available to look at.

expCompare:

expCompare [-v <modifier_list> ]
[ <expId_spec> ]
[ <viewType> ]
[ -m <expMetric_list> ]
[ <target_list> ]

Simple compare view.
The tool determines the columninformation.

This looks a lot like the 'expView' command, but the
semantics are slightly different. The purpose of the
'expView' command is to display a summary that includes
the specified items from the specified (or implied)
<target_list>. The whole point of the 'expCompare'
command is to find multiple things to display in
side-by-side columns, rather than summarized in a single
column.

The semantic routine for the command looks through the
components of the <target_list> for non-singular
components. That is what defines the separate
components to display. For example, the following
command:

expCompare -x 1 usertime -v function -f foo -m time -h host1, host2

will produce a report that contains a column for the
time spend in function 'foo' on 'host1', along side the
time spent in function 'foo' on 'host2'. Similarly, the
following command:

expCompare -x 1 usertime -v function -f foo -m time -h host1, -p
11513,11514

will produce a report that contains a column for the
time spend in function 'foo' on 'host1' pid 1153, along
side the time spent in function 'foo' on 'host1' pid
1154.

It is an error to specify more than one sub-list of
items within the <target_list>.

expData:

expData [-v <modifier_list>] [-x <expId>] [<viewType>] [-m
<expMetric_list>] [<target_spec>]

Return data from experiment.

See expView for full options.

expDetach:

expDetach [ -x <expId> ] [ <target_list> ] [ <expType_list> ]

Detach applications from collectors.

The remaining applications will not execute until an
'expGo' command is issued.

The experiment retains knowledge of both the applications
and the collectors and they can be referenced on later commands.

All previously collected data samples will be retained.

No new data, of the specified type, will be collected
for the specified application.

- If there is no -x <expId> provided, the focused experiment
is used.

- If <target_list> is provided without <expType_list>,
all collectors are detached for those applications.

- If <expType_list> is provided without <target_list>,
the specified data collectors are detached from all
the executable routines that are ttached to the indicated
experiment.

- If both <target_list> and <expType_list> are provided,
only the specified collectores in <expType_list> are
detached from just those applications described in <target_list>.

- If neither <target_list> nor <expType_list> is provided,
the command does nothing.

Examples:

expDetach -f my_prog pcsamp

Detach the pcsampling plugin from the targeted
application 'my_prog' in the focused experiment.

expDetach -x 2 -f my_prog pcsamp

Detach the pcsampling plugin from the targeted
application 'my_prog' in experiment #2 only.

expDetach -x 2 pcsamp

Detach the pcsampling plugin from the a.out and all dsos
in experiment #2.

expDisable:

expDisable [ -v all || -x <expId> ]

Turn off data collection.

The experiment remains defined.

Data collection is "turned off" and no new
information is saved.

Instrumentation is disabled but may not removed
from the application.

Previously collected data is still available and
can be viewed or saved.

The execution state of the application will not be changed.
If running, it will continue to do so.

- See related commands:

'expEnable' can be used to restart data collection.

'expSave' can be used to save previously collected data.

'expClose' can be used to destroy the experiment and
free all data space used to save the collected data.

- '-v all' will result in data collection being
stopped for all defined experiments.

- If this command is issued with no arguments, data
collection is stopped for the focused experiment.

Examples:

expDisable

Turn off collection of performance data in the focused
experiment.

expDisable all

Turn off collection of performance data in all defined
experiments.

expDisable -x 3

Turn off collection of performance data in the defined
experiment #3.

expEnable:

expEnable [ -v all || -x <expId> ]

Restart data collection.

Existing instrumentation is enabled.

Data collection is "turned on" and new information
will be saved.

New information will be merged with any previously
collected data.

The execution state of the application will not be changed.
If running, it will continue to do so. If not running, an
'expGo' command must be issued before it will start running,

- The all keyword will result in data collection being
restarted for all defined experiments.

- If this command is issued with no arguments, data
collection for the focused experiment is restarted.

Examples:

expEnable

Turn on collection of performance data in the focused
experiment that was previously disabled by an expDisable
command.

expEnable all

Turn on collection of performance data in all defined
experiments that was previously disabled by an
'expDisable' command.

expEnable -x 3

Turn on collection of performance data in the defined
experiment #3 that was previously disabled by an
'expDisable' command.

If #3 was not the currently focused experiment this will
NOT bring it into focus. Use 'expFocus -x 3' to do that.

The next logical action would be to run the experiment
using 'expGo -x 3'.

expFocus:

<expId> = expFocus [ -x <expId> ]

Focus the current input window on a specified experiment.

- In many commands -x <expId> is optional. When omitted,
  the currently "focused" experiment is used.
 - If no argument is present on this command, return
  the focused experiment.
 - If the -x <expId> argument has an experiment id
  of zero (0), then clear the experiment focus.
         A command that uses an experiment id of zero is illegal
         and will produce a fatal error.
 - This command does not change the execution state of
  an experiment.

Examples:

expFocus

    Return the id of the currently focused experiment

expFocus -x 5

    Bring into focus experiment #5.

Only one experiment can be in focus at a time.

expPause:
expPause [ -v all || -x <expId> ]

Temporarily interrupt the specified experiment.

o The experiment will remain suspended until an expGo is
issued or until the performance tool is exited, at which time
all the instrumentation is removed from the experiment and
the program is allowed to continue running.

- If -x <expId> is not provided, the focused experiment is used.

- If the keywork all is present, every defined experiment is halted.

Examples:

expPause

Suspend/stop execution of the currently focused
experiment

expPause -x 5

Suspend/stop execution of experiment #5. If #5 was not
the currently focused experiment this will NOT bring it
into focus. Use 'expFocus' to do that.

expRestore:

<exp_Id> = expRestore -f <file_name>

Restore experiment's definition and previously saved
data from a data base that was saved with an expSave command.

A new experiment is created and intitialize with
this information.

This command also sets the focus to the new experiment's
Identifier.

The new experiment will be in the same state as that
established with the expDisable command.

If the original applications are still running, the
user can reconnect to them and reinsert instrumentation
into them, through them, through use of the expEnable command.
Doing this will interrupt the executing application and an
expGo command will be required before the application continues
and new data samples can be collected.

Previously collected data can be looked at with the
expView command without reconnecting to the original
applications.

If an expGo command is issued, new data samples will
be placed into the data base specified by -f <file_name>.
A different data base can be specified with the expSave command.

Examples:

expRestore -f my_experiment.data

Reads the data stored in 'my_experiment.data' and
defines a new experiment that is initialized with
information in the file. The focus is set to
this new experiment.

No attempt is made to determine if the saved information
refers to a still running application however, a subsequent
'expGo' command will attempt to find and attach to it.

expSave:

expSave [-v copy] [-x <expId>] -f <file_name>

Save the experiment data for further analysis.

For each experiment, information about the executable
programs that are instrumented, information about the
instrumentation that is used and any performance data
that is collected are saved in a data base.

The data base is created by the OpenSS tool in a temporary
location and will be deleted when an expClose command is
issued for the experiment, or when an exit command is
issued that terminates the OpenSS tool.

The entire data base can be saved with the expSave command
and can be reloaded into the OpenSS tool with the expRestore
command.

The data base is in a special format that supports the
opertaions that the OpenSS tool must perform.

- Without '-v copy'

If the original, temporary data base file is still in use,
it is copied to the specified file name and the OpenSS tool
will use the new file to collect any new information about
the experiment. When the experiment completes, this data
base will be retained.

If a previous expSave command has caused the experiment's
information to be directed to a user defined data base

The contents of the previous data base

- without any of the data samples

- is copied to the new data base.

The previous data base is closed.

The new data base is used by the OpenSS tool to collect any
new information that is added to or collected for, the experiment.

- With '-v copy' the given location receives a copy
of the current state of the data base.

The current contents of the data base is duplicated and
saved to the given location.

No new information will be added to the given location.

New information will be added to the original data base.

- This command does not change the execution state of the
experiment, although it may temporarily suspend execution
while the command is being executed.

Examples:

expSave -f my_experiment.data

Saves data and state of the focused experiment to the
file 'my_experiment.data'.

expSave -x 3 -f my_experiment.data

Saves data and state of experiment #3 to the
file 'my_experiment.data'.

expSetParam:

expSetParam [-x <expId>] <expParam>=<expParamValue> [, 
<expParam>=<expParamValue>] ...

Change the value an experiment's argument.

- <expParam> will be unique within the experiment.

- The parameter names can be used by other experiments.

- This command does not change the execution state of
the experiment, although it may temporarily suspend
execution while the command is being executed.

Examples:

expSetParam pcsamp::sampling_rate=10

Will set the sampling rate in the pcsamp collector for
the focused experiment.

expSetParam -x 3 pcsamp::sampling_rate=10, usertime::sampling_rate=20

Will set the sampling rate of the pcsamp collector to 10
and the sampling rate of the usertime collector to 20,
both in experiment #3.

expSetParam -x 5 sampling_rate=10

Will set the sampling_rate to 10 in all the collectors
that are part of experiment #5.

Here are the parameters available to be set per experiment type:

fpe
event events to be traced

hwc
sampling_rate threshold valure or also called sampling_rate
event papi event name, like PAPI_TOT_CYC

hwctime
sampling_rate threshold valure or also called sampling_rate
event papi event name, like PAPI_TOT_CYC

io
traced_functions I/O functions to be traced

iot
traced_functions I/O functions to be traced

mpi
traced_functions MPI functions to be traced

mpiotf
traced_functions MPI functions to be traced

mpit
traced_functions MPI functions to be traced

pcsamp
sampling_rate The numeric value representing samples per second known as "the sampling rate"

usertime
sampling_rate The numeric value representing samples per second known as "the sampling rate"
expStatus:

expStatus [-v all || -x <expId>]

Report the current state of user define experiments.

- The absence of any option will result in information
Being reported for the focused experiment

- The use of -x <expId> will result in imformation
being reported for the specified experiment.

- The use of '-v all' will result in information being
reported for all the known experiments.

Examples:

expStatus

Displays the status of the currently focused experiment.

expStatus -x 2

Displays the status of experiment #2.

expStatus all

Displays the status of all defined experiments.


Users may want to create their own custom views for their experiments. The commands
cViewCreate, cViewDelete, cViewInfo, and cView are for that use.


cViewCreate:

cViewCreate [-v <modifier_list>] [-x <expId>] [<viewType>] [-m
<expMetric_list>] [<target_spec>]

Create a custom view id.

Defines the elements of a report. Except for the
name of the command, the form and meaning of each
field is identical to that of the 'expView' command.
In addition, this returns an ID that can be
referenced on the 'cView' command.

This command has a different purpose than the
'expView' command. Where the 'expView' command
generates a the report immediately, this command
will not produce a report until the ID is used on a
'cView' command. This allows the same report to be
generated multiple times, for different purposes.

Create an id that represents a defined view. This
View will then be represented in a single column of
a cView() command result.


cViewDelete:

cViewDelete -v all

Delete a custom view id from the openss database.

This command purges the definition of previously
created ID that was returned as the result of a
'cViewCreate' command. The ID can no longer be
referenced on any 'cView' commands.

Default: delete the last cvid defined.
-v all: delete all defined cvids.
-c <cvid_list>: delete listed cvids.


cViewInfo:

cViewInfo -v all

Dump the description of one or more custom view ids.

Default: dump the last cvid defined.
-v all: dump all defined cvids.
-c <cvid_list>: dump name cvids.

This command will echo back the information that was
specified on a 'cViewCreate' command. If no ID is
provided, the information of the most recently
defined cView ID is returned. If an ID list is
provided, information is returned for each of the
listed cView IDs. If the '-v all' option is
specified, information is returned for all of the
currently defined IDs.


cViews:

cView -c <cvid_list>

Detailed compare view where the user has defined the
contents of each custom view.

Where <cvid_list> represents a list or range of 2 or
more custom view ids

This command uses the information provided on
'cViewCreate commands to generate a report. If there
is only one ID provided in the '-c <id_list>', the
output of this command is equivalent to executing an
'expView' command with the same fields that were on
the 'cViewCreate' command.

When multiple IDs are provided in the '-c
<id_list>', a report is generated for each ID and
the output from each is merged into a final report,
using the right most field as a key. The final
report will have columns for each of the fields that
the initial reports contain, except that the right
most column (the key) is only printed once. Keys
that exist in one of the generated reports and not
others may have columns that are blank filled.

Example:

cViewCreate -x 4 usertime -m usertime::exclusive_time
cview -c 3, 5


Below are commands (list, clearBreak, exit, history, log, openGui, playBack
record, setBreak, and wait) that are used to control Open|SpeedShop.

list:

<ListOf_info> = list -v <listType> [-v all || -x <expId>]

List information based on <listType>.

Where <listType> is one of:

breaks: List breakpoints (help listBreaks)
expId: List the experiments that have been defined.
(help listExpId)
exptypes: List the available performance measurement
utilities that can be used to collect data in
an experiment. (help listExpTypes)
hosts: Lists the hosts that define the specified cluster.
(help listHosts)
metrics: Lists the metrics that are associated with a
ViewTypeList object. (help listMetrics)
obj: List the objects of the applications that are part
of the specified experiment. (help listObj)
params: List the parameters that are associated with a
particular <viewType> or set of <viewType>s.
(help listParams)
pids: List running processes associated with a specific
experiment and, optionally, on a specific machine.
(help listPids)
ranks: List the mpi ranks associated with a specific
experiment, a specific Pid or on a specific machine.
(help listRanks)
src: List the source filenames of the modules that are part
of the specified experiment. (help listSrc)
status: List the status of user experiments. (help listStatus)
threads: List the Threads associated with a specific experiment,
a specific Pid or on a specific machine.
(help listThreads)
views: List the views that are available for an ExpTypeList.
(help listViews)

The format of the output list is dependant on the type
of list requested.



*********
<list> ::= <list element> [ , <list element> ]*

Comma delimited list of elements.

A list element can consist of a single item such as
a filename or a range tuple.

The only time a list delimiter is NOT a comma ','is when
the list elements are <target>. In this case the delimiter
is a semicolon ';'.

Examples:

-f file1,file2,file3

A list of file name strings.

-r 44,55,66:69,77

A list of the rank numbers 44,55,66,67,68,69 and 77.
Note that a <range can be an element of a list.

An example of a list of <target>
-h host1 -p 333:400 ; -h host2 -p 500:540

An example of a list of <target>

All the pids for this experiment on host1 from.
333 ti 400 and all the pids on host2 from 500 to 540


clearBreak:

clearBreak <breakId>

Remove a breakpoint.

- This command does not change the execution state of an
experiment, although it may be temporarily suspended
during execution of the command.


history:

history [<integer>]

Print a list of previously executed commands.

- If no <integer> is provided, print out only
requested number of history entries.


log:


log [-f <file_name>]

Echo executed commands and their results to a file.

- Stop echoing if no file is specified on a log command.

- This is primarily intended to be an internal debug aid
for the OpenSS tool developer since the generated files
can quickly become huge.


openGUI:

openGui

Open the Graphical User Interface.


playBack:

playBack -f <file_name>

Read and execute commands stored in a file.


record:


record [-f <file_name>]

Begin echoing executed commands to a file.

Recording is preformed relative to the input source that
issues this command.

This design causes commands issued by the GUI to be
recorded separately from commands issued by the Command window.

- Commands read from a file specified on a succeeding playback
command will not be echoed.

- Stop echoing if no file is specified on a record command.

- Stop echoing if the end of the input file containing the
original record command is encountered.



setBreak:

<breakId> setBreak [-x <expId>] [<target>] <address_description>

Enter a breakpoint, which will halt the application when reached.

- If -x <expId> is not provided, the focused experiment is used.

- The break location is specified through the combination of the
<target> and <address_description> arguments.

- The break location must be a location in the specified experiment.

- If <target> is not provided, the <address_description> must be
valid on every host and executable attached to the experiment.

wait:

Wait [ -v terminate [ , all ] ] [ -x <expId> ]

Don't execute any more commands until an experiment terminates.

- The 'expGo' command initiates asynchronous execution of
applications and allows new commands to be executed while the
application is still running and still collecting performance
data. Often times, this is convenient because it allows for the
display of intermediate data, before the application completes.
However, there may be situations particularly when running a
predefined sequence of commands through a 'playback' command,
when it is desirable to block the execution of new commands until
the application has completed. The 'Wait' command will do this.

- If the tool is initiated with the '-batch' option, a 'Wait'
command is implied before every 'expView' command. This behavior
greatly reduces the need for the 'Wait' command.

- If no options are provided, the '-v terminate' option is assumed
and command execution will be suspended until the focused
experiment stops execution.

- If only the '-x <expId>' option is provided, the '-v terminate'
option is assumed and command execution will be suspended until
that experiment stops execution.

- If the '-v terminate, all' option is provided, command execution
will be suspended until all experiments stop execution.

Examples:

expGo
Wait
expView

This sequence will cause the 'expView' command to issue
after the running experiment completes and all the
performance data is available.

Without the 'Wait' command, the 'expView' command will
be issued immediately after the 'expGo' when there may
not be any data available to display.