A Swift script describes data, application components, invocations of
applications components, and the inter-relations (data flow) between
those invocations.

Data is represented in a script by strongly-typed single-assignment
variables. The syntax superficially resembles C and Java. For example,
{ and } characters are used to enclose blocks of statements.

Types in Swift can be atomic or composite. An atomic type can be either a
primitive type or a mapped type. Swift provides a fixed set of primitive types,
such as integer and string. A mapped type indicates that the actual data does
not reside in CPU addressable memory (as it would in conventional programming
languages), but in POSIX-like files. Composite types are further subdivided
into structures and arrays. Structures are similar in most respects to
structure types in other languages. In Swift, structures are defined using the
type keyword (there is no struct keyword). Arrays use numeric indices, but
are sparse. They can contain elements of any type, including other array types,
but all elements in an array must be of the same type. We often refer to
instances of composites of mapped types as datasets.

Atomic types such as string, int, float and double work the same way as in
C-like programming languages. A variable of such atomic types can be defined as
follows:

A struct variable is defined using the type keyword as discussed above.
Following is an example of a variable holding employee data:

The members of the structure defined above can be accessed using the dot
notation. An example of a variable of type Employee is as follows:

Arrays of structures are allowed in Swift. A convenient way of populating
structures and arrays of structures is to use the readData() function.

Mapped type and composite type variable declarations can be annotated
with a mapping descriptor indicating the file(s) that make up that
dataset. For example, the following line declares a variable named
photo with type image. It additionally declares that the data for
this variable is stored in a single file named shane.jpg.

Component programs of scripts are declared in an app declaration, with
the description of the command line syntax for that program and a list
of input and output data. An app block describes a functional/dataflow
style interface to imperative components.

For example, the following example lists a procedure which makes use of
the ImageMagick http://www.imagemagick.org/ convert command to
rotate a supplied image by a specified angle:

A procedure is invoked using the familiar syntax:

While this looks like an assignment, the actual unix level execution
consists of invoking the command line specified in the app
declaration, with variables on the left of the assignment bound to the
output parameters, and variables to the right of the procedure
invocation passed as inputs.

The examples above have used the type image without any definition of
that type. We can declare it as a marker type which has no structure
exposed to Swift script:

This does not indicate that the data is unstructured; but it indicates
that the structure of the data is not exposed to Swift. Instead,
Swift will treat variables of this type as individual opaque files.

With mechanisms to declare types, map variables to data files, and
declare and invoke procedures, we can build a complete (albeit simple)
script:

This script can be invoked from the command line:

This executes a single convert command, hiding from the user features
such as remote multisite execution and fault tolerance that will be
discussed in a later section.

Arrays of values can be declared using the [] suffix. Following is an example
of an array of strings:

An array may be mapped to a collection of files, one element per file, by using a
different form of mapping expression. For example, the filesys_mapper
maps all files matching a particular unix glob pattern into an array:

The foreach construct can be used to apply the same block of code to
each element of an array:

Sequential iteration can be expressed using the iterate construct:

This fragment will initialise the 0-th element of the step array to
some initial condition, and then repeatedly run the simulate
procedure, using each execution’s outputs as input to the next step.

By default, array keys are integers. However, other primitive types are also
allowed as array keys. The syntax for declaring an array with a key type different
than the default is:

For example, the following code declares and assigns items to an array with string
keys and float values:

In addition to primitive types, a special type named auto can be used to
declare an array for which an additional append operation is available:

Items in an array with auto keys cannot be accessed directly using a primitive
type. The following example results in a compile-time error:

However, it is possible to use auto key values from one array to access another:

Non-array variables are single-assignment, which means that they must
be assigned to exactly one value during execution. A procedure or
expression will be executed when all of its input parameters have been
assigned values. As a result of such execution, more variables may
become assigned, possibly allowing further parts of the script to execute.

In this way, scripts are implicitly parallel. Aside from serialisation
implied by these dataflow dependencies, execution of component programs
can proceed in parallel.

In this fragment, execution of procedures p and q can happen in
parallel:

while in this fragment, execution is serialised by the variable y,
with procedure p executing before q.

Arrays in Swift are more monotonic – a generalisation of being
assignment. Knowledge about the content of an array increases during
execution, but cannot otherwise change. Each element of the array is
itself single assignment or monotonic (depending on its type). During a
run all values for an array are eventually known, and that array is
regarded as closed.

Statements which deal with the array as a whole will often wait for the
array to be closed before executing (thus, a closed array is the
equivalent of a non-array type being assigned). However, a foreach
statement will apply its body to elements of an array as they become
known. It will not wait until the array is closed.

Consider this script:

Initially, the foreach statement will have nothing to execute, as the
array a has not been assigned any values. The procedures r and s
will execute. As soon as either of them is finished, the corresponding
invocation of procedure p will occur. After both r and s have
completed, the array a will be closed since no other statements in the
script make an assignment to a.

As with many other programming languages, procedures consisting of
Swift script can be defined. These differ from the previously
mentioned procedures declared with the app keyword, as they invoke
other Swift procedures rather than a component program.

This will invoke two procedures, with an intermediate data file named
anonymously connecting the first and second procedures.

Ordering of execution is generally determined by execution of app
procedures, not by any containing compound procedures. In this code block:

then a valid execution order is: A1 S(x) A2 S(y). The compound
procedure A does not have to have fully completed for its return
values to be used by subsequent statements.

Each variable and procedure parameter in Swift script is strongly typed.
Types are used to structure data, to aid in debugging and checking
program correctness and to influence how Swift interacts with data.

The image type declared in previous examples is a marker type.
Marker types indicate that data for a variable is stored in a single
file with no further structure exposed at the Swift script level.

Arrays have been mentioned above, in the arrays section. A code block
may be applied to each element of an array using foreach; or
individual elements may be references using [] notation.

There are a number of primitive types:

Complex types may be defined using the type keyword:

Members of a complex type can be accessed using the . operator:

Sometimes data may be stored in a form that does not fit with Swift’s
file-and-site model; for example, data might be stored in an RDBMS on
some database server. In that case, a variable can be declared to have
external type. This indicates that Swift should use the variable to
determine execution dependency, but should not attempt other data
management; for example, it will not perform any form of data stage-in
or stage-out it will not manage local data caches on sites; and it will
not enforce component program atomicity on data output. This can add
substantial responsibility to component programs, in exchange for
allowing arbitrary data storage and access methods to be plugged in to
scripts.

Some external database is represented by the database variable. The
populateDatabase procedure populates the database with some data, and
the analyseDatabase procedure performs some subsequent analysis on
that database. The declaration of database contains no mapping; and
the procedures which use database do not reference them in any way;
the description of database is entirely outside of the script. The
single assignment and execution ordering rules will still apply though;
populateDatabase will always be run before analyseDatabase.

Data processed by Swift is strongly typed. It may be take the form of
values in memory or as out-of-core files on disk. Language constructs
called mappers specify how each piece of data is stored.

When a DSHandle represents a data file (or container of datafiles), it
is associated with a mapper. The mapper is used to identify which files
belong to that DSHandle.

A dataset’s physical representation is declared by a mapping descriptor,
which defines how each element in the dataset’s logical schema is stored
in, and fetched from, physical structures such as directories, files,
and remote servers.

Mappers are parameterized to take into account properties such as
varying dataset location. In order to access a dataset, we need to know
three things: its type, its mapping, and the value(s) of any
parameter(s) associated with the mapping descriptor. For example, if we
want to describe a dataset, of type imagefile, and whose physical
representation is a file called “file1.bin” located at
“/home/yongzh/data/”, then the dataset might be declared as follows:

The above example declares a dataset called f1, which uses a single file
mapper to map a file from a specific location.

Swift has a simplified syntax for this case, since
single_file_mapper is frequently used:

Swift comes with a number of mappers that handle common mapping
patterns. These are documented in the mappers section of this
guide.

The syntax of Swift script has a superficial resemblance to C and Java.
For example, { and } characters are used to enclose blocks of statements.

A Swift script consists of a number of statements. Statements may
declare types, procedures and variables, assign values to variables, and
express operations over arrays.

There are two kinds of procedure: An atomic procedure, which describes
how an external program can be executed; and compound procedures which
consist of a sequence of Swift script statements.

A procedure declaration defines the name of a procedure and its input
and output parameters. Swift script procedures can take multiple inputs
and produce multiple outputs. Inputs are specified to the right of the
function name, and outputs are specified to the left. For example:

The above example declares a procedure called myproc, which has two
inputs in1 (of type type1) and in2 (of type type2) and two
outputs out1 (of type type3) and out2 (of type type4).

A procedure input parameter can be an optional parameter in which case
it must be declared with a default value. When calling a procedure, both
positional parameter and named parameter passings can be passed,
provided that all optional parameters are declared after the required
parameters and any optional parameter is bound using keyword parameter
passing. For example, if myproc1 is defined as:

Then that procedure can be called like this, omitting the optional

or like this supplying a value for the optional parameter s:

Swift script provides if, switch, foreach, and iterate
constructs, with syntax and semantics similar to comparable constructs
in other high-level languages.

The following infix operators are available for use in Swift script
expressions.

At the top level of a Swift script program, the global modified may be
added to a declaration so that it is visible throughout the program,
rather than only at the top level of the program. This allows global
constants (of any type) to be defined. (since Swift 0.10)

The import directive can be used to import definitions from another
Swift file.

For example, a Swift script might contain this:

which would import the content of defs.swift:

Imported files are read from two places. They are either read from
the path that is specified from the import command, such as:

or they are read from the environment variable SWIFT_LIB. This
environment variable is used just like the PATH environment
variable. For example, if the command below was issued to the bash
shell:

then the import command will check for the file defs.swift in both
“${HOME}/Swift/defs” and “${HOME}/Swift/functions” first before trying
the path that was specified in the import command.

Other valid imports:

There is no requirement that a module is imported only once. If a module
is imported multiple times, for example in different files, then Swift
will only process the imports once.

Imports may contain anything that is valid in a Swift script,
including the code that causes remote execution.

The single_file_mapper maps a single physical file to a dataset.

Example:

There is a simplified syntax for this mapper:

The simple_mapper maps a file or a list of files into an array by
prefix, suffix, and pattern. If more than one file is matched, each of
the file names will be mapped as a subelement of the dataset.

The above maps all filenames that start with foo and have an extension
.txt into file f.

This will output the string hi to the file foo.txt.

The simple_mapper can be used to map arrays. It will map the array
index into the filename between the prefix and suffix.

simple_mapper can be used to map structures. It will map the name of
the structure member into the filename, between the prefix and the suffix.

This will output the string “hello” into the file qux.left.txt and the
string “goodbye” into the file qux.right.txt.

The concurrent_mapper is almost the same as the simple mapper, except that
it is used to map an output file, and the filename generated will
contain an extract sequence that is unique. This mapper is the default
mapper for variables when no mapper is specified.

Example:

The above example would use concurrent mapper for f1 and f2, and
generate f2 filename with prefix “foo” and extension “.txt”

The filesys_mapper is similar to the simple mapper, but maps a file or a
list of files to an array. Each of the filename is mapped as an element
in the array. The order of files in the resulting array is not defined.

TODO: note on difference between location as a relative vs absolute path
w.r.t. staging to remote location – as mihael said: It’s because you
specify that location in the mapper. Try location=”.” instead of
location=”/sandbox/…”

Example:

The above example would map all filenames that start with “foo” and
have an extension “.txt” into the array texts. For example, if the
specified directory contains files: foo1.txt, footest.txt,
foo__1.txt, then the mapping might be:

The fixed_array_mapper maps from a string that contains a list of
filenames into a file array.

Example:

would cause a mapping like this:

The array_mapper maps from an array of strings into a file

Example:

This will establish the mapping:

The regexp_mapper transforms one file name to another using regular
expression matching.

Example:

This example transforms a string ending gif into one ending jpg and
maps that to a file.

The structured_regexp_mapper is similar to the regexp_mapper. The
structured_regexp_mapper can be applied to lists while the regexp_mapper cannot.

Example:

This example transforms all strings in a list that end in gif to end in jpg and maps
the list to those files.

The csv_mapper maps the content of a CSV (comma-separated value) file
into an array of structures. The dataset type needs to be correctly
defined to conform to the column names in the file. For instance, if the
file contains columns: name age GPA then the type needs to have member
elements like this:

If the file does not contain a header with column info, then the column
names are assumed as column1, column2, etc.

Example:

The above example would read a list of student info from file
“stu_list.txt” and map them into a student array. By default, the file
should contain a header line specifying the names of the columns. If
stu_list.txt contains the following:

then some of the mappings produced by this example would be:

The external mapper, ext maps based on the output of a supplied Unix
executable.

The output (stdout) of the executable should consist of two columns of data,
separated by a space. The first column should be the path of the mapped
variable, in Swift script syntax (for example [2] means the 2nd element of an
array) or the symbol $ to represent the root of the mapped variable. The
following table shows the symbols that should appear in the first column
corresponding to the mapping of different types of swift constructs such as
scalars, arrays and structs.

Example: With the following in mapper.sh,

then a mapping statement:

would map

Advanced Example: The following mapper.sh is an advanced example of an external
mapper that maps a two-dimensional array to a directory of files. The files in
the said directory are identified by their names appended by a number between
000 and 099. The first index of the array maps to the first part of the
filename while the second index of the array maps to the second part of the
filename.

The mapper definition is as follows:

Assuming there are 4 files with name aaa, bbb, ccc, ddd and a mod_index of 10,
we will have 4×10=40 files mapped to a two-dimensional array in the following
pattern:

The swift command is the main command line tool for executing
Swift scripts.

The swift-osg-ress-site-catalog command generates a site catalog based
on OSG http://www.opensciencegrid.org/‘s ReSS information system
(since Swift 0.9)

Usage: swift-osg-ress-site-catalog [options]

–help

–vo=[name]

–engage-verified

–out=[filename]

–condor-g

swift-plot-log generates summaries of Swift run log files.

Usage: swift-plot-log [logfile] [targets]

When no targets are specified, swift-plog-log will generate an HTML
report for the run. When targets are specified, only those named targets
will be generated.

This section describes how an app procedure invocation is translated
into a (remote) unix process execution. It does not describe the
mechanisms by which Swift performs that translation; that is described
in the next section.

In this section, this example Swift script is used for reference:

The executable for wc will be looked up in tc.data.

This unix executable will then be executed in some application
procedure workspace. This means:

Each application procedure workspace will have an application workspace
directory. (TODO: can collapse terms application procedure workspace
and application workspace directory ?

This application workspace directory will not be shared with any other
application procedure execution attempt; all application procedure
execution attempts will run with distinct application procedure
workspaces. (for the avoidance of doubt: If a Swift script procedure
invocation is subject to multiple application procedure execution
attempts (due to Swift-level restarts, retries or replication) then each
of those application procedure execution attempts will be made in a
different application procedure workspace. )

The application workspace directory will be a directory on a POSIX
filesystem accessible throughout the application execution by the
application executable.

Before the application executable is executed:

During/after the application executable execution, the following must
be true:

Things to not assume:

This section describes the implementation of the semantics described in
the previous section.

Swift executes application procedures on one or more sites.

Each site consists of:

There is no assumption that the site shared file system for one site is
accessible from another site.

For each workflow run, on each site that is used by that run, a run
directory is created in the site working directory, by the Swift client
side.

In that run directory are placed several subdirectories:

Application execution looks like this:

For each application procedure call:

The Swift client side selects a site; copies the input files for that
procedure call to the site shared file cache if they are not already in
the cache, using the file transfer mechanism; and then invokes the
wrapper script on that site using the execution mechanism.

The wrapper script creates the application workspace directory; places
the input files for that job into the application workspace directory
using either cp or ln -s (depending on a configuration option);
executes the application unix executable; copies output files from the
application workspace directory to the site shared directory using cp;
creates a status file under the status/ directory; and exits,
returning control to the Swift client side. Logs created during the
execution of the wrapper script are stored under the info/ directory.

The Swift client side then checks for the presence of and deletes a
status file indicating success; and copies files from the site shared
directory to the appropriate client side location.

The job directory is created (in the default mode) under the jobs/
directory. However, it can be created under an arbitrary other path,
which allows it to be created on a different file system (such as a
worker node local file system in the case that the worker node has a
local file system).

The execution layer causes an application program (in the form of a unix
executable) to be executed either locally or remotely.

The two main choices are local unix execution and execution through
GRAM. Other options are available, and user provided code can also be
plugged in.

The kickstart utility can be used to capture environmental
information at execution time to aid in debugging and provenance capture.

Step i: text to XML intermediate form parser/processor. parser written
in ANTLR – see resources/VDL.g. The XML Schema Definition (XSD) for the
intermediate language is in resources/XDTM.xsd.

Step ii: XML intermediate form to Karajan workflow. Karajan.java – reads
the XML intermediate form. compiles to karajan workflow language – for
example, expressions are converted from Swift script syntax into Karajan
syntax, and function invocations become karajan function invocations
with various modifications to parameters to accomodate return parameters
and dataset handling.

Some Swift functionality is provided in the form of Karajan libraries
that are used at runtime by the Karajan workflows that the Swift
compiler generates.

Takes a command line parameter name as a string parameter and an
optional default value and returns the value of that string parameter
from the command line. If no default value is specified and the command
line parameter is missing, an error is generated. If a default value is
specified and the command line parameter is missing, @arg will return
the default value.

Command line parameters recognized by @arg begin with exactly one
hyphen and need to be positioned after the script name.

For example:

@extractInt(file) will read the specified file, parse an integer from
the file contents and return that integer.

@filename(v) will return a string containing the filename(s) for the
file(s) mapped to the variable v. When more than one filename is
returned, the filenames will be space separated inside a single string
return value.

@filenames(v) will return multiple values (!) containing the
filename(s) for the file(s) mapped to the variable v. (compare to
@filename)

@regexp(input,pattern,replacement) will apply regular expression
substitution using the Java java.util.regexp API
http://java.sun.com/j2se/1.4.2/docs/api/java/util/regex/Pattern.html.
For example:

will assign the value “abmonkeyhi” to the variable v.

@sprintf(spec, variable list) will generate a string based on the specified format.

Format specifiers

@strcat(a,b,c,d,…) will return a string containing all of the
strings passed as parameters joined into a single string. There may be
any number of parameters.

The + operator concatenates two strings: @strcat(a,b) is the same as
a + b

@strcut(input,pattern) will match the regular expression in the
pattern parameter against the supplied input string and return the
section that matches the first matching parenthesised group.

For example:

This will output the message: Your name is John.

@strjoin(array, delimiter) will combine the elements of an array
into a single string separated by a given delimiter. The array
passed to @strjoin must be of a primitive type (string, int, float,
or boolean). It will not join the contents of an array of files.

Example:

This will print the string “this is a test”.

@strsplit(input,pattern) will split the input string based on
separators that match the given pattern and return a string array.

Example:

This will output one word of the sentence on each line (though not
necessarily in order, due to the fact that foreach iterations execute in
parallel).

@toInt(input) will parse its input string into an integer. This can be
used with @arg to pass input parameters to a Swift script as
integers.

@toFloat(input) will parse its input string into a floating point number. This can be
used with @arg to pass input parameters to a Swift script as
floating point numbers.

@toString(input) will parse its input into a string. Input can be an int, float, string,
or boolean.

@length(array) will return the length of an array in Swift. This function will wait for all elements in the array to be written before returning the length.

@java(class_name, static_method, method_arg) will call a java static method of the class class_name.

readData will read data from a specified file and assign it to Swift variable. The format of the input file is
controlled by the type of the return value. For scalar return types, such as
int, the specified file should contain a single value of that type. For arrays
of scalars, the specified file should contain one value per line. For complex types
of scalars, the file should contain two rows. The first row should be structure
member names separated by whitespace. The second row should be the
corresponding values for each structure member, separated by whitespace, in the
same order as the header row. For arrays of structs, the file should contain a
heading row listing structure member names separated by whitespace. There
should be one row for each element of the array, with structure member elements
listed in the same order as the header row and separated by whitespace. The following example shows how readData() can be used to populate an array of Swift struct-like complex type:

Where the contents of the “emps.txt” file are:

This will result in the array “emps” with 3 members. This can be processed within a Swift script using the foreach construct as follows:

readStructured will read data from a specified file, like readdata, but
using a different file format more closely related to that used by the
ext mapper.

Input files should list, one per line, a path into a Swift structure,
and the value for that position in the structure:

which can be read into a structure defined like this:

(since Swift 0.7, was readData2(deprecated))

trace will log its parameters. By default these will appear on both
stdout and in the run log file. Some formatting occurs to produce the
log message. The particular output format should not be relied upon.
(since Swift 0.4)

tracef(spec, variable list) will log its parameters as formatted
by the formatter spec. spec must be a string. Checks the type of
the specifiers arguments against the variable list and allows for
certain escape characters.

Example:

Specifiers:

Escape sequences:

writeData will write out data structures in the format described for
readData

maxSubmitRate limits the maximum rate of job submission, in jobs per
second. For example:

will limit job submission to 0.2 jobs per second (or equivalently, one
job every five seconds).

jobThrottle allows the job throttle factor (see Swift property
throttle.score.job.factor to be
set per site.

initialScore allows the initial score for rate limiting and site
selection to be set to a value other than 0.

delayBase controls how much a site will be delayed when it performs
poorly. With each reduction in a sites score by 1, the delay between
execution attempts will increase by a factor of delayBase.

status.mode allows the status.mode property to be set per-site instead
of for an entire run. See the Swift configuration properties section for
more information. (since Swift 0.8)

storagesize limits the amount of space that will be used on the remote
site for temporary files. When more than that amount of space is used,
the remote temporary file cache will be cleared using the algorithm
specified in the caching.algorithm property.

wrapperInterpreter – The wrapper interpreter indicates the command
(executable) to be used to run the Swift wrapper script. The default is
“/bin/bash” on Unix sites and “cscript.exe” on Windows sites.

wrapperInterpreterOptions – Allows specifying additional options to the
executable used to run the Swift wrapper. The defaults are no options on
Unix sites and “Nologo” on Windows sites.

wrapperScript – Specifies the name of the wrapper script to be used on a
site. The defaults are “_swiftwrap” on Unix sites and “_swiftwrap.vbs”
on Windows sites. If you specify a custom wrapper script, it must be
present in the “libexec” directory of the Swift installation.

cleanupCommand Indicates the command to be run at the end of a Swift
run to clean up the run directories on a remote site. Defaults are
“/bin/rm” on Unix sites and “cmd.exe” on Windows sites

cleanupCommandOptions Specifies the options to be passed to the
cleanup command above. The options are passed in the argument list to
the cleanup command. After the options, the last argument is the
directory to be deleted. The default on Unix sites is “-rf”. The default
on Windows sites is [“/C”, “del”, “/Q”].

maxwalltime specifies a walltime limit for each job, in minutes.

The following formats are recognized:

Example:

When replication is enabled (see replication), then
walltime will also be enforced at the Swift client side: when a job has
been active for more than twice the maxwalltime, Swift will kill the job
and regard it as failed.

When clustering is used, maxwalltime will be used to select which jobs
will be clustered together. More information on this is available in the
clustering section.

When coasters as used, maxwalltime influences the default coaster
worker maxwalltime, and which jobs will be sent to which workers. More
information on this is available in the coasters section.

queue is used by the PBS, GRAM2 and GRAM4 providers. This profile
entry specifies which queue jobs will be submitted to. The valid queue
names are site-specific.

host_types specifies the types of host that are permissible for a job
to run on. The valid values are site-specific. This profile entry is
used by the GRAM2 and GRAM4 providers.

condor_requirements allows a requirements string to be specified when
Condor is used as an LRM behind GRAM2. Example:

slots When using coasters, this parameter specifies the
maximum number of jobs/blocks that the coaster scheduler will have
running at any given time. The default is 20.

jobsPerNode – This parameter determines how many coaster workers are
started one each compute node. The default value is 1.

nodeGranularity – When allocating a coaster worker block, this parameter
restricts the number of nodes in a block to a multiple of this value.
The total number of workers will then be a multiple of workersPerNode *
nodeGranularity. The default value is 1.

allocationStepSize – Each time the coaster block scheduler computes a
schedule, it will attempt to allocate a number of slots from the number
of available slots (limited using the above slots profile). This
parameter specifies the maximum fraction of slots that are allocated in
one schedule. Default is 0.1.

lowOverallocation – Overallocation is a function of the walltime of a
job which determines how long (time-wise) a worker job will be. For
example, if a number of 10s jobs are submitted to the coaster service,
and the overallocation for 10s jobs is 10, the coaster scheduler will
attempt to start worker jobs that have a walltime of 100s. The
overallocation is controlled by manipulating the end-points of an
overallocation function. The low endpoint, specified by this parameter,
is the overallocation for a 1s job. The high endpoint is the
overallocation for a (theoretical) job of infinite length. The
overallocation for job sizes in the [1s, +inf) interval is determined
using an exponential decay function: overallocation(walltime) = walltime
* (lowOverallocation – highOverallocation) * exp(-walltime *
overallocationDecayFactor) + highOverallocation The default value of
lowOverallocation is 10.

highOverallocation – The high overallocation endpoint (as described
above). Default: 1

overallocationDecayFactor – The decay factor for the overallocation
curve. Default 0.001 (1e-3).

spread – When a large number of jobs is submitted to the a coaster
service, the work is divided into blocks. This parameter allows a rough
control of the relative sizes of those blocks. A value of 0 indicates
that all work should be divided equally between the blocks (and blocks
will therefore have equal sizes). A value of 1 indicates the largest
possible spread. The existence of the spread parameter is based on the
assumption that smaller overall jobs will generally spend less time in
the queue than larger jobs. By submitting blocks of different sizes,
submitted jobs may be finished quicker by smaller blocks. Default: 0.9.

reserve – Reserve time is a time in the allocation of a worker that sits
at the end of the worker time and is useable only for critical
operations. For example, a job will not be submitted to a worker if it
overlaps its reserve time, but a job that (due to inaccurate walltime
specification) runs into the reserve time will not be killed (note that
once the worker exceeds its walltime, the queuing system will kill the
job anyway). Default 10 (s).

maxnodes – Determines the maximum number of nodes that can be allocated
in one coaster block. Default: unlimited.

maxtime – Indicates the maximum walltime, in seconds, that a coaster
block can have.
Default: unlimited.

remoteMonitorEnabled – If set to “true”, the client side will get a
Swing window showing, graphically, the state of the coaster scheduler
(blocks, jobs, etc.). Default: false

internalhostname – If the head node has multiple network interfaces,
only one of which is visible from the worker nodes. The choice of
which interface is the one that worker nodes can connect to is a
matter of the particular cluster. This must be set in the your
sites file to clarify to the workers which exact interface on the
head node they are to try to connect to.

Profile keys set in the env namespace will be set in the unix
environment of the executed job. Some environment variables influence
the worker-side behaviour of Swift:

PATHPREFIX – set in env namespace profiles. This path is prefixed onto
the start of the PATH when jobs are executed. It can be more useful
than setting the PATH environment variable directly, because setting
PATH will cause the execution site’s default path to be lost.

SWIFT_JOBDIR_PATH – set in env namespace profiles. If set, then Swift
will use the path specified here as a worker-node local temporary
directory to copy input files to before running a job. If unset, Swift
will keep input files on the site-shared filesystem. In some cases,
copying to a worker-node local directory can be much faster than having
applications access the site-shared filesystem directly.

SWIFT_EXTRA_INFO – set in env namespace profiles. If set, then Swift
will execute the command specified in SWIFT_EXTRA_INFO on execution
sites immediately before each application execution, and will record the
stdout of that command in the wrapper info log file for that job. This
is intended to allow software version and other arbitrary information
about the remote site to be gathered and returned to the submit side.
(since Swift 0.9)

SWIFT_GEN_SCRIPTS – set in the env namespace profiles. This variable
just needs to be set, it doesn’t matter what it is set to. If set, then Swift
will keep the script that was used to execute the job in the job directory.
The script will be called run.sh and will have the command line that Swift
tried to execute with.

To set a profile setting based on the value of a Swift variable, you
must use dynamic profiles. This allows you to set profile settings in
the globus namespace.

This would be equivalent to the sites file settings:

except, of course, the number of MPI processes may not be dynamically
set by the value of Swift variable p in the sites file.

Additional beneficial use cases of dynamic profiles may be to set the
maxwalltime or queue profile settings.

Swift profiles, generally speaking, are converted into Java CoG
“attributes”, which are attached to each CoG task. Thus, when reading
the log or debugging, look for messages regarding “attributes”.

Each pool element must have a handle attribute, giving a symbolic
name for the site. This can be any name, but must correspond to entries
for that site in the transformation catalog.

Optionally, the gridlaunch attribute can be used to specify the path
to kickstart on the site.

Each pool must specify a file transfer method, an execution method and
a remote working directory. Optionally, profile settings can
be specified.

Transfer methods are specified with either the <gridftp> element or
the <filesystem> element.

To use gridftp or local filesystem copy, use the <gridftp> element:

The url attribute may specify a GridFTP server, using the gsiftp URI
scheme; or it may specify that filesystem copying will be used (which
assumes that the site has access to the same filesystem as the
submitting machine) using the URI local://localhost.

Filesystem access using scp (the SSH copy protocol) can be specified
using the <filesystem> element:

For additional ssh configuration information, see the ssh execution
provider documentation below.

Filesystem access using CoG coasters can be also be
specified using the <filesystem> element. More detail about
configuring that can be found in the CoG coasters section.

Execution methods may be specified either with the <jobmanager> or
<execution> element.

The <jobmanager> element can be used to specify execution through
GRAM2. For example,

The universe attribute should always be set to vanilla. The url
attribute should specify the name of the GRAM2 gatekeeper host, and the
name of the jobmanager to use. The major attribute should always be set
to 2.

The <execution> element can be used to specify execution through other
execution providers:

To use GRAM4, specify the gt4 provider. For example:

The url attribute should specify the GRAM4 submission site. The
jobmanager attribute should specify which GRAM4 jobmanager will be used.

For local execution, the local provider should be used, like this:

For PBS execution, the pbs provider should be used:

The GLOBUS::queue profile key can be used to
specify which PBS queue jobs will be submitted to.

For execution through a local Condor installation, the condor provider
should be used. This provider can run jobs either in the default vanilla
universe, or can use Condor-G to run jobs on remote sites.

When running locally, only the <execution> element needs to be specified:

When running with Condor-G, it is necessary to specify the Condor grid
universe and the contact string for the remote site. For example:

For execution through SSH, the ssh provider should be used:

with configuration made in ~/.ssh/auth.defaults with the string
www11.i2u2.org changed to the appropriate host name:

For execution using the CoG Coaster mechanism, the coaster
provider should be used:

More details about configuration of coasters can be found in the section
on coasters.

The workdirectory element specifies where on the site files can be
stored.

This file must be accessible through the transfer mechanism specified in
the <gridftp> element and also mounted on all worker nodes that will
be used for execution. A shared cluster scratch filesystem is
appropriate for this.

Profile keys can be specified using the <profile> element.
For example:

The site catalog format is an evolution of the VDS site catalog format
which is documented here
http://vds.uchicago.edu/vds/doc/userguide/html/H_SiteCatalog.html.

It is often useful to set environment variables when running an application.
This can be accomplished using env in the profile entry. For example,
the following application sets an environment variable called R_LIBS to
/home/user/R_libs.

Multiple profile entries can be added by using a semicolon. The example below
sets two environment variables: R_LIBS and R_HOME.

If an application procedure execution fails, Swift will attempt that
execution again repeatedly until it succeeds, up until the limit defined
in the execution.retries configuration property.

Site selection will occur for retried jobs in the same way that it
happens for new jobs. Retried jobs may run on the same site or may run
on a different site.

If the retry limit execution.retries is reached for an application
procedure, then that application procedure will fail. This will cause
the entire run to fail – either immediately (if the lazy.errors
property is false) or after all other possible work has been attempted
(if the lazy.errors property is true).

If a run fails, Swift can resume the program from the point of failure.
When a run fails, a restart log file will be left behind in a file named
using the unique job ID and a .rlog extension. This restart log can
then be passed to a subsequent Swift invocation using the -resume
parameter. Swift will resume execution, avoiding execution of
invocations that have previously completed successfully. The Swift
source file and input data files should not be modified between runs.

Every run creates a restart log file with a named composed of the file
name of the workflow being executed, an invocation ID, a numeric ID, and
the .rlog extension. For example, example.swift, when executed,
could produce the following restart log file:
example-ht0adgi315l61.0.rlog. Normally, if the run completes
successfully, the restart log file is deleted. If however the workflow
fails, swift can use the restart log file to continue execution from a
point before the failure occurred. In order to restart from a restart
log file, the -resume logfile argument can be used after the
Swift script file name. Example:

When an execution job has been waiting in a site queue for a certain
period of time, Swift can resubmit replicas of that job (up to the limit
defined in the replication.limit configuration property). When any of
those jobs moves from queued to active state, all of the other replicas
will be cancelled.

This is intended to deal with situations where some sites have a
substantially longer (sometimes effectively infinite) queue time than
other sites. Selecting those slower sites can cause a very large delay
in overall run time.

Replication can be enabled by setting the replication.enabled
configuration property to true. The maximum number of replicas that
will be submitted for a job is controlled by the replication.limit
configuration property.

When replication is enabled, Swift will also enforce the maxwalltime
profile setting for jobs as documented in the profiles section.

Coasters are the Swift’s implementation of pilot job abstraction.

In many applications, Swift performance can be greatly enhanced by the use of
coasters. Coasters provide a low-overhead job submission and file transfer
mechanism suited for the execution of jobs and the transfer of files for which
other grid protocols such as GRAM and GridFTP are poorly suited.

Much of the overhead associated with other grid protocols (such as
authentication and authorization, and allocation of worker nodes by the site’s
local resource manager) is reduced, because that overhead is associated with
the allocation of a coaster pilot or coaster worker, rather than with every
Swift-level procedure invocation; potentially hundreds or thousands of
Swift-level procedure invocations can be run through a single worker. Coasters
can be configured for two purposes: job execution and file staging. In
practice, the Swift script remains the same while working with coasters. A
detailed description of coaster mechanism is explained in the next section.

Coasters run at the task management layer logically under the Swift script. The
jobs and data movement requirements resulting after the interpretation of a
Swift script are handled by the coasters. The coaster mechanism submits a pilot
job using some other execution mechanism such as GRAM, SGE or PBS scheduler,
and for each worker node that will be used in a remote cluster, it submits a
worker job, again using some other execution mechanism such as GRAM. Details on
the design of the coaster mechanism can be found here:
http://wiki.cogkit.org/wiki/Coasters. The pilot job manages file transfers
and the dispatch of execution jobs to workers.

To use for job execution, specify a sites.xml execution element like this:

The jobmanager string contains more detail than with other providers. It
contains either two or three colon separated fields: 1:the provider to
be use to execute the coaster pilot job – this provider will submit from
the Swift client side environment. Commonly this will be one of the GRAM
providers; 2: the provider to be used to execute coaster worker jobs.
This provider will be used to submit from the coaster pilot job
environment, so a local scheduler provider can sometimes be used instead
of GRAM. 3: optionally, the jobmanager to be used when submitting worker
job using the provider specified in field 2.

To use for file transfer, specify a sites.xml filesystem element like this:

The url parameter should be a pseudo-URI formed with the URI scheme
being the name of the provider to use to submit the coaster pilot job,
and the hostname portion being the hostname to be used to execute the
coaster pilot job. Note that this provider and hostname will be used for
execution of a coaster pilot job, not for file transfer; so for example,
a GRAM endpoint should be specified here rather than a GridFTP endpoint.

Coasters are affected by the following profile settings, which are
documented in the Globus namespace profile section:

This section presents information on coaster configuration parameters and their effect on application and scheduler jobs submitted by Swift. In order to achieve optimal performance, the number of application tasks must correspond to the total number of coaster workers. Coaster configuration parameters influence both application tasks and the LRM (Local Resource Manager) jobs that are submitted by Swift. Specifically, the following quantities are influenced by coasters configuration parameters:

Where nforeach is the number of independent foreach loops appearing in a Swift script.

Considering the above factors, the following paramter expressions must match in order for a Swift run to be optimal:

If you have a UChicago Computation Institute account, run this command
in your submit directory after each run. It will copy all your logs and
kickstart records into a directory at the CI for reporting, usage
tracking, support and debugging.

TeraGrid users with no default project or with several project
allocations can specify a project allocation using a profile key in the
site catalog entry for a TeraGrid site:

More information on the TeraGrid allocations process can be found here
http://www.teragrid.org/userinfo/access/allocations.php.

There are several ways to run MPI jobs under Swift. Two will be discussed here –
calling mpiexec from a wrapper script, and using the MPICH/coasters interface.

Swift has the ability to run on a Windows machine, as well as the
ability to submit jobs to a Windows site (provided that an appropriate
provider is used).

In order to launch Swift on Windows, use the provided batch file
(swift.bat). In certain cases, when a large number of jar libraries are
present in the Swift lib directory and depending on the exact location
of the Swift installation, the classpath environment variable that the
Swift batch launcher tries to create may be larger than what Windows can
handle. In such a case, either install Swift in a directory closer to
the root of the disk (say, c:\swift) or remove non-essential jar files
from the Swift lib directory.

Due to the large differences between Windows and Unix environments,
Swift must use environment specific tools to achieve some of its goals.
In particular, each Swift executable is launched using a wrapper script.
This script is a Bourne Shell script. On Windows machines, which have no
Bourne Shell interpreter installed by default, the Windows Scripting
Host is used instead, and the wrapper script is written in VBScript.
Similarly, when cleaning up after a run, the “/bin/rm” command available
in typical Unix environments must be replaced by the “del” shell command.

It is important to note that in order to select the proper set of tools
to use, Swift must know when a site runs under Windows. To inform Swift
of this, specify the “sysinfo” attribute for the “pool” element in the
site catalog. For example:

Collective Data Management (CDM) is a set of optimizations in Swift to
improve data access patterns by Swift. In particular, it can be used
to avoid data staging (extra file copies) on an HPC system or cluster
with a shared file system.

A CDM policy file contains four space separated fields as follows:
. The keyword rule
. The filename pattern expressed as a regexp
. The rulename: DIRECT, GATHER, BROADCAST etc.
. The path where to look for the files (optional)

To troubleshoot CDM, check the Swift log and the wrapper logs
(*-info files). These will indicate the CDM policy that Swift finds
for each file and resulting action, such as skipping stage in for
DIRECT and the creation of links. Common errors include specifying
the wrong directory which will result in an error that the file was
not found, or that a link could not be created.

A Swift log file is typically a text file with the name of the Swift run and
its timestamp in the filename and an extension “.log”. In addition, a “.rlog”
file is Swift’s resume log which is used by Swift when a run is resumed using
the “-resume” option. The .rlog file is only for Swift’s internal purpose and
not to be interpreted by the user.

Each line in the log file typically consists of three parts. The first part
is the timestamp, the second is the type of log message and the third is the
message itself. The types of log messages follows the java log4j standard types
of TRACE, DEBUG, INFO, WARN, ERROR and FATAL.

This section lists the various Swift log messages and explains the meaning and
likely interpretation of those messages. Note that the list is not
comprehensive at this time. Also note that we will ignore the timestamps here.