From: ftp-bimas.cit.nih.gov                       Last mod: October 6, 1997

            Running FASTLINK 3.0P or later in parallel on networks
            of workstations

|*| Introduction
    ------------
    TreadMarks is a parallel programming system developed at Rice
University to allow programs written in a shared-memory style to run on
network of unix workstations.  The TreadMarks project is separate from the
FASTLINK project, but we collaborate on making FASTLINK run in parallel on
top of TreadMarks.  References for TreadMarks can be found in
README.parallel. 


|*| Retrieving and Installing TreadMarks
    ------------------------------------
    TreadMarks is not available by ftp without getting a license first.
Write to treadmarks@ece.rice.edu and to schaffer@helix.nih.gov if you
are interested and we will arrange with you to do the installation.

    You can obtain a free 30-day demo copy of TreadMarks or you can
get a normal license. Licensing and pricing are available by writing to
treadmarks@ece.rice.edu.


|*| Hooking TreadMarks and FASTLINK together
    ----------------------------------------
The file called Makefile has two variables that need to be set:

TmkDIR should be set to the root directory of your TreadMarks installation
The ARCH flag needs to be set for your specific architecture
The PARINCLPATH flag line that reads
    -I$(TmkDIR)/include -I/usr/local/include
needs to be uncommented.
See README.Makefile for more details.

To compile parallel ilink, mlink, or linkmap, do

  make ilink.udp
  make mlink.udp
  make linkmap.udp

respectively. 

These will put the corresponding executables wherever the BINDIR flag
points to.


|*| Running TreadMarks FASTLINK
    ---------------------------
You would typically want to make a soft link between the regular name
without the udp extension in your data directory and the actual
executable. E.g.

  ln -s ../bin/ilink.udp ilink

There are some compilation flags you may want to set to prepare for a
run. See README.Makefile or the Makefile itself for instructions.


|*| Running TreadMarks FASTLINK, Command line flags
    -----------------------------------------------
Specific command line flags are discussed below.  First, though, it is
important to note how TreadMarks and FASTLINK distinguish which flags are
meant for which program.  Every command line *must* contain the string `--'
(without the quotes).  This string delimits FASTLINK flags from TreadMarks
flags.  All arguments *before* the `--' are read by FASTLINK, while those
appearing *after* the `--' are read by TreadMarks.

For example, in the command:

  linkmap -w 40 -- -f machines

the "-w 40" is seen by FASTLINK, while the "-f machines" is seen by
TreadMarks. 


|*| Running TreadMarks FASTLINK, Specifying number of processors
    ------------------------------------------------------------
One small modification is needed either in the command line (if you call
ilink, linkmap, or mlink directly) or in the lcp-produced shell script. At
the line where the main program is invoked, the string

  -- -f machines 

is appended. 

E.g.

  linkmap

becomes

  linkmap -- -f machines

When modifying lcp-produced scripts be careful that the first occurrence of
the string ilink, linkmap, or mlink is a parameter to lsp. It is the second
occurrence which is actually the call to the program.

In the data directory you need to have a file called

   machines

that specifies which machines should be used. The format is one machine
name per line.  This is the only thing you need to change if you want to
change which machines the program runs on.


|*| Running TreadMarks FASTLINK, Specifying maxworkingset
    -----------------------------------------------------
You may also specify a value for the variable maxworkingset, which
represents the maximum number of people active during the analysis
(sometimes known as maximum cutset).  In FASTLINK 3.0P, maxworkingset
is estimated automatically at runtime. In some pedigrees
with loops, the estimate is unnecessarily high, so you may wish to override
the estimate with a different value.

If you use the automatic estimate of maxworkingset, and the code complains
that this estimate is too low, you have hit a bug and should report it (see
README.bugreport). However, you can still use the -w flag to work
around the bug, while I fix it.


For example, to run ILINK with maxworkingset defined to 40, you would type:

  ilink -w 40 -- -f machines

The error message you would encounter if maxworkingset is too low will
report what the current value is.  You may try incrementally larger values
until the run succeeds.


|*| Running TreadMarks FASTLINK, Analyzing memory usage
    ---------------------------------------------------
Depending on your system configuration, you may not be able to complete
arbitrarily large (in terms of memory usage) FASTLINK runs.  As described
in README.makefile, there are different levels of memory usage for
FASTLINK.

The -m option for parallel FASTLINK provides a simple way to determine
shared memory requirements for a specific run.  When you run with this
option, FASTLINK will do some brief i/o and computation, and then exit
(before starting the actual linkage analysis) with a diagnostic message.
Because TreadMarks itself is never actually started up for this kind of
run, you must also specify the number of processors you plan to run on, so
FASTLINK can properly calculate memory requirements.  You can specify
number of processors with

  -n <nprocs>

where <nprocs> is the number of processors.  Also note that in this case,
the `--' is not necessary.

For example, to calculate memory usage for a LINKMAP run on 2 processors,
you would type:

  linkmap -n 2 -m

A sample run of LINKMAP with this command yielded:

  LINKMAP is currently compiled with PRECOMPUTE=1.
  This run will require at least:

          7004071 bytes of shared memory on 2 processor(s)
          6421943 bytes of shared memory on 1 processor(s)

  Recompiling with PRECOMPUTE=0 would yield:

          6733607 bytes of shared memory on 2 processor(s)
          6094621 bytes of shared memory on 1 processor(s)

  Please refer to the README.makefile and README.TreadMarks for details.

You can see from this message the difference in memory usage between
compiling with PRECOMPUTE=0 and PRECOMPUTE=1.  This can be useful when
determining whether or not you can expect to compute a given run on your
system.


As with the -f flag above, -w, -n, and -m would need to be added to your
lcp-produced script if you use them in conjunction with a script.


|*| Parallel FASTLINK, Running times
    --------------------------------
Due to the way theta evaluations are done in parallel, unlike the
sequential version, output will not appear on the screen after each theta
is complete.  However, you will see periodic reports of execution times for
each group of thetas as they are evaluated.  A sample run might show:

  Execution time (!parallelThetas) =   0.099
  Execution time (parallelThetas) for 1 =   0.052
  Execution time (parallelThetas) for 2 =   0.896
  Execution time (parallelThetas) for 3 =   0.096
  Execution time (!parallelThetas) =   0.086
  Elapsed time: 1.28 seconds

Each of the "Execution time" statements will appear one at a time as
computation progresses.  The difference between parallelThetas and
!parallelThetas has to do with whether all processors are working together
on a single theta, or whether they are working independently on different
thetas.  The "Elapsed time" statement shows the total execution time for
the entire run. This is wall-clock time.


