source: CMT/v1r16p20040901/doc/tbroadcast.html @ 1

Last change on this file since 1 was 1, checked in by arnault, 19 years ago

Import all tags
File size: 5.5 KB
Line 
1<!--
2//-----------------------------------------------------------
3// Copyright Christian Arnault LAL-Orsay CNRS
4// arnault@lal.in2p3.fr
5// See the complete license in cmt_license.txt "http://www.cecill.info".
6//-----------------------------------------------------------
7-->
8
9<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
10<html>
11  <head>
12    <title>MultiThreaded broadcast utility</title>
13  </head>
14
15  <font face="Arial, Helvetica" color="#000000">
16
17    <body bgcolor="#ffffff" link="#550088" alink="#007777" alink="#007777">
18      <h1>MultiThreaded broadcast utility</h1>
19
20      <p>This document presents a multithreaded version of the cmt
21      broadcast facility.</p>
22
23      <p>This is a standalone program and NOT a new feature of CMT. If
24        it appears to be safe and reliable, then of course it may become
25        part of the plain CMT.</p>
26     
27      <p>This program (named tbroadcast - for "threaded broadcast")
28        basically operates very much like "cmt broadcast", ie it accepts
29        most of the same options and produces the same output.</p>
30     
31      <p>Report on the CMT mailing list your experience, bugs, comments,
32        ... and measurements ! Of course the expected improvement will
33        largely depend on the type of command and on the number of
34        processors...</p>
35     
36      <hr />
37      <h2>Principle</h2>
38     
39      <blockquote>
40       
41        The use network forms a Directed Acyclic Graph [DAG] (or if
42        there are cycles, they are detected and properly handled).
43       
44        <p><b>tbroadcast</b> first gets all "leaves" of the use graph, and
45          launches the requested command onto them in parallel in
46          individual [POSIX]threads.</p>
47       
48        <p>Then, once all children of a non-leaf package are completed,
49          the requested command is launched onto this non-leaf package
50          [still in an individual thread].</p>
51       
52        <p>This is recursively repeated until all packages have been
53          reached.</p>
54       
55        <p>At any time the number of active threads is controlled so as
56          to never exceed a limit [set to 20 threads by default but can be
57          modified using the <b>-threads=n</b> option]. A local scheduler
58          maintains a queue of threads when the limit is reached.</p>
59       
60        <p>The individual thread activity consists in running the
61          requested command - as usual - in the context of the reached
62          package. The difference with the sequential <b>cmt broadcast</b>
63          of course is that the output of the run is accumulated and
64          displayed at once when every action is completed [to avoid
65          intermixing of outputs from different threads].</p>
66       
67        <p>When one activity returns an error, the error status is
68          propagated to its parent which will prematurely complete, but
69          not before all other simultaneously running threads are actually
70          completed.</p>
71       
72        <p>It's only when the threads of ALL children are completed
73          (with success or error) that a node either starts (if all
74          children successfully completed) or immediately terminates (if
75          one of its children failed).</p>
76       
77      </blockquote>
78     
79      <hr />
80     
81      <h2>How to use it:</h2>
82     
83      <blockquote>
84       
85        <ul>
86         
87          <li> The source kit is available for Linux platforms only from CVS:
88           
89            <pre>
90&gt; export CVSROOT=:pserver:anonymous@cvsserver.lal.in2p3.fr:/projects/cvs
91&gt; cvs login
92[no password]
93&gt; cd /somewhere
94&gt; cvs co -d tbroadcast CMT/tbroadcast
95&gt; cd tbroadcast/cmt
96&gt; cmt config
97&gt; gmake
98            </pre>
99   
100          </li>
101   
102          <li> The application is also available for Linux platforms
103          at CERN:
104           
105            <pre>
106${CMTROOT}/../tbroadcast/v1/Linux/tbroadcast
107            </pre>
108   
109          </li>
110   
111          <li> The options are:
112     
113            <pre>
114&gt; tbroadcast [option...] [shell-command]
115
116    where option can be:
117
118      -select="&lt;selection-list&gt;"
119      -exclude="&lt;exclusion-list&gt;"
120      -begin=&lt;begin-pattern&gt;
121      -global                   Reach packages in all CMTPATH items
122      -threads=&lt;n&gt;              Change the maximum thread number [default is 20]
123
124      &lt;selection-list&gt; is a space-separated list of sub-strings that actual paths of
125                       all reached packages must match
126
127      &lt;exclusion-list&gt; is a space-separated list of sub-strings that actual paths of
128                       all reached packages must NOT match
129
130      &lt;begin-pattern&gt;  is a sub-string matching a package path for beginning the broadcast loop
131
132      -select=, -exclude=, -begin= can be combined
133
134      -select=, -exclude= can be accumulated
135
136            </pre>
137
138            <p>If the shell-command is omitted, the broadcast loop
139            simply traverses the graph, reaching all packages and
140            doing nothing. This is useful to test the traversal.</p>
141
142          </blockquote>
143
144        <hr />
145        <h2>Examples</h2>
146
147        <blockquote>
148
149          <i>This example shows how to use tbroadcast in the context of the Atlas project</i>
150
151          <pre>&gt; source setup.sh -tag=7.5.0
152&gt; cd .....dist/7.5.0/AtlasRelease/*/cmt
153&gt alias tbroadcast=$CMTROOT/../tbroadcast/v1/Linux/tbroadcast
154&gt; tbroadcast -select=/Database/
155&gt; tbroadcast -global
156&gt; tbroadcast -begin=/StoreGate/
157
158          </pre>
159
160        </blockquote>
161
162        <hr>
163    <address><a href="mailto:arnault@lal.in2p3.fr">Christian Arnault</a></address>
164<!-- Created: Fri Jan 30 12:20:51 CET 2004 -->
165<!-- hhmts start -->
166Last modified: Fri Jan 30 12:50:46 CET 2004
167<!-- hhmts end -->
168    </body>
169  </font>
170</html>
Note: See TracBrowser for help on using the repository browser.