source: cmtsite/tbroadcast.html@ 93

Last change on this file since 93 was 73, checked in by arnault, 20 years ago

Transfer cmtsite documents into SVN repository
  • Property svn:eol-style set to native
File size: 5.3 KB
RevLine 
[2]1<!--
2//-----------------------------------------------------------
3// Copyright Christian Arnault LAL-Orsay CNRS
4// arnault@lal.in2p3.fr
5// See the complete license in cmt_license.txt "http://www.cecill.info".
6//-----------------------------------------------------------
7-->
8
9<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
10<html>
11 <head>
12 <title>MultiThreaded broadcast utility</title>
13 </head>
14
15 <font face="Arial, Helvetica" color="#000000">
16
17 <body bgcolor="#ffffff" link="#550088" alink="#007777" alink="#007777">
18 <h1>MultiThreaded broadcast utility</h1>
19
20 <p>This document presents a multithreaded version of the cmt
21 broadcast facility.</p>
22
23 <p>This is a standalone program and NOT a new feature of CMT. If
24 it appears to be safe and reliable, then of course it may become
25 part of the plain CMT.</p>
26
27 <p>This program (named tbroadcast - for "threaded broadcast")
28 basically operates very much like "cmt broadcast", ie it accepts
29 most of the same options and produces the same output.</p>
30
31 <p>Report on the CMT mailing list your experience, bugs, comments,
32 ... and measurements ! Of course the expected improvement will
33 largely depend on the type of command and on the number of
34 processors...</p>
35
36 <hr />
37 <h2>Principle</h2>
38
39 <blockquote>
40
41 The use network forms a Directed Acyclic Graph [DAG] (or if
42 there are cycles, they are detected and properly handled).
43
44 <p><b>tbroadcast</b> first gets all "leaves" of the use graph, and
45 launches the requested command onto them in parallel in
46 individual [POSIX]threads.</p>
47
48 <p>Then, once all children of a non-leaf package are completed,
49 the requested command is launched onto this non-leaf package
50 [still in an individual thread].</p>
51
52 <p>This is recursively repeated until all packages have been
53 reached.</p>
54
55 <p>At any time the number of active threads is controlled so as
56 to never exceed a limit [set to 20 threads by default but can be
57 modified using the <b>-threads=n</b> option]. A local scheduler
58 maintains a queue of threads when the limit is reached.</p>
59
60 <p>The individual thread activity consists in running the
61 requested command - as usual - in the context of the reached
62 package. The difference with the sequential <b>cmt broadcast</b>
63 of course is that the output of the run is accumulated and
64 displayed at once when every action is completed [to avoid
65 intermixing of outputs from different threads].</p>
66
67 <p>When one activity returns an error, the error status is
68 propagated to its parent which will prematurely complete, but
69 not before all other simultaneously running threads are actually
70 completed.</p>
71
72 <p>It's only when the threads of ALL children are completed
73 (with success or error) that a node either starts (if all
74 children successfully completed) or immediately terminates (if
75 one of its children failed).</p>
76
77 </blockquote>
78
79 <hr />
80
81 <h2>How to use it:</h2>
82
83 <blockquote>
84
85 <ul>
86
87 <li> The source kit is available for Linux platforms only from CVS:
88
89 <pre>
90&gt; export CVSROOT=:pserver:anonymous@cvsserver.lal.in2p3.fr:/projects/cvs
91&gt; cvs login
92[no password]
93&gt; cd /somewhere
94&gt; cvs co -d tbroadcast CMT/tbroadcast
95&gt; cd tbroadcast/cmt
96&gt; cmt config
97&gt; gmake
98 </pre>
99
100 </li>
101
102 <li> The application is also available for Linux platforms
103 at CERN:
104
105 <pre>
106${CMTROOT}/../tbroadcast/v1/Linux/tbroadcast
107 </pre>
108
109 </li>
110
111 <li> The options are:
112
113 <pre>
114&gt; tbroadcast [option...] [shell-command]
115
116 where option can be:
117
118 -select="&lt;selection-list&gt;"
119 -exclude="&lt;exclusion-list&gt;"
120 -begin=&lt;begin-pattern&gt;
121 -global Reach packages in all CMTPATH items
122 -threads=&lt;n&gt; Change the maximum thread number [default is 20]
123
124 &lt;selection-list&gt; is a space-separated list of sub-strings that actual paths of
125 all reached packages must match
126
127 &lt;exclusion-list&gt; is a space-separated list of sub-strings that actual paths of
128 all reached packages must NOT match
129
130 &lt;begin-pattern&gt; is a sub-string matching a package path for beginning the broadcast loop
131
132 -select=, -exclude=, -begin= can be combined
133
134 -select=, -exclude= can be accumulated
135
136 </pre>
137
138 <p>If the shell-command is omitted, the broadcast loop
139 simply traverses the graph, reaching all packages and
140 doing nothing. This is useful to test the traversal.</p>
141
142 </blockquote>
143
144 <hr />
145 <h2>Examples</h2>
146
147 <blockquote>
148
149 <i>This example shows how to use tbroadcast in the context of the Atlas project</i>
150
151 <pre>&gt; source setup.sh -tag=7.5.0
152&gt; cd .....dist/7.5.0/AtlasRelease/*/cmt
153&gt alias tbroadcast=$CMTROOT/../tbroadcast/v1/Linux/tbroadcast
154&gt; tbroadcast -select=/Database/
155&gt; tbroadcast -global
156&gt; tbroadcast -begin=/StoreGate/
157
158 </pre>
159
160 </blockquote>
161
162 <hr>
163 <address><a href="mailto:arnault@lal.in2p3.fr">Christian Arnault</a></address>
164<!-- Created: Fri Jan 30 12:20:51 CET 2004 -->
165<!-- hhmts start -->
166Last modified: Fri Jan 30 12:50:46 CET 2004
167<!-- hhmts end -->
168 </body>
169 </font>
170</html>
Note: See TracBrowser for help on using the repository browser.