source: BAORadio/AmasNancay/trunk/analysis-scriptsAndprg-description.txt@ 589

Last change on this file since 589 was 575, checked in by campagne, 14 years ago

introduce SCRIPTPATH environment variable (jec)
File size: 8.8 KB
RevLine 
[507]1Title: Status of the programs and scripts used for Amas@Nancay analysis at CCIN2P3
[573]2Revision: 11/10/11 (JEC)
[507]3Author: J.E Campagne 22/7/11
4----------------------------
5
6Intro:
7All these tools are located for the while in J.E Campagne private directory, so they are not
8yet available to public usage and this note is essentially for information and software tracking.
9
10-----------------------------------------------------------
11Analysis program
12------------------------------------------------------------
13o analysis.cc
14 - perfom
15 . gain spectra determination + monitoring
[539]16 . ON-OFF spectra without calibration
[507]17 . calibration coefficient determination + monitoring
18 . ON-OFF spectra determination (no RT calibration included)
19 - Todo
[539]20 . include RT calibration, or a RT/BAO calibration using a calibrator source
[507]21 - comments
[539]22 . the STANDARD usage since end september 11 is GAIN + ON-OFF wo calibration
[507]23
24-----------------------------------------------------------
25Collection of Bourne SHELL script for analysis
26------------------------------------------------------------
27
28A) Utilitary scripts to follow the DAQ & Analysis progress
29++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[574]30o mkarchive.sh
31---------------
32 - perform the archive of a processing in Irods
33 . create a <ppTag>Arch<apTag> directory
34 . move the cabli_*.txt files
35 . move the *.ppf / *.fits files results of the analysis
36 . for the gain...fits only a copy is done to keep a version
37 . move the Off and On directorys
38 . create new fresh On and Off directory
39
40
[507]41o statusdaq.sh
42--------------
43 - perform
44 . scan all the /baoradio/data/AmamNancay to look at the directories and and make
45 a summary of the Data Acquisition
46
47o statusheader.sh
48-----------------
49 - perform
50 . scan Irods directories to look at header fits files extraction progress
51
52o statusgain.sh
53---------------
54 - perform
55 . scan Irods directories to look at FINAL gain_* files production progress
56
57o statuscalib.sh
58----------------
59 - perform
60 . scan Irods directories to look at INTERMEDIATE calibcycle files production progress
61 - Todo
62 . scan also for FINAL calib_* files progress
[539]63o statusonoff.sh
64----------------
65 - perform
66 . scan Irods On and Off directories to look at datacycle<cycle> contents
[507]67
[539]68o statusCycles.sh
69-----------------
70 - perform
71 . scan Irods directories to look at sca<>.sum files to count the number of cycles per run
72
[507]73B) Utilitary scripts to send to BQS system high level task
74++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
75o submit2bqs-hdrfitsextractor.sh
76--------------------------------
77 - SHOULD BE DONE BEFORE gain/calib/analysis
78 - send the hdrfitsextractor.sh script to BQS system
79 - Todo list
80 - TO MIGRATE TO NEW BATCH
81
82o submit2bqs-scaextractor.sh
83--------------------------------
84 - SHOULD BE DONE BEFORE gain/calib/analysis
85 - send the scaextractor.sh script to BQS system
86 although this script is called by getscafiles lower level script
87 - Todo list
88 - TO MIGRATE TO NEW BATCH
89
[573]90o submit2bqs-procspecmfib.sh
91-----------------------------
92 - SHOULD BE DONE for processing signals for FFT/paquet median filtering
93 - GAIN/CALIB/ON, OFF...
94 - prepare and send in batch the proc_specmfib.sh to BQS
95 - exemple
[575]96 ./submit2bqs-procspecmfib.sh -src Abell85 -date 20110422 -type CALIBOFF -fcycle 8 -lcycle 14
[507]97 - Todo list
[573]98 - TO MIGRATE TO NEW BATCH
99
100o submit2bqs-anagainmaker.sh
101----------------------------
102 - done after proc_specmfib
103 - prepare and send the analyse.cc prg for gain computation
104 - exemple
105 - Todo list
106 - TO MIGRATE TO NEW BATCH
107
108o submit2bqs-anacalibmaker.sh
109----------------------------
110 - done after proc_specmfib
111 - prepare and send the analyse.cc prg for calibration computation
112 - exemples:
113 > $./submit2bqs-anacalibmaker.sh -src Abell85 -date 20110410
114
[507]115
[573]116 - Todo list
117 - TO MIGRATE TO NEW BATCH
118
119o submit2qs-onoffmaker.sh (OBSOLETE ???)
120--------------------------
121 - send the onoffmaker.sh script to BQS system
122 - Todo list
123 - TO MIGRATE TO NEW BATCH
124
125o submit2bqs-rawonoffmaker.sh (OBSOLETE ???)
[539]126-----------------------------
[573]127 - used after the anagainmaker
[539]128 - prepare and send in batch the proc_rawonoff.sh to BQS system
129 - exemple submit2bqs-rawonoffmaker.sh -src Abell85 -date 20110417 -fcycle 8 -lcycle 23 -type ON
130 - Todo list
131 - TO MIGRATE TO NEW BATCH
132
[573]133o submit2qs-gainmaker.sh (OBSOLETE)
134-------------------------
135 - send the gainmaker.sh script to BQS system
[507]136 - Todo list
[573]137 - TO MIGRATE TO NEW BATCH
[507]138
[573]139o submit2qs-calibmaker.sh (OBSOLETE)
[519]140--------------------------
[573]141 - send the calibmaker.sh script to BQS system
[519]142 - Todo list
143 - TO MIGRATE TO NEW BATCH
144
[507]145
146C) High level scripts
147++++++++++++++++++++++
148o hdrfitsextractor.sh
149---------------------
150 - SHOULD BE DONE BEFORE ANYTHING
151 - perform
152 . extraction of the header part of the signalXYZ.fits files and
153 . put the results into Irods
154 - use
155 . statusdaq.sh
156
[573]157o anagainmaker.sh
158-----------------
159 - perform
160 . the analysis of the above fits files to determine the
161 gains of each channel: produce gain_* files (monitoring + gain files)
162 . put the results into Irods
163 - use
164 - getscafiles.sh
165 - analysis.cc
166
167o anacalibmaker.sh
168------------------
169 - perfom
170 . prepare the analysis of the above fits files
171 . put calib_* files (monitoring + calibration files) into Irods
172 - use
173 - getscafiles.sh
174 - analysis.cc
175
176
177o gainmaker.sh (OBOLETE)
[507]178---------------
179 - perform
180 1) the extraction of signalXYZ.fits files
181 concerned by the OFF-calibration time window
182 of the cycle located at the middle of the run
183 2) . the analysis of the above fits files to determine the
184 gains of each channel: produce gain_* files (monitoring + gain files)
185 . put the results into Irods
186 - use
187 - statusdaq.sh, proc_script.sh, getscafiles.sh
188 - analysis.cc
189
[573]190o calibmaker.sh (OBSOLETE)
[507]191---------------
192 - perfom
[519]193 1) several x-checks to see at which stage of the analysis process we are
194 2) trig the proc_script.sh script if necessary to create the spectra fits files via specmfib.cc prg
195 3) . prepare the analysis of the above fits files
196 . put calib_* files (monitoring + calibration files) into Irods
[507]197 - use
198 - statusdaq.sh, proc_script.sh, getscafiles.sh
199 - analysis.cc
200
[519]201o onoffmaker.sh (preliminary)
202----------------------------
203 - perform
204 1) several x-checks to see at which stage of the analysis process we are
205 2) trig the proc_script.sh script if necessary to create the spectra fits files via specmfib.cc prg
206 3) prepare the analysis of the above fits files
207 4) put dataRaw_* (raw spectra),
208 dataBAOCalib_* (after BAO calibration),
209 dataBAORTCalib_* (after BAO & RT calibration),
210 diffOnOff_* (ON-OFF spectra)
211
[507]212D) Lower level scripts
213++++++++++++++++++++++
[539]214
[573]215o proc_specmfib.sh
[539]216------------------
[573]217 - perform all the preparation work to send in batch the "specmfib"
218 program which perform the FFT and median/mean spectra
219 for the GAIN/CALIB/ON-OFF... task
220 - use
221 - getscafiles.sh, select.sh, statusdaq.sh, getsignalfiles.sh
222 - comments
223 - this is the pre-processing of the data to be used before
224 anaXYZ script which are post-processing
225
226
227o proc_rawonoff.sh (OBSOLETE)
228------------------
[539]229 - this is the script to use to perform the "datacycle" contents
230 - perform all the preparation work to send in batch the "specmfib" program
231 which perform the FFT and median/mean spectra for the ON-OFF task
232 - use
233 - getscafiles.sh, select.sh, statusdaq.sh, getsignalfiles.sh
234 - comments
235 - contrary to proc_script (below) specmfib is activated in the same thread
236
[573]237o proc_script.sh (OBSOLETE)
[507]238-----------------
239 - VERY BUSY script
240 - perform
241 1) all the preparation work to send in batch the "specmfib" program
242 which perform the FFT and median/mean spectra for GAIN, CALIBRATION, ON-OFF task
243 2) the script that is sent in batch put the results into Irods
244 - use
245 - getscafiles.sh, select.sh, getsignalfiles.sh
246 - Todo list
247 - TO MIGRATE TO NEW BATCH
248
249
250o scaextractor.sh
251-----------------
252 - perform
253 1) extraction of the scaXYZ file
254 2) if the scaXYZ.sum and the scaXYZ.sum.trans files do not exist
255 . create and put them into Irods
256 . scaXYZ.sum = summary of the SCA file
257 . scaXYZ.sum.trans = transformation of the date format of the scaXYZ.sum file
258 - use
259 - sca.sh, transdate.sh
260
261o sca.sh
262--------
263 - perform
264 . produce the scaXYZ.sum (= summary of the SCA file) from a scaXYZ file
265
266o transdate.sh
267--------------
268 - perform
269 . produce the scaXYZ.sum.trans (= transformation of the date format) from the scaXYZ.sum file
270
271o select.sh
272-----------
273 -perform
274 . the selection of the signalXYZ.fits files according to the type of task requested
275 as ON, OFF, CALIBON, CALIBOFF, GAIN
276
277o getscafiles.sh
278----------------
279 - perform
280 . download from Irods the scaXYZ, scaXYZ.sum and scaXYZ.sum.trans files
281 - use
282 scaextractor.sh
283
284o getsignals.sh
285---------------
286 - perform
287 . download from Irods the signalXYZ.fits files
288
289
290
291
Note: See TracBrowser for help on using the repository browser.