source: BAORadio/AmasNancay/trunk/analysis-scriptsAndprg-description.txt @ 574

Last change on this file since 574 was 574, checked in by campagne, 13 years ago

mkarchive.sh script description (jec)
File size: 8.7 KB
Line 
1Title: Status of the programs and scripts used for Amas@Nancay analysis at CCIN2P3
2Revision: 11/10/11 (JEC)
3Author: J.E Campagne 22/7/11
4----------------------------
5
6Intro:
7All these tools are located for the while in J.E Campagne private directory, so they are not
8yet available to public usage and this note is essentially for information and software tracking.
9
10-----------------------------------------------------------
11Analysis program
12------------------------------------------------------------
13o analysis.cc
14        - perfom
15                . gain spectra determination + monitoring
16                . ON-OFF spectra without calibration
17                . calibration coefficient determination + monitoring
18                . ON-OFF spectra determination (no RT calibration included)
19        - Todo
20                . include RT calibration, or a RT/BAO calibration using a calibrator source
21        - comments
22                . the STANDARD usage since end september 11 is GAIN + ON-OFF wo calibration
23               
24-----------------------------------------------------------
25Collection of  Bourne SHELL script for analysis
26------------------------------------------------------------
27
28A) Utilitary scripts to follow the DAQ & Analysis progress
29++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
30o mkarchive.sh
31---------------
32        - perform the archive of a processing in Irods
33          . create a <ppTag>Arch<apTag> directory
34            . move the cabli_*.txt files
35            . move the *.ppf / *.fits files results of the analysis
36            . for the gain...fits only a copy is done to keep a version
37            . move the Off and On directorys
38          . create new fresh On and Off directory
39
40
41o statusdaq.sh
42--------------
43        - perform
44                . scan all the /baoradio/data/AmamNancay to look at the directories and and make
45                        a summary of the Data Acquisition
46
47o statusheader.sh
48-----------------
49        - perform
50                . scan Irods directories to look at header fits files extraction progress
51               
52o statusgain.sh
53---------------
54        - perform
55                . scan Irods directories to look at FINAL gain_* files production progress
56
57o statuscalib.sh
58----------------
59        - perform
60                . scan Irods directories to look at INTERMEDIATE calibcycle files production progress
61        - Todo
62                . scan also for FINAL calib_* files progress
63o statusonoff.sh
64----------------
65        - perform
66                . scan Irods On and Off directories to look at datacycle<cycle> contents
67
68o statusCycles.sh
69-----------------
70        - perform
71                . scan Irods directories to look at sca<>.sum files to count the number of cycles per run
72
73B) Utilitary scripts to send to BQS system high level task
74++++++++++++++++++++++++++++++++++++++++++++++++++++++++++             
75o submit2bqs-hdrfitsextractor.sh
76--------------------------------
77        - SHOULD BE DONE BEFORE gain/calib/analysis
78        - send the hdrfitsextractor.sh script to BQS system     
79        - Todo list
80                - TO MIGRATE TO NEW BATCH
81
82o submit2bqs-scaextractor.sh
83--------------------------------
84        - SHOULD BE DONE BEFORE gain/calib/analysis
85        - send the scaextractor.sh script to BQS system
86                although this script is called by getscafiles lower level script
87        - Todo list
88                - TO MIGRATE TO NEW BATCH
89
90o submit2bqs-procspecmfib.sh
91-----------------------------
92        - SHOULD BE DONE for processing signals for FFT/paquet median filtering
93        - GAIN/CALIB/ON, OFF...
94        - prepare and send in batch the proc_specmfib.sh to BQS
95        - exemple
96        - Todo list
97                - TO MIGRATE TO NEW BATCH
98
99o submit2bqs-anagainmaker.sh
100----------------------------
101        - done after proc_specmfib
102        - prepare and send the analyse.cc prg for gain computation
103        - exemple
104        - Todo list
105                - TO MIGRATE TO NEW BATCH
106
107o submit2bqs-anacalibmaker.sh
108----------------------------
109        - done after proc_specmfib
110        - prepare and send the analyse.cc prg for calibration computation
111        - exemples:
112        > $./submit2bqs-anacalibmaker.sh -src Abell85 -date 20110410
113
114       
115        - Todo list
116                - TO MIGRATE TO NEW BATCH
117       
118o submit2qs-onoffmaker.sh (OBSOLETE ???)
119--------------------------
120        - send the onoffmaker.sh script to BQS system
121        - Todo list
122                - TO MIGRATE TO NEW BATCH
123
124o submit2bqs-rawonoffmaker.sh (OBSOLETE ???)
125-----------------------------
126        - used after the anagainmaker
127        - prepare and send in batch the proc_rawonoff.sh to BQS system
128        - exemple submit2bqs-rawonoffmaker.sh -src Abell85 -date 20110417 -fcycle 8 -lcycle 23 -type ON
129        - Todo list
130                - TO MIGRATE TO NEW BATCH
131
132o submit2qs-gainmaker.sh (OBSOLETE)
133-------------------------
134        - send the gainmaker.sh script to BQS system
135        - Todo list
136                - TO MIGRATE TO NEW BATCH
137
138o submit2qs-calibmaker.sh (OBSOLETE)
139--------------------------
140        - send the calibmaker.sh script to BQS system
141        - Todo list
142                - TO MIGRATE TO NEW BATCH
143
144               
145C) High level scripts
146++++++++++++++++++++++         
147o hdrfitsextractor.sh
148---------------------
149        - SHOULD BE DONE BEFORE ANYTHING
150        - perform
151                . extraction of the header part of the signalXYZ.fits files and
152                . put the results into Irods   
153        - use
154                . statusdaq.sh 
155
156o anagainmaker.sh
157-----------------
158        - perform
159          . the analysis of the above fits files to determine the
160                        gains of each channel: produce gain_* files (monitoring + gain files)
161          . put the results into Irods
162        - use
163                - getscafiles.sh
164                - analysis.cc   
165
166o anacalibmaker.sh
167------------------
168        - perfom
169          . prepare the analysis of the above fits files
170          . put calib_* files (monitoring +  calibration files) into Irods
171        - use
172                - getscafiles.sh
173                - analysis.cc   
174
175
176o gainmaker.sh (OBOLETE)
177---------------
178        - perform
179                1) the extraction of signalXYZ.fits files
180                        concerned by the OFF-calibration time window
181                        of the cycle located at the middle of the run
182                2) . the analysis of the above fits files to determine the
183                        gains of each channel: produce gain_* files (monitoring + gain files)
184                        . put the results into Irods
185        - use
186                - statusdaq.sh, proc_script.sh, getscafiles.sh
187                - analysis.cc   
188       
189o calibmaker.sh (OBSOLETE)
190---------------
191        - perfom
192                1) several x-checks to see at which stage of the analysis process we are
193                2) trig the proc_script.sh script if necessary to create the spectra fits files via specmfib.cc prg
194                3) . prepare the analysis of the above fits files
195                   . put calib_* files (monitoring +  calibration files) into Irods
196        - use
197                - statusdaq.sh, proc_script.sh, getscafiles.sh
198                - analysis.cc   
199
200o onoffmaker.sh (preliminary)
201----------------------------
202        - perform
203                1) several x-checks to see at which stage of the analysis process we are
204                2) trig the proc_script.sh script if necessary to create the spectra fits files via specmfib.cc prg
205                3) prepare the analysis of the above fits files
206                4) put dataRaw_*        (raw spectra),
207                       dataBAOCalib_*   (after BAO calibration),
208                       dataBAORTCalib_* (after BAO & RT calibration),
209                       diffOnOff_*      (ON-OFF spectra)
210
211D) Lower level scripts
212++++++++++++++++++++++
213
214o proc_specmfib.sh
215------------------
216        - perform all the preparation work to send in batch the "specmfib"
217          program which perform the FFT and median/mean spectra
218          for the GAIN/CALIB/ON-OFF... task
219        - use
220          - getscafiles.sh, select.sh, statusdaq.sh, getsignalfiles.sh
221        - comments
222          - this is the pre-processing of the data to be used before
223            anaXYZ script which are post-processing
224
225
226o proc_rawonoff.sh (OBSOLETE)
227------------------
228        - this is the script to use to perform the "datacycle" contents
229        - perform all the preparation work to send in batch the "specmfib" program
230                        which perform the FFT and median/mean spectra for the ON-OFF task
231        - use
232          - getscafiles.sh, select.sh, statusdaq.sh, getsignalfiles.sh
233        - comments
234          - contrary to proc_script (below) specmfib is activated in the same thread
235
236o proc_script.sh (OBSOLETE)
237-----------------
238        - VERY BUSY script
239        - perform
240                1) all the preparation work to send in batch the "specmfib" program
241                        which perform the FFT and median/mean spectra for GAIN, CALIBRATION, ON-OFF task
242                2) the script that is sent in batch put the results into Irods 
243        - use
244                - getscafiles.sh, select.sh, getsignalfiles.sh
245        - Todo list
246                - TO MIGRATE TO NEW BATCH
247               
248
249o scaextractor.sh
250-----------------
251        - perform
252                1) extraction of the scaXYZ file
253                2) if the scaXYZ.sum and the scaXYZ.sum.trans files do not exist
254                        . create and put them into Irods
255                        . scaXYZ.sum = summary of the SCA file
256                        . scaXYZ.sum.trans = transformation of the date format of the scaXYZ.sum file   
257        - use
258                - sca.sh, transdate.sh
259               
260o sca.sh
261--------
262        - perform
263                . produce the scaXYZ.sum (= summary of the SCA file) from a scaXYZ file
264               
265o transdate.sh
266--------------
267        - perform
268                . produce the scaXYZ.sum.trans (= transformation of the date format) from the scaXYZ.sum file   
269
270o select.sh
271-----------
272        -perform
273                . the selection of the signalXYZ.fits files according to the type of task requested
274                        as ON, OFF, CALIBON, CALIBOFF, GAIN
275               
276o getscafiles.sh
277----------------
278        - perform
279                . download from Irods the scaXYZ, scaXYZ.sum and scaXYZ.sum.trans files
280        - use
281                scaextractor.sh
282               
283o getsignals.sh
284---------------
285        - perform
286                . download from Irods the signalXYZ.fits files
287               
288               
289               
290
Note: See TracBrowser for help on using the repository browser.