Title: Status of the programs and scripts used for Amas@Nancay analysis at CCIN2P3
Revision: 14/9/11
Author: J.E Campagne 22/7/11
----------------------------

Intro:
All these tools are located for the while in J.E Campagne private directory, so they are not
yet available to public usage and this note is essentially for information and software tracking.

-----------------------------------------------------------
Analysis program
------------------------------------------------------------
o analysis.cc
	- perfom
		. gain spectra determination + monitoring
		. calibration coefficient determination + monitoring
		. ON-OFF spectra determination (no RT calibration included)
	- Todo
		. include RT calibration
	- comments
		. ON-OFF part only validate for a single Abell85 run and not in batch mode (preliminary stage)
		. GAIN part should be revalidated according to inPath argument
		
-----------------------------------------------------------
Collection of  Bourne SHELL script for analysis
------------------------------------------------------------

A) Utilitary scripts to follow the DAQ & Analysis progress
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
o statusdaq.sh
--------------
	- perform
		. scan all the /baoradio/data/AmamNancay to look at the directories and and make
			a summary of the Data Acquisition

o statusheader.sh
-----------------
	- perform
		. scan Irods directories to look at header fits files extraction progress 
		
o statusgain.sh
---------------
	- perform
		. scan Irods directories to look at FINAL gain_* files production progress 

o statuscalib.sh
----------------
	- perform
		. scan Irods directories to look at INTERMEDIATE calibcycle files production progress
	- Todo
		. scan also for FINAL calib_* files progress

B) Utilitary scripts to send to BQS system high level task
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++		
o submit2bqs-hdrfitsextractor.sh
--------------------------------
        - SHOULD BE DONE BEFORE gain/calib/analysis
	- send the hdrfitsextractor.sh script to BQS system	
	- Todo list
		- TO MIGRATE TO NEW BATCH 

o submit2bqs-scaextractor.sh
--------------------------------
        - SHOULD BE DONE BEFORE gain/calib/analysis
	- send statusdaq if necessary
	- send the scaextractor.sh script to BQS system
		although this script is called by getscafiles lower level script
	- Todo list
		- TO MIGRATE TO NEW BATCH 

o submit2qs-gainmaker.sh
-------------------------
	- send the gainmaker.sh script to BQS system 
	- Todo list
		- TO MIGRATE TO NEW BATCH
	
o submit2qs-calibmaker.sh
--------------------------
	- send the calibmaker.sh script to BQS system
	- Todo list
		- TO MIGRATE TO NEW BATCH 

o submit2qs-onoffmaker.sh
--------------------------
	- send the onoffmaker.sh script to BQS system
	- Todo list
		- TO MIGRATE TO NEW BATCH 

		
C) High level scripts 
++++++++++++++++++++++		
o hdrfitsextractor.sh
---------------------
	- SHOULD BE DONE BEFORE ANYTHING
	- perform
		. extraction of the header part of the signalXYZ.fits files and
		. put the results into Irods	
	- use 
		. statusdaq.sh	

o gainmaker.sh
---------------
	- perform 
		1) the extraction of signalXYZ.fits files 
			concerned by the OFF-calibration time window
			of the cycle located at the middle of the run
		2) . the analysis of the above fits files to determine the 
			gains of each channel: produce gain_* files (monitoring + gain files)
			. put the results into Irods
	- use 
		- statusdaq.sh, proc_script.sh, getscafiles.sh
		- analysis.cc	
	
o calibmaker.sh
---------------
	- perfom
		1) several x-checks to see at which stage of the analysis process we are
		2) trig the proc_script.sh script if necessary to create the spectra fits files via specmfib.cc prg 
		3) . prepare the analysis of the above fits files
		   . put calib_* files (monitoring +  calibration files) into Irods
	- use 
		- statusdaq.sh, proc_script.sh, getscafiles.sh
		- analysis.cc	

o onoffmaker.sh (preliminary)
----------------------------
	- perform
		1) several x-checks to see at which stage of the analysis process we are
		2) trig the proc_script.sh script if necessary to create the spectra fits files via specmfib.cc prg 
		3) prepare the analysis of the above fits files
		4) put dataRaw_*        (raw spectra), 
		       dataBAOCalib_*   (after BAO calibration), 
		       dataBAORTCalib_* (after BAO & RT calibration), 
		       diffOnOff_*      (ON-OFF spectra) 

D) Lower level scripts
++++++++++++++++++++++
		
o proc_script.sh
-----------------
	- VERY BUSY script
	- perform 
		1) all the preparation work to send in batch the "specmfib" program 
			which perform the FFT and median/mean spectra for GAIN, CALIBRATION, ON-OFF task
		2) the script that is sent in batch put the results into Irods	
	- use
		- getscafiles.sh, select.sh, getsignalfiles.sh
	- Todo list
		- TO MIGRATE TO NEW BATCH 
		

o scaextractor.sh
-----------------
	- perform
		1) extraction of the scaXYZ file
		2) if the scaXYZ.sum and the scaXYZ.sum.trans files do not exist
			. create and put them into Irods
			. scaXYZ.sum = summary of the SCA file
			. scaXYZ.sum.trans = transformation of the date format of the scaXYZ.sum file	
	- use
		- sca.sh, transdate.sh
		
o sca.sh
--------
	- perform
		. produce the scaXYZ.sum (= summary of the SCA file) from a scaXYZ file
		
o transdate.sh
--------------
	- perform
		. produce the scaXYZ.sum.trans (= transformation of the date format) from the scaXYZ.sum file	

o select.sh
-----------
	-perform 
		. the selection of the signalXYZ.fits files according to the type of task requested
			as ON, OFF, CALIBON, CALIBOFF, GAIN
		
o getscafiles.sh
----------------
	- perform
		. download from Irods the scaXYZ, scaXYZ.sum and scaXYZ.sum.trans files	
	- use
		scaextractor.sh
		
o getsignals.sh
---------------
	- perform
		. download from Irods the signalXYZ.fits files