Title: Status of the programs and scripts used for Amas@Nancay analysis at CCIN2P3 Revision: 28/11/11 (JEC+AST) Author: J.E Campagne 22/7/11 ---------------------------- Intro: All these tools are located for the while in J.E Campagne private directory, so they are not yet available to public usage and this note is essentially for information and software tracking. ----------------------------------------------------------- Analysis program ------------------------------------------------------------ o analysis.cc - perfom . gain spectra determination + monitoring . ON-OFF spectra without calibration . calibration coefficient determination + monitoring . ON-OFF spectra determination (no RT calibration included) - Todo . include RT calibration, or a RT/BAO calibration using a calibrator source - comments . the STANDARD usage since end september 11 is GAIN + ON-OFF wo calibration o etude_gain.pic - perform . plot gain spectra/monitoring from a run dataset o etude_gain_medfiltmtx.pic - perform . plot all the medfiltmtx*.fits from a specific calibcycle o etude_dab.pic - perform . extract baseline, dab1 and dab2 spectra o dabchrom_plot.pic - perform . compare dab spectra from our data vs. other calibrations o etude_mergeAna.pic - perform . compare results wo calibration vs. calibration per run/cycle . ON-OFF, (ON-OFF)/OFF_filt, ON/OFF, OFF/OFF in the whole range [1250,1500]MHz . ON/OFF, OFF/OFF in the range [1410,1415]MHz . evolution of ON-OFF integrated intensity at calibration frequency, 1420MHz and its side-bands, and in [1400-1420]MHz . evolution of ON-OFF integrated intensity of (1420MHz line - sidebands) . dependence of ON-OFF sensitivity with integration time at 1420MHz and its side-bands, and in [1400-1420]MHz o plotRawDiffOnOff.pic - perform . plot results wo calibration . (ON-OFF)/OFF_filt in the whole range [1250,1500]MHz . ON/OFF, OFF/OFF in the range [1410,1415]MHz . evolution of (ON-OFF)/OFF_filt integrated intensity at 1420MHz and its side-bands, and in [1400-1420]MHz . evolution of (ON-OFF)/OFF_filt integrated intensity of (1420MHz line - sidebands) . dependence of ON-OFF sensitivity with integration time at 1420MHz side-bands, and in [1410-1415]MHz ----------------------------------------------------------- Collection of Bourne SHELL script for analysis ------------------------------------------------------------ A) Utilitary scripts to follow the DAQ & Analysis progress ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ o mkarchive.sh --------------- - perform the archive of a processing in Irods . create a Arch directory . move the calib_*.txt files . move the *.ppf / *.fits files results of the analysis . for the gain...fits only a copy is done to keep a version . move the Off and On directorys . create new fresh On and Off directory o statusdaq.sh -------------- - perform . scan all the /baoradio/data/AmasNancay to look at the directories and and make a summary of the Data Acquisition o statusheader.sh ----------------- - perform . scan Irods directories to look at header fits files extraction progress o statusgain.sh --------------- - perform . scan Irods directories to look at FINAL gain_* files production progress o statuscalib.sh ---------------- - perform . scan Irods directories to look at INTERMEDIATE calibcycle files production progress - Todo . scan also for FINAL calib_* files progress o statusonoff.sh ---------------- - perform . scan Irods On and Off directories to look at datacycle contents o statusCycles.sh ----------------- - perform . scan Irods directories to look at sca<>.sum files to count the number of cycles per run B) Utilitary scripts to send to GE system high level task (note old BQS batch is obsolete now) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ o submit2ge-hdrfitsextractor.sh -------------------------------- - SHOULD BE DONE BEFORE gain/calib/analysis - send the hdrfitsextractor.sh script to GE system o submit2ge-procspecmfib.sh ----------------------------- - SHOULD BE DONE for processing signals for FFT/paquet median filtering - GAIN/CALIB/ON, OFF... - prepare and send in batch the proc_specmfib.sh to GE - exemple ./submit2ge-procspecmfib.sh -src Abell85 -date 20110422 -type CALIBOFF -fcycle 8 -lcycle 14 o submit2ge-anagainmaker.sh ---------------------------- - done after proc_specmfib - prepare and send the analyse.cc prg for gain computation - exemple o submit2ge-anacalibmaker.sh ---------------------------- - done after proc_specmfib - prepare and send the analyse.cc prg for calibration computation - exemples: > $./submit2ge-anacalibmaker.sh -src Abell85 -date 20110410 o submit2ge-scaextractor.sh -------------------------------- - SHOULD BE DONE BEFORE gain/calib/analysis - send the scaextractor.sh script to GE system although this script is called by getscafiles lower level script C) High level scripts ++++++++++++++++++++++ o hdrfitsextractor.sh --------------------- - SHOULD BE DONE BEFORE ANYTHING - perform . extraction of the header part of the signalXYZ.fits files and . put the results into Irods - use . statusdaq.sh o anagainmaker.sh ----------------- - perform . the analysis to determine the gains of each channel: produce gain_* files (monitoring + gain files) . put the results into Irods - use - getscafiles.sh - analysis.cc o anacalibmaker.sh ------------------ - perfom . prepare the analysis . put calib_* files (monitoring + calibration files) into Irods - use - getscafiles.sh - analysis.cc o anarawonoffmaker.sh --------------------- - perform . prepare the analysis . put dataRaw_* (raw spectra), diffOnOffRaw_* (ON-OFF spectra although not used by final analysis) D) Lower level scripts ++++++++++++++++++++++ o proc_specmfib.sh ------------------ - perform all the preparation work to send in batch the "specmfib" program which perform the FFT and median/mean spectra for the GAIN/CALIB/ON-OFF... task - use - getscafiles.sh, select.sh, statusdaq.sh, getsignalfiles.sh - comments - this is the pre-processing of the data to be used before anaXYZ script which are post-processing o scaextractor.sh ----------------- - perform 1) extraction of the scaXYZ file 2) if the scaXYZ.sum and the scaXYZ.sum.trans files do not exist . create and put them into Irods . scaXYZ.sum = summary of the SCA file . scaXYZ.sum.trans = transformation of the date format of the scaXYZ.sum file - use - sca.sh, transdate.sh o sca.sh -------- - perform . produce the scaXYZ.sum (= summary of the SCA file) from a scaXYZ file o transdate.sh -------------- - perform . produce the scaXYZ.sum.trans (= transformation of the date format) from the scaXYZ.sum file o select.sh ----------- -perform . the selection of the signalXYZ.fits files according to the type of task requested as ON, OFF, CALIBON, CALIBOFF, GAIN o getscafiles.sh ---------------- - perform . download from Irods the scaXYZ, scaXYZ.sum and scaXYZ.sum.trans files - use scaextractor.sh o getsignals.sh --------------- - perform . download from Irods the signalXYZ.fits files