Changeset 519


Ignore:
Timestamp:
Sep 15, 2011, 2:13:45 PM (13 years ago)
Author:
campagne
Message:

give the status of ON-Off analysis (jec)

Location:
BAORadio/AmasNancay
Files:
1 added
3 edited

Legend:

Unmodified
Added
Removed

  • BAORadio/AmasNancay/analysis-scriptsAndprg-description.txt

    r507 r519  
    11Title: Status of the programs and scripts used for Amas@Nancay analysis at CCIN2P3
    2 Revision: 7/9/11
     2Revision: 14/9/11
    33Author: J.E Campagne 22/7/11
    44----------------------------
    … …  
    1919                . include RT calibration
    2020        - comments
    21                 . ON-OFF part only validate for a single Abell85 run and not in batch mode
     21                . ON-OFF part only validate for a single Abell85 run and not in batch mode (preliminary stage)
    2222                . GAIN part should be revalidated according to inPath argument
    2323               
    … …  
    8181                - TO MIGRATE TO NEW BATCH
    8282
     83o submit2qs-onoffmaker.sh
     84--------------------------
     85        - send the onoffmaker.sh script to BQS system
     86        - Todo list
     87                - TO MIGRATE TO NEW BATCH
     88
    8389               
    8490C) High level scripts
    … …  
    109115---------------
    110116        - perfom
    111                 1) the extraction of signalXYZ.fits files
    112                         concerned by the OFF & ON calibration time window of all cycles concerned
    113                 2) . the analysis of the above fits files to determine the
    114                         calibration constantes of each channel: produce calib_* files (monitoring +  calibration files)
    115                         . put the results into Irods
     117                1) several x-checks to see at which stage of the analysis process we are
     118                2) trig the proc_script.sh script if necessary to create the spectra fits files via specmfib.cc prg
     119                3) . prepare the analysis of the above fits files
     120                   . put calib_* files (monitoring +  calibration files) into Irods
    116121        - use
    117122                - statusdaq.sh, proc_script.sh, getscafiles.sh
    118123                - analysis.cc   
     124
     125o onoffmaker.sh (preliminary)
     126----------------------------
     127        - perform
     128                1) several x-checks to see at which stage of the analysis process we are
     129                2) trig the proc_script.sh script if necessary to create the spectra fits files via specmfib.cc prg
     130                3) prepare the analysis of the above fits files
     131                4) put dataRaw_*        (raw spectra),
     132                       dataBAOCalib_*   (after BAO calibration),
     133                       dataBAORTCalib_* (after BAO & RT calibration),
     134                       diffOnOff_*      (ON-OFF spectra)
    119135
    120136D) Lower level scripts

  • BAORadio/AmasNancay/onoffmaker.sh

    r518 r519  
    175175#
    176176  somethingTodo=1
    177   input4dataOff=`ils ${i}/Off 2>&1 | $GREP -i "datacycle" | $AWK 'BEGIN{FS="C- "}{print $2}' | $XARGS -i basename {} | $SORT -k1.11n` 
    178   input4dataOn=`ils ${i}/On 2>&1 | $GREP -i "datacycle" | $AWK 'BEGIN{FS="C- "}{print $2}' | $XARGS -i basename {} | $SORT -k1.11n` 
     177  input4dataOff=`ils ${i}/Off 2>&1 | $GREP -i "datacycle" | $AWK 'BEGIN{FS="C- "}{print $2}' | $XARGS -i basename {} | $SORT -k1.10n` 
     178  input4dataOn=`ils ${i}/On 2>&1 | $GREP -i "datacycle" | $AWK 'BEGIN{FS="C- "}{print $2}' | $XARGS -i basename {} | $SORT -k1.10n` 
    179179#
    180180  if [ "<${input4dataOff}>" != "<>" -a "<${input4dataOn}>" != "<>"  ]; then
    … …  
    267267      inFileDirectory="./${mode}"
    268268      $MKDIR -p ${inFileDirectory}
    269       listOfDataCycle=( `ils ${i}/${mode} 2>&1 | $GREP -i "datacycle"|$AWK 'BEGIN{FS="C- "}{print $2}' | $XARGS -i basename {} | $SORT -k1.11n` )
     269      listOfDataCycle=( `ils ${i}/${mode} 2>&1 | $GREP -i "datacycle"|$AWK 'BEGIN{FS="C- "}{print $2}' | $XARGS -i basename {} | $SORT -k1.10n` )
    270270      IFS='
    271271'
    … …  
    302302      $MKDIR -p ${inFileDirectory}
    303303
    304       listOfDataCycle=( `ils ${i}/${mode} 2>&1 | $GREP -i "datacycle" |$AWK 'BEGIN{FS="C- "}{print $2}' | $XARGS -i basename {} | $SORT -k1.11n` )
     304      listOfDataCycle=( `ils ${i}/${mode} 2>&1 | $GREP -i "datacycle" |$AWK 'BEGIN{FS="C- "}{print $2}' | $XARGS -i basename {} | $SORT -k1.10n` )
    305305      IFS='
    306306'
    … …  
    380380
    381381      scaFile=`$FIND . -name "sca*.sum.trans" | $XARGS -i basename {}`
     382
     383
     384#Get the calibration files
     385      OUT=./getCalib.$$
     386      iget -f -K -v ${i}/calib_*.txt ${aboveOnOffDir}> ${OUT} 2>&1
     387      igetStatus=`$GREP "^ERROR" ${OUT}`
     388      if [ "<$igetStatus>" != "<>" ]; then
     389          $ECHO "FATAL (${scriptName}): error while iget calib files:"
     390          $ECHO $igetStatus
     391          exit 1
     392      fi
     393      $RM -f ${OUT}
    382394     
    383395#prepare job submission

  • BAORadio/AmasNancay/proc_script.sh

    r518 r519  
    205205#for GAIN the reference cycle takes the middle of the run
    206206#
     207#nwinmean donne le nombre de fenetre tspwin utilise pour faire calculer la moyenne des medianes de chaque fenetre
     208#This is the nimber of paquets of the time window which change according to processing type
    207209$ECHO "DEBUG (${scriptName}): START switch on $typeofproc"
    208210#
    … …  
    212214      ingain=""
    213215      freqfilter="-freqfilter -"
     216      nwinmean="1"
    214217      tspwin="5120"
    215218      dirName="Off"
    … …  
    226229      ingain="-gain gain_${dateDAQ}_${srclower}.fits"
    227230      freqfilter="-freqfilter 16"
    228       tspwin="25000"
     231      nwinmean="5"
     232      tspwin="5120"
    229233      dirName="On"
    230234      cyclebasename="datacycle"
    … …  
    239243      ingain="-gain gain_${dateDAQ}_${srclower}.fits"
    240244      freqfilter="-freqfilter 16"
    241       tspwin="25000"
     245      nwinmean="5"
     246      tspwin="5120"
    242247      dirName="Off"
    243248      cyclebasename="datacycle"
    … …  
    252257      ingain="-gain gain_${dateDAQ}_${srclower}.fits"
    253258      freqfilter="-freqfilter 16"
     259      nwinmean="1"
    254260      tspwin="1024"
    255261      dirName="On"
    … …  
    265271      ingain="-gain gain_${dateDAQ}_${srclower}.fits"
    266272      freqfilter="-freqfilter 16"
     273      nwinmean="1"
    267274      tspwin="1024"
    268275      dirName="Off"
    … …  
    351358#force single channel
    352359forceSingle="-singlechan"
    353 #nwinmean donne le nombre de fenetre tspwin utilise pour faire calculer la moyenne des medianes de chaque fenetre
    354 #This is the nimber of paquets of the time window which change according to processing type
    355 nwinmean="1"
    356360#debug
    357361prtlevel="1"
    … …  
    390394# 25sec/signal_file * empirical_conversion_factor (=50)
    391395    cpu=`expr ${nFiles} \* 25 \* 50`
    392 #Virtual Memory (MB)
     396#Virtual Memory (MB) Nb: at CCIN2P3 the reasonable max is 1GB
    393397    virtualmem=1000
    394398#Job and batch names
    … …  
    403407#
    404408    $CAT > ${jobScriptName} <<EOF
    405 #!/bin/sh
     409#!/bin/sh -xvf
    406410#    jobBatchName=$jobBatchName
    407411#    jobScriptName=$jobScriptName
    … …  
    456460    outDir=\${curDir}/${dirName}/${cycle}
    457461    $MKDIR -p \${outDir}
     462
     463#
     464# where ami...
     465#
     466    $ECHO "We are here \`pwd\` to send the batch"
     467    $LS -lrth
     468    $FIND . -name "igetStatus.*" | $XARGS -i  $CAT {}
     469    $LS ./Fiber1 | $WC -l
     470    $LS ./Fiber2 | $WC -l
     471    $LS -lR \${outDir}
     472
    458473#Note: here act=gain means only that specmfib will use median-like algorithms (not only for gains...)
    459474    $prg -act gain $forceSingle $freqfilter  -prt $prtlevel -out \${outDir} -nmean $nwinmean $ingain $tmproc -tspwin ${tspwin},0,0 -in $infiles \${fibfile}
    460475
    461476    $ECHO    \${outDir}
    462     $LS -lrt \${outDir}/*
     477    $LS -lrt \${outDir}
    463478
    464479#save results to Irods (use option -f ONLY to force override)
Note: See TracChangeset for help on using the changeset viewer.