source: Sophya/trunk/DocHFI_L2/wbsl2.tex@ 1168

\documentclass[twoside,11pt]{article}

%  Definition pour Documentation L2
\usepackage{defdocl2}

%  Package standard : Utilisation de caracteres accentues, mode francais et graphique
% \usepackage[latin1]{inputenc}
% \usepackage[T1]{fontenc}
% \usepackage{babel}[english]
% \usepackage{graphicx}
% package a mettre pour faire du pdf
% \usepackage{palatino}


%  Extension de symboles mathematiques
% \usepackage{amssymb}
\usepackage{amsmath}


%  Definition de taille de page (c'est ds defdocl2.sty)
\newcommand{\bul}{$\bullet \ $}
\newcommand{\gph}[1]{ {\bf GPH#1} }
\newcommand{\ngph}[1]{ GPH#1 }

%  Symboles en bold 
\newcommand{\bnu}{\boldsymbol{\nu}}
\newcommand{\bbold}{\boldsymbol{b}}
\newcommand{\dbold}{\boldsymbol{d}}

\begin{document}

\begin{titlepage}
\titleph{
Planck HFI L2 \\ 
Work Breakdown Structure \\
Package Breakdown Structure
}
%  Authors list
\authors{
R. Ansari & LAL, Orsay \\ % & {\tt ansari@lal.in2p3.fr} \\
I.J. Grivell  & ICSTM, London \\ 
G. Le Meur & LAL, Orsay \\
R.G. Mann  &  IfA, Edinburgh \\ 
M. Rowan-Robinson  & ICSTM, London \\ 
S. Serjeant  & ICSTM \\ 
J.F. Sygnet & IAS, Orsay \\ % {\tt Jean-Francois.Sygnet@ias.fr} \\
}

\titlepfd{PL-HFI-LAL-PW-L2002}{1.2}

\end{titlepage}

%   -------- Revision history
\revisionhistory
\begin{itemize}
\item Version 1.0 (February 2000 - CVS version 1.1) \\
Draft version presented at the HFI consortium meeting 
\item Version 1.1 (April 2000 - CVS version 1.2) \\
Draft updated using the document prepared at ISCTM and IfA (Edinburgh)
\item Version 1.2 (August 2000 - CVS version 1.6) \\
Inclusion of a section describing notation and conventions, 
flow charts. A more detailed description of deliverables is also included.
\end{itemize}

%   -------- Table of content
\newpage
\tableofcontents
\newpage

\section{Introduction}
This document contains a preliminary version a the 
Work Breakdown Structure (WBS), Package Breakdown Structure 
(PBS) for Planck-HFI Level 2 data processing.

The Level 2 activity of the HFI Data Processing Centre (DPC) 
is divided between the Paris-Orsay-Saclay Data Analysis Centre 
(POSDAC) and the London Planck Analysis Centre (LPAC).
POSDAC will run the Level 2 data processing pipeline and have overall 
responsibility for the collaborative work involved in its 
design and implementation. LPAC will a play major role for in the 
activities that iterate and overlap between Level 2 
and Level 3 (for which the Cambridge Planck Analysis Centre, 
CPAC, plays the role analogous to POSDAC at Level 2).
IPAC (at Caltech) and Canada (?) are the two other major
laboratories involved in the development of HFI level 2
software.

\section{HFI L2 processing software packages}
The Level 2 workpackages are listed below, with a numbering 
scheme derived from that in the 
{\it HFI Science Implementation Plan (SIP: draft of 19/11/99) }.
All packages should be compliant with the general software architecture and use the services of the corresponding support modules (\gph{428}).

\begin{itemize}
\item[]\gph{421}  RTA/QLA : Real Time Assesment, Quick Look Analysis.
\item[]\gph{422}  TOI processing
\item[]\gph{423}  Polarised TOI processing
\item[]\gph{424}  2-D map reconstruction from TOI's
\item[]\gph{425}  2-D polarisation map reconstruction from TOI's
\item[]\gph{426}  Optimal 2-D map reconstruction and analysis
\item[]\gph{427}  Early Compact Source Catalogue
\item[]\gph{428}  General software architecture and support modules
\item[]\gph{429}  Integration and Validation
\end{itemize}

The package \gph{424} should provide a resonably fast and simple 
method for reconstructing 2-D maps from cleaned TOI's (from 
\gph{422} and \gph{423}), removing systematic effects (destriping) 
and deconvolving the antenna beam pattern. An iterative method 
is foreseen, capable of dealing with partial knowledge of 
instrument caracteristics. \gph{425} would contain the 
complementary functionalities needed for reconstructing 
polarisation maps, using data from polarised channels. 
The optimal 2-D map reconstruction packages (\gph{426})
will implement the best feasible method for analysing the whole
data set from the HFI instrument and building the full sky frequency 
maps.

\par
In addition to the specific Level 2 workpackages, the following items
are also related to the L2 data processing:

\begin{itemize}
\item[]\gph{320} infrastructure procurement and maintenance (POSDAC)
\item[]\gph{360} infrastructure procurement and maintenance (LPAC)
\item[]\gph{140} Project management (POSDAC)
\item[]\gph{180} Project management (LPAC)
\end{itemize}

The table below shows a very preliminary cost estimate in
FTE-MY (Full Time Equivalent Man Years) for each package. \\

\begin{table}
\begin{center}
\begin{tabular}[h]{|c|c||c|c|}
\hline
\makebox[2.5cm][c]{Package} &  \makebox[2.5cm][c]{Cost} &  
\makebox[2.5cm][c]{Package} &  \makebox[2.5cm][c]{Cost} \\
\hline
%               &           &           &       \\
\ngph{421}     &    5    &     \ngph{426}     &   ?     \\
%
\ngph{422}     &   15    &     \ngph{427}     &   10     \\
%
\ngph{423}     &    5    &     \ngph{428}     &   30    \\
%
\ngph{424}     &   15    &     \ngph{429}     &   10    \\
%
\ngph{425}     &   15    &                    &         \\
%               &           &           &       \\
%
\hline 
\ngph{320}     &   3     & \ngph{140}     &   4     \\
\ngph{360}     &   2     & \ngph{180}     &   1     \\
\hline \hline
\multicolumn{2}{c}{\bf Total}          &   \multicolumn{2}{c}{\bf 115 + ?} \\
\hline 
\end{tabular}
\end{center}
\caption{Preliminary estimate of staff effort 
(in Full Time Equivalent Man-Years FTE-MY)}
\end{table}

\section{HFI data processing steps}
The major steps for processing Planck-HFI data are briefly described here.
For the sake of clarity, we define here the notation and conventions used
throughout this document for referring to the different data components 
in planck HFI. Most of these data components are also defined in 
\cite{datamodel}.

\subsection{Notation and conventions}
\begin{itemize}
\item[] {\bf Coordinates:} The spacecraft attitude is defined by the spin 
axis $\overrightarrow{\Omega}$ direction $(\theta, \phi)$ and the 
roll angle $\psi$. The beam direction and orientation can be computed from 
the spacecraft attitude ($\overrightarrow{\Omega}, \psi$ using
the focal plane geometry.
The spin velocity would be denoted $\omega_z$ or $\omega_{\psi}$. \\
A given sky direction is denoted $(\alpha,\delta)$ in equatorial coordinates, 
$(l,b)$ in Galactic coordinates or $(\lambda,\beta)$ in ecliptic coordinate.

\item[] {\bf Detector:} A given HFI bolometer read-out channel. The corresponding
signal is denoted $\dbold(t)$ or $\bbold(t)$ $\dbold^j(t)$ or $\bbold^j(t)$ 
where $j$ is the bolometer number.

\item[] {\bf Frequency channel:} The set of detectors working in the 
same frequency range. This information is only obtained by combining 
information from several detectors in the corresponding frequency band.
The subscript $\bnu$ is used to denote a frequency channel. 

\item[] {\bf Scan:} The time interval corresponding to a given nominal 
spin axis direction

\item[] {\bf Sky map:} The reconstructed signal in a given pixelisation 
scheme from a single or a set of detectors. The sky map in a given 
frequency channel is denoted $\boldsymbol{T}^{\bnu}(\theta,\phi)$.
$\boldsymbol{I}^{\bnu}$,$\boldsymbol{Q}^{\bnu}$,$\boldsymbol{U}^{\bnu}$ are used 
for stokes parameters defining the polarisation state. 
\end{itemize}


\subsection{Processing steps}

The level 2 uses the instrument and spacecraft data obtained 
from level 1 {\bf DS1} and the instrument calibration data
base {\bf DSCal}. calibration data base contains the 
focal plane geometry, detector response and readout 
characteristics, beam patterns, filters and polariser
response. The main processing steps for HFI-L2 are
briefly described here:

\begin{itemize}
\item {\bf L1:} Unpacking the telemetry packets, 
uncompressing the science data, time ordering the data and 
archiving the data in a format usable 
by the L2 pipeline is the responsability of Level 1. Detailed 
description of L1 software and processing steps is beyond the scope
of this document. {\bf DS0} denotes the complete data set used
as input by level 1. {\bf DS0} contains the instrument 
science and House-keeping (HSK) data telemetry packets, as well
as the auxiliary data (orbit and attitude).
{\bf DS1} is the processed data from level 1 containing the TOI's
(Time Ordered Information). 
\begin{center}
\framebox{
$ \boldmath{L1} : \text{DS0} \longrightarrow \text{DS1} $
}  
\end{center}
{\bf DS1} data set contains three main components, with different 
sampling frequency for each component
\begin{itemize}
\item Raw Time Ordered bolometer signals : $b_{raw}^j(t1)$
\item Time Ordered House-keeping data : $HSK(t2)$
\item Associated orbit and attitude data : Space craft
position $ \boldmath{P}_{xyz}(t3) $, spin axis direction 
$\overrightarrow{\Omega}(t3) - (\theta, \phi)(t3)$ and the
roll angle $\psi(t3)$. 
\end{itemize}

\item {\bf L2-2 , L2-3 :} Cleaning and calibrating the TOI's for
non polarised (L2-2) and polarised (L2-3) channels. 
These processings correspond to \gph{422} and \gph{423} 
packages. 
\begin{center}
\framebox{
$ \boldmath{L2-2} : \text{DS1} \oplus \text{DSCal} \longrightarrow \text{DS2-2} $ 
} (non polarised channels) \\
\framebox{
$ \boldmath{L2-3} : \text{DS1} \oplus \text{DSCal} \longrightarrow \text{DS2-3} $ 
} (polarised channels)
\end{center}
In addition to cleaned and calibrated bolometer signals 
{\bf DS2-2} and {\bf DS2-3} contain data quality flag, 
the parameters necessary to compute the reconstructed spacecraft 
orbit and attitude for each time sample, as well as each 
detector beam pattern direction and orientation.
\begin{itemize}
\item Cleaned and calibrated Time Ordered bolometer signals : $\bbold_{c}^j(t_1)$
\item Data quality flag : $qf_{c}^j(t_1)$
\item Reconstructed astrometric calibration parameters \\
The pointing direction and beam orientation $(\alpha , \delta)^j(t1) \gamma(t1)$
for each detector at any given time can be computed using these parameters.
\item Detector response and photometric calibration parameters
\item Low frequency drifts and noise statistics
\item Reconstructed lobe
\item Size reduced TOI's through appropriate processings such as phase
binning
\item Time Ordered House-keeping data : $HSK(t_2)$
\item Associated orbit and attitude data : 
$\{ \boldmath{P}_{xyz}(t3) , \overrightarrow{\Omega}(t3) , \psi(t3) \} $
\end{itemize}

\item {\bf L2-4 , L2-5 :} Reconstructing 2-D sky maps using 
cleaned, calibrated TOI's. The redundancies provided by 
the scan strategy the sky and the set of detectors in 
a given frequency channel are used in these steps for assessing 
and removing the systematic and instrumental effects (low frequency drifts, 
noise, main lobe and far side lobe effects, \ldots).
An iterative, sub-optimal processing method is foreseen for
L2-4 and L2-5 (\gph424 and \gph425).
\begin{center}
\framebox{
$ \boldmath{L2-2} : \text{DS2-2} \oplus \text{DSCal} \longrightarrow \text{DS2-4} $ 
} \\
\framebox{
$ \boldmath{L2-5} : \text{DS2-3} \oplus \text{DSCal} \longrightarrow \text{DS2-5} $ 
} 
\end{center}
Reprocessed TOI's and full sky temperature and polarisation state 
maps are the main outputs of {\bf L2-4} and {\bf L2-5}:
\begin{itemize}
\item Reprocessed, calibrated Time Ordered bolometer signals : $\bbold_{rc}^j(t_1)$
\item Full sky maps , temperature and polarisation maps (\gph425) 
$\boldsymbol{I}\bnu (\alpha,\delta)$, $\boldsymbol{Q}\bnu (\alpha,\delta)$,
$\boldsymbol{U}\bnu (\alpha,\delta)$. 
\end{itemize}

\item {\bf L2-6} More complex methods for 2-D map reconstruction and 
data analysis are performed at the next step (\bf L2-6 , \gph426)
\begin{center}
\framebox{
$ \boldmath{L2-6} : \text{DS2-2} \oplus \text{DS2-3} \oplus \text{DSCal} \longrightarrow \text{DS2-6} $ 
}
\end{center}

\item {\bf L2-7} The Early Compact Source Catalogue (ECSC) is an 
important data product from HFI Level 2 processing. 
\begin{center}
\framebox{
$ \boldmath{L2-7} : \text{DS2-2} \oplus \text{DS2-3} \oplus \text{DSCal} \longrightarrow \text{DS2-7} $ (ECSC) 
}
\end{center}

The \bf{DS2-2} and  \bf{DS2-3} are used to produce the source catalogue  
containing the source positions, brightness and local 
sky maps in each frequency bands \bf{DS2-7} :
\begin{itemize}
\item Source catalogue: $\{ (\alpha,\delta)_i , F_i^{\bnu 1} , 
F_i^{\bnu 2} , \ldots \}$
\item local sky maps: $\{ lm_i(x,y)^{\bnu 1}, lm_i(x,y)^{\bnu 2}, ldots \}$
\end{itemize}

\end{itemize}
 
\section{GPH421: RTA/QLA Real Time Assesment, Quick Look Analysis}
\begin{itemize}
\item[]\gph{421.1} Quick look analysis
\item[]\gph{421.2} Evaluation of instrument health
\item[]\gph{421.3} Exchange of information with LFI  
of mission/spacecraft/payload-dependent effects common 
to both instruments
\end{itemize}


\section{GPH422: TOI  processing}

This package contains all the processing which has to be 
perfomed on TOI's (Time Ordered Information ) for either a 
single-detector or multpile-detector, single-frequency set 
of TOI's. A very rough sketch of data flow and processing 
steps for GPH422 is given in figure \ref{fig422}.

\begin{figure}[htbp]
\mbox{\hspace*{1cm} \includegraphics[width=7cm]{gph422_1.eps}
      \includegraphics[width=7cm]{gph422_2.eps} }
\caption{Data flow and processing steps for GPH422}
\label{fig422}
\end{figure}

\begin{itemize}

\item[]\gph{422.1}  TOI statistical characterisation 
\begin{itemize}
\item[]422.1.1 Raw time-line statistical checks
\item[]422.1.2 Evaluation of noise components in TOI \\
(including use of redundancy to produce independent coadds)
\item[]422.1.3 Low frequency drifts
\item[]422.1.4 Correlation of TOI with housekeeping data and various 
coordinate systems to look for systematic trends
\item[]422.1.5 correlation of TOI between channels to identify 
cross-talk
\end{itemize}

\item[]\gph{422.2}  TOI filters
\begin{itemize}
\item[]422.2.1 Time domain filtering
\item[]422.2.2 Fourier filtering
\end{itemize}

\item[]\gph{422.3}  Deglitching and source extraction
\begin{itemize}
\item[]422.3.1 production of deglitched TOI, flagged to indicate cosmic
ray hits etc.
\item[]422.3.2 Point source detectors
\item[]422.3.3 Galaxy crossing detector
\item[]422.3.4 CMB dipole finder 
\end{itemize}

\item[]\gph{422.4}  Astrometric calibration \\
Astrometric calibration using known point sources,
bright sources from the ECSC and galaxy crossing in TOI.

\item[]\gph{422.5}  Photometric calibration 
\begin{itemize}
\item[]422.5.1 extended-source photometric calibration using known 
sources (Galaxy) and COBE/FIRAS data
\item[]422.5.2 point-source photometric calibration using known sources
\item[]422.5.3 reconstruction of beam pattern seen by each detector 
from observations of bright 
\end{itemize}

\item[]\gph{422.6} reconstruction of beam pattern seen by each 
detector from observations of bright point sources \gph{422.3.2}
and galaxy crossing \gph{422.3.3}

\item[]\gph{422.7}  Production of single pointing co-added rings 
or 
 
\item[]\gph{422.9}  Deliverables: Cleaned, calibrated TOI's : 
\begin{itemize} 
\item $(\alpha, \delta)$, roll angle, calibrated signal  
\item Noise statistics
\item Offset/drifts
\item Co-added rings
\end{itemize}

\end{itemize}

\section{GPH423: Polarised TOI processing}

\gph{423.1} Additional TOI processing, as required to 
reconstruct Stokes I,Q,U parameters from detector signals 
in polarised channels.

\section{GPH424: 2-D map reconstruction}

This package contains all the necessary modules for 
single frequency 2-D map reconstruction using the cleaned
TOI's from \gph{422} and \gph{423}. However \gph{424}
should only implement simple versions of 2-D map making 
and systematic removal algorithms, as opposed to 
\gph{424} which must provide a global, optimal 2-D map 
reconstruction with systematic effects assessment.

\begin{itemize} 
\item[]\gph{424.1} 1D $\rightarrow$ 2D Projector
\item[]\gph{424.2} Estimation and subtraction of systematic effects  
\begin{itemize}
\item[]424.2.1 Low frequency noise estimation
\item[]424.2.2 far side determination using moon/earth during  
commissioning phases.
\item[]424.2.3 far side lobe subtraction
\item[]424.2.4 use of redundancy to produce independent maps for 
assessment of noise contributions
\end{itemize}

\item[]\gph{424.3} Main lobe 
\begin{itemize}
\item[]424.3.1 supplementing source extraction from TOI with 
use of sources selected from frequency maps
\item[]424.3.2 reconstruction of beam pattern from maps using 
known point and bright ECSC sources.
\end{itemize}


\item[]\gph{424.4} Astrometric calibration using known and 
bright ECSC sources.
\item[]\gph{424.5} Photometric calibration
\item[]\gph{424.6} Postage stamp maps for compact sources

\item[]\gph{424.9} Deliverables: Full sky maps \\
\begin{itemize}
\item Sky maps per frequency channels $map_\nu (\alpha, \delta) $
\item Reprocessed TOI's 
\end{itemize} 

\end{itemize} 

\section{GPH425: 2-D polarisation map reconstruction}
The modified version of \gph424 software modules, suitable
for processing polarised TOI's for producing polarisation 
sky maps are grouped in this package.
Compared with \gph424 , 
\begin{itemize}
\item[]\gph{425.9} Deliverables: Full sky polarisation maps \\
\begin{itemize}
\item Sky maps per frequency channels $map_\nu (\alpha, \delta) $
\item Reprocessed TOI's 
\end{itemize}
 
\end{itemize} 

\section{GPH426: Optimal 2-D map reconstruction and analysis}

Implementation of an optimal 2-D full sky map reconstruction,
with systematics (low frequency drifts, side lobes, \ldots)
estimation and removal. This package definition and 
breakdown structure can only be refined at a later stage,
after preliminary studies of applicable methods.

\begin{itemize}
\item[]\gph{425.1} Optimal 2-D map reconstruction and analysis
for non polarised channels
\item[]\gph{425.2} Optimal 2-D map reconstruction and analysis
for polarised channels

\item[]\gph{425.9} Deliverables: Full sky maps \\
\begin{itemize}
\item Full sky maps for each frequency channels $map_\nu (\alpha, \delta) $
\item Reprocessed, cleanead TOI's
\item Beam patterns per detector and per frequency channel
\item Estimation of systematics 
\end{itemize} 

\end{itemize} 

\section{GPH426: Global polarised 2-D map reconstruction}

This module, a extended version of \gph{425} has to take 
into account specific constraints related to polarised TOI's
and sky map reconstruction

\begin{itemize}

\item[]\gph{426.9} Deliverables: full sky maps for each 
polarisation component (I,Q,U) \\
\begin{itemize}
\item Full sky maps for each frequency channels $map_\nu (\alpha, \delta) $
for each component (I,Q,U)
\item Reprocessed, cleanead TOI's for I,Q,U
\item Noise and systematic effect maps and characteristics 
\end{itemize} 

\end{itemize} 


\section{GPH427: Early Compact Source Catalogue} 

\begin{itemize}

\item[]\gph{427.9} Deliverables: Source catalogue \\
\begin{itemize}
\item Astrometrically calibrated source positions and 
photometrically calibrated flux in each frequency band \\
$$(\alpha, \delta)_i , F_i^\nu, i=1,..,N$$
\item Llocal sky maps in each frequency bands for areas 
around all source positions.
\item Related software and documentation
\end{itemize} 

\end{itemize} 

\section{GPH428: General software architecture and support modules}

This package defines the general architecture for the level 2 processing
software and insures the compliance with IDIS framework.
It provides also the support modules for the different 
level 2 packages (\gph{42x}) and their interfaces with IDIS 
components.

\begin{itemize}

\item[]\gph{428.1} Definition and construction of the software architecture

\item[]\gph{428.2} Development and maintenance of TOI manipulation module
(including diagnostics and display)

\item[]\gph{428.3} Development and maintenance of maps manipulation module
(including diagnostics and display of both full and local sky maps)

\item[]\gph{428.4} Instrument and satellite representation 
\begin{itemize}
\item[]428.4.1 Focal plane geometry
\item[]428.4.2 Detector's frequency response
\item[]428.4.3 Main and secondary lobes
\item[]428.4.4 Sampling and noise characteristics
\item[]428.4.5 Orbit and attitude
\item[]428.4.6 Thermal model
\end{itemize} 

\item[]\gph{428.5} Development/supply and maintenence 
of standard numerical analysis tools
\begin{itemize}
\item[]428.5.1 Arrays (Matrix, Vectors, \ldots) and linear algebra
\item[]428.5.2 Fourier analysis, Wavelet analysis
\item[]428.5.3 Minimisation, optimisation
\item[]428.5.4 Statistical analysis
\end{itemize} 

\item[]\gph{428.6} Interface with DMC

\item[]\gph{428.7} Interface with the process coordinator

\end{itemize} 

\section{GPH429: Integration and Validation}

The assembly of the different L2 processing modules and
integration in the pipeline, as well as the 
assessment of the overall quality of the delivered 
data products is the responsibility of \gph{429}.
The data quality validation activity includes iterative 
work using Level 2 and 3 data products, and comparison of 
HFI and LFI data for the common 100 GHz channel:
for example, activity in this workpackage may influence the details 
of how the Global polarised 2D map reconstruction of \gph{426} 
is performed, leading to the changes in the deliverables 
from that workpackage.
\begin{itemize}
\item[]\gph{429.1} Pipeline management
\begin{itemize}
\item[]429.1.1 management and construction of pipeline 
modules and their interfaces 
\item[]429.1.2 IDIS and DMC interface 
\item[]429.1.2 Management of pipeline operations
\end{itemize}
\item[]\gph{429.2} Iterative study of systematic effects 
(e.g. map striping, etc) involving both level 2 and level 3 products
\item[]\gph{429.3} Combination, comparison and validation 
of source catalogues derived from Levels 2 and 3 source extraction
\item[]\gph{429.4} Comparison of TOI data and maps between 
HFI and LFI for common 100GHz channel
\item[]\gph{429.9} Deliverables: Full sky maps for each 
polarisation component (I,Q,U) \\
\begin{itemize}
\item Sky maps per frequency channels $map_\nu (\alpha, \delta) $
for each component (I,Q,U)
\item Reprocessed, cleanead TOI's for I,Q,U
\item Noise and systematic effect maps and characteristics 
\item Photometric and Astrometric calibration
\item Associated software and documentation
\end{itemize} 

\end{itemize} 

\begin{thebibliography}{10}
\bibitem{datamodel}
\newblock {\bf PL-COM-OAT-TN-003}
\newblock {Planck IDIS Data Model Description Document}
\end{thebibliography}


\end{document}